Coding & Development

BERT-flow offers a TensorFlow implementation of the research paper "On the Sentence Embeddings from Pre-trained Language Models" (EMNLP 2020). This tool is designed for researchers and developers working with natural language processing, specifically focusing on enhancing the quality of sentence embeddings derived from pre-trained BERT models. It provides scripts and configurations for fine-tuning BERT with NLI supervision and for unsupervised learning of flow-based generative models. The repository includes detailed instructions for setting up the environment, downloading pre-trained BERT models and GLUE data, and running experiments for both fine-tuning and flow-based model training and evaluation. BERT-flow is a valuable resource for academic research and experimentation in the field of sentence representation.

Building-llm-applications-from-scratch offers an open-sourced course designed for professionals to master the development of Large Language Model (LLM) applications. Unlike many courses that rely on pre-built frameworks, this program delves into the foundational building blocks of retrieval systems, empowering participants to design, build, and deploy custom LLM-powered solutions from scratch. The curriculum covers essential topics such as Transformer Architecture, Retrieval-Augmented Generation (RAG), and open-source LLM deployment. It includes 29 in-depth lessons, 6 real-world projects, interactive live sessions, and direct instructor access, culminating in a certificate upon completion. The course is ideal for those with existing Python and basic machine learning knowledge.

The Bamberg Center for Artificial Intelligence (BaCAI) is a research institution dedicated to advancing open AI research with national and international visibility. Its core mission involves the responsible translation of AI algorithms into practical applications, with a strong emphasis on developing human-centric AI systems. BaCAI fosters interdisciplinary cooperation to achieve its goals, aiming to become a central hub for AI expertise and talent. The center's work contributes to the broader academic landscape by integrating AI research within the Otto-Friedrich-Universität Bamberg's various faculties, including humanities, social sciences, and applied computer science.

Deep-Learning-Experiments is an open-source GitHub repository designed to help users understand deep learning through a combination of videos, detailed notes, and practical experiments. It offers comprehensive lecture notes covering fundamental deep learning topics such as Supervised Learning, Multilayer Perceptrons (MLP), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Transformers. The repository also includes code implementations for many of these concepts, allowing users to run and experiment with models like Mamba, Autoencoders (AE), Variational Autoencoders (VAE), Generative Adversarial Networks (GAN), and Diffusion Models. Additionally, it provides resources for setting up development environments, including Python, Numpy, PyTorch, and Hugging Face, making it a valuable resource for both theoretical understanding and practical application in deep learning.

SwissNLP is an association dedicated to advancing Natural Language Processing (NLP), Computational Linguistics, and Text Analytics within Switzerland. It serves as a bridge between AI and human language understanding and application, bringing together experts, solution providers, and customers from both industry and academia. The association organizes various events and projects to foster growth in the NLP field and also distributes datasets created through its initiatives. SwissNLP aims to promote innovation and knowledge sharing, offering membership opportunities for updates and collaboration, including a new 'Young Professionals' membership starting in 2026.

DB-GPT-Hub is an experimental open-source project focused on leveraging Large Language Models (LLMs) for Text-to-SQL parsing. It provides a comprehensive workflow including data collection, preprocessing, model selection, construction, and fine-tuning of model weights. The project aims to improve Text-to-SQL capabilities while reducing training costs, making it accessible for more developers to contribute. It supports various base models like CodeLlama, Baichuan2, LLaMa/LLaMa2, Falcon, Qwen, XVERSE, ChatGLM2, ChatGLM3, internlm, sqlcoder-7b, and sqlcoder2-15b. DB-GPT-Hub also supports Text2NLU fine-tuning for enhanced semantic understanding and Text2GQL fine-tuning for generating graph queries. The project has achieved high execution accuracy rates on the Spider dataset, demonstrating its effectiveness in enabling automated question-answering based on databases using natural language.

Kartoffel-1B-v0.1-Llasa 1b Tts is an AI tool hosted on Hugging Face Spaces, specializing in German zero-shot voice cloning. Users can generate speech from text by providing a reference audio sample, enabling personalized voice synthesis. The application also offers the flexibility to choose from a selection of predefined speakers or opt for a random voice, providing diverse options for audio output. This tool is fine-tuned with Llasa 1b, ensuring high-quality voice generation. The output is an audio file, making it suitable for various applications requiring synthesized German speech.

Databend is an open-source enterprise data warehouse built in Rust, offering a unified architecture for analytics, search, AI, and Python sandbox environments. It provides core capabilities such as large-scale analytics, vector search, full-text search, and auto schema evolution. Databend is agent-ready, featuring sandbox UDFs for agent logic, SQL for orchestration, transactions for reliability, and branching for safe experimentation on production data. Its architecture supports flexible agent orchestration with a control plane for resource scheduling, an execution plane for SQL orchestration, and a compute plane for isolated sandbox workers. Databend is cloud-native, elastic, and compatible with S3, Azure, and GCS, making it suitable for enterprise-scale AI workloads.

Custom Diffusion is an open-source tool developed by Adobe Research for multi-concept customization of text-to-image diffusion models, such as Stable Diffusion. It allows users to fine-tune these models with as few as 4-20 images of a new concept, significantly reducing training time to approximately 6 minutes on two A100 GPUs. The method is efficient, fine-tuning only key and value projection matrices in cross-attention layers, which limits the extra storage per concept to 75MB. Custom Diffusion supports combining multiple concepts, like new objects with new artistic styles, and includes a dataset of 101 concepts (CustomConcept101) along with SDXL integration. It provides scripts for single-concept and multi-concept fine-tuning, as well as optimization-based weight merging.

Rispose, formerly Unfetch, is an AI Agents & Automation tool designed to help businesses build and embed custom AI agents directly onto their websites or platforms. It enables automation of support, sales, and customer engagement through AI-powered assistants. Users can train their agents with up to 1,000 files, including PDFs, documents, and text files, and customize their behavior with specific instructions to match brand voice. The platform integrates with popular services like Shopify, WordPress, Notion, Wix, and Webflow. Rispose offers detailed history and metrics to track agent performance, understand user interactions, and facilitate continuous improvement. It provides a seamless and budget-friendly solution for integrating LLMs into existing web applications.

deep-learning-pytorch-huggingface is an open-source GitHub repository dedicated to providing comprehensive instructions, examples, and tutorials for individuals looking to get started with deep learning using PyTorch and Hugging Face libraries. It covers a wide range of topics, including fine-tuning large language models like FLAN-T5 and Falcon 180B with advanced techniques such as DeepSpeed ZeRO, LoRA, and Flash Attention. The repository also includes guidance on using transformers and datasets, quantizing open LLMs with optimum and GPTQ, and implementing RLHF with DPO. It's a valuable resource for learning about efficient distributed training with FSDP and Q-LoRA, as well as various inference examples for text generation and other tasks.

LLaMA Board is a platform designed for fine-tuning large language models (LLMs) through an intuitive Gradio user interface. It enables users to enter text prompts to generate and receive text responses, making it ideal for various applications such as writing, brainstorming, and more. The tool is built on the LlamaFactory framework, providing a robust environment for model customization and training. While the live website currently indicates a runtime error, its core functionality aims to support AI researchers and machine learning engineers in developing and adapting LLMs for specialized tasks. The platform leverages Hugging Face Spaces for deployment, offering a collaborative environment for AI projects.

DISC-LawLLM is an intelligent legal system powered by large language models (LLMs), developed and open-sourced by Fudan University's Data Intelligence and Social Computing Laboratory (Fudan-DISC). It offers comprehensive legal services, including legal text processing for information extraction and summarization, and legal reasoning capabilities enhanced by legal syllogism theory. The system also features a retrieval-augmented module for improved knowledge adherence, utilizing a vast knowledge base of laws and legal exam questions. DISC-LawLLM provides high-quality training datasets, effective training paradigms, and a robust evaluation framework, with its performance on the Lawbench benchmark ranking second only to GPT-4 among legal LLMs.

finetune-embedding provides a comprehensive workflow for fine-tuning embedding models to significantly improve Retrieval-Augmented Generation (RAG) performance. This tool is particularly valuable as it enables fine-tuning even in the absence of labeled data, by generating synthetic datasets using Large Language Models (LLMs). Users can process documents into text chunks, generate hypothetical questions for each chunk, and then use these query-document pairs to train an open-source embedding model. The repository includes step-by-step notebooks for dataset generation, model fine-tuning, and evaluation, demonstrating how this approach can substantially boost retrieval performance on small-scale datasets.

EasyLM is a comprehensive open-source framework designed to streamline the development and deployment of large language models (LLMs). Built on JAX/Flax, it offers a unified solution for pre-training, fine-tuning, evaluating, and serving LLMs. A key differentiator is its ability to scale LLM training across hundreds of TPU/GPU accelerators by leveraging JAX's pjit functionality, enabling the training of models that exceed the capacity of a single accelerator. The framework supports multi-host training on Google Cloud TPU Pods and integrates with Hugging Face's transformers and datasets, providing an easy-to-use and customizable codebase. EasyLM currently supports popular models like LLaMA, LLaMA 2, and LLaMA 3, and has been used to train models such as OpenLLaMA and Koala.

Grably is a multi-modal human interaction data research company specializing in providing high-quality conversational and interaction datasets for AI development. They offer a wide range of data applications, including large-scale multilingual and multimodal datasets for LLM pretraining, low-resource language modeling, and multimodal model training. Grably also provides specialized datasets for embodied AI, robotics, long-form video analysis, audio/speech understanding, code intelligence, and scientific/technical domain modeling. Their process involves defining critical human activities, capturing synchronized multi-signal data, structuring it with precise annotation, and scaling to diverse populations. They also offer custom dataset design and delivery tailored to specific research, legal, and infrastructure requirements.

GODEL offers large-scale pretrained models specifically designed for goal-directed dialog, built on a Transformer-based encoder-decoder architecture. These models are trained for response generation grounded in external text, making them highly effective for dialog tasks requiring conditioning on external information, such as retrieved documents. The repository provides the dataset, source code, and pre-trained models, allowing for efficient fine-tuning on new dialog tasks with minimal task-specific data. GODEL V1.1, for instance, was trained on 551 million multi-turn dialogs from Reddit and 5 million instruction and knowledge-grounded dialogs, demonstrating improved performance, especially in zero-shot settings. It supports fine-tuning and evaluation across various dialog tasks and includes a demo interface for interaction.

Hands-On Graph Neural Networks Using Python, published by Packt, is a comprehensive resource for machine learning practitioners, data scientists, and students interested in graph neural networks. The book covers fundamental concepts, practical implementation using Python and PyTorch Geometric, and applications ranging from natural language processing to drug discovery. It teaches users how to classify nodes, graphs, and edges, predict graph topologies, combine heterogeneous data sources, and forecast events. The resource includes all necessary code organized into folders, along with detailed instructions for setting up the required software and hardware, including Python, PyTorch, PyTorch Geometric, and optional GPU acceleration with CUDA and cuDNN. It also provides a list of required Python libraries and offers alternative access via Google Colab.

HCP-Diffusion is a comprehensive Diffusion model toolbox built on the RainbowNeko Engine, designed to simplify and unify Stable Diffusion workflows. It boasts a clean code structure and a flexible Python-based configuration system, making it ideal for conducting and managing complex experiments. The tool supports a wide array of training components and is highly extensible, flexible, and user-friendly compared to existing frameworks. Users can leverage a single Python config file to manage various training methods and model architectures, including Prompt-tuning (Textual Inversion), DreamArtist, Fine-tuning, DreamBooth, LoRA, and ControlNet. It also implements DreamArtist++, an upgraded version of DreamArtist based on LoRA, offering enhanced generalization, controllability, and faster training with minimal data.

gpt-fast is a highly efficient PyTorch-native transformer text generation tool, designed for minimal latency and a compact codebase of under 1000 lines of Python. It supports advanced features like int8/int4 quantization, speculative decoding, and tensor parallelism, making it suitable for high-performance applications. The tool is compatible with both Nvidia and AMD GPUs and is intended to showcase optimal performance achievable with native PyTorch, rather than serving as a comprehensive framework. Developers are encouraged to copy, paste, and fork the codebase for their specific needs, leveraging its efficiency for various LLM inference tasks.

Keras-transformer is a Python library designed to facilitate the construction of (Universal) Transformer models within the Keras framework. It offers essential building blocks such as positional encoding, embeddings, attention masking, and memory-compressed attention. The library also supports Adaptive Computation Time (ACT) and provides a general implementation for BERT models, making it highly relevant for Natural Language Processing (NLP) tasks. Developers can flexibly piece together multi-step Transformer models using its Keras layers, or customize existing components like self-attention and activation functions. The repository includes practical examples demonstrating its application in language modeling with BERT and GPT on datasets like WikiText-2.

KaJ Labs is a research organization founded in 2017 by J. King Kasr, dedicated to supporting teams building next-generation internet technologies. The foundation focuses on early Web3 projects, prioritizing innovation in areas like AI and Deep Learning. Key initiatives include Lithosphere (LITHO), a cross-chain network powered by AI; Imagen Network (IMAGE), the first decentralized social network with AI-generated content management; and Colle AI (COLLE), a multi-chain AI NFT platform for creating unique NFTs from prompts. KaJ Labs also develops Lithic, a smart-contract language for AI workflows, and LAX, an adaptive digital currency for the Lithosphere ecosystem.

LLM-PowerHouse is a comprehensive GitHub repository designed as an ultimate resource for developers, researchers, and enthusiasts looking to leverage Large Language Models (LLMs). It provides a curated guide with tutorials, best practices, and ready-to-use code for custom training and inferencing of LLMs. The resource covers foundational concepts in mathematics, Python, neural networks, and natural language processing, progressing to advanced topics like LLM architecture, instruction dataset building, pretraining, fine-tuning, RLHF, and evaluation. It also includes sections on model compression, inference optimization, open LLMs, and resources for cost analysis. LLM-PowerHouse aims to empower users to build intelligent applications and push the boundaries of natural language understanding.

GPT4All is a free, open-source AI chatbot designed to run entirely on your local device, providing private and high-performance AI without requiring cloud connectivity. This means your data remains on your machine, ensuring privacy and security. It supports various operating systems including Windows, macOS, and Linux, and can run on CPU-only machines or systems with NVIDIA or AMD GPUs for faster inference. GPT4All is ideal for developers, teams, and AI power-users, offering full customization, local document chat (LocalDocs), and compatibility with thousands of open-source models. This empowers users to build assistants and workflows with maximum control, security, and speed, making it a robust solution for offline and privacy-sensitive AI applications.