Coding & Development

mPLUG-Owl is a family of multi-modal large language models (MLLMs) designed to enhance language models with multimodality through a modular approach. The project includes several iterations: mPLUG-Owl, mPLUG-Owl2, and mPLUG-Owl3, each building upon the previous version to offer improved capabilities. mPLUG-Owl2, for instance, was accepted by CVPR 2024 as a Highlight, and mPLUG-Owl2.1 provides a Chinese-enhanced version. The latest iteration, mPLUG-Owl3, focuses on long image-sequence understanding. The source code and weights for these models are available on HuggingFace, making them accessible for researchers and developers to integrate and experiment with.

mteb (Massive Text Embedding Benchmark) is an open-source Python library designed for comprehensive evaluation of text and multimodal embeddings. It offers a standardized framework to benchmark the performance of different embedding models across a wide array of tasks, including classification, clustering, semantic textual similarity (STS), retrieval, and reranking. The tool supports both monolingual and multilingual evaluations, with a focus on reproducibility and ease of use. Developers and researchers can use mteb to select models, define custom models, run evaluations, and analyze results, contributing to an interactive leaderboard that tracks the state-of-the-art in embedding performance. Its modular design allows for easy integration of new models, datasets, and benchmarks.

OllamaSharp offers .NET bindings for the Ollama API, making it straightforward to integrate Ollama into .NET applications for both local and remote interactions. It provides comprehensive API coverage, including chats, embeddings, model management (listing, pulling, pushing, copying, deleting, showing), and real-time streaming of responses and progress reports. The library is designed for ease of use, powering Microsoft Semantic Kernel and .NET Aspire, and supports advanced features like a sophisticated tools engine with source generators, multi-modality for vision models, and native AOT for improved performance. It also integrates seamlessly with Microsoft.Extensions.AI, allowing developers to use OllamaSharp as an IChatClient or IEmbeddingGenerator alongside other AI providers.

DyNet is a powerful open-source neural network library, primarily developed by Carnegie Mellon University, with contributions from many others. Written in C++ and offering Python bindings, it's engineered for efficiency on both CPU and GPU architectures. A key differentiator is its ability to handle dynamic neural network structures, which can adapt and change for each training instance. This makes DyNet particularly well-suited for complex natural language processing tasks, where it has been successfully applied to build state-of-the-art systems for syntactic parsing, machine translation, and morphological inflection. The toolkit provides comprehensive documentation, tutorials for both C++ and Python, and examples to help users get started with its auto-batching feature and other functionalities.

DropoutUncertaintyExps is an open-source project containing the experimental code for the paper "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning." The repository provides a framework for researchers to replicate and build upon the uncertainty experiments, with adaptations reflecting community feedback and bug fixes. It is based on José Miguel Hernández-Lobato's work on probabilistic backpropagation for scalable learning of Bayesian Neural Networks. The code utilizes datasets from the UCI machine learning repository, with specific data splits to ensure comparability of results. It details the methodology for hyperparameter tuning using grid-search and reports RMSE and log-likelihood metrics for various datasets, offering a valuable resource for academic research in deep learning uncertainty.

ego-planner-swarm is an open-source, efficient single/multi-agent trajectory planner specifically designed for multicopters. This tool extends the capabilities of EGO-Planner for swarm navigation, offering a fully autonomous and decentralized solution for multi-robot navigation in complex, unknown environments using only onboard resources. It supports ROS integration and is compatible with Ubuntu 16.04, 18.04, and 20.04, with a dedicated ROS2 version available on a separate branch. Developers can easily compile and run simulations, with options to configure for GPU usage for depth image generation or CPU for broader compatibility. The project also provides recommendations for optimizing CPU performance for stable computation times, making it a robust solution for advanced robotics development.

finetrainers is a work-in-progress library from Hugging Face designed for scalable and memory-optimized training of diffusion models. It provides support for various commonly used training algorithms, including DDP, FSDP-2, HSDP, and CP. Key features include LoRA and full-rank finetuning, conditional control training, and memory-efficient single-GPU training. The library also supports multiple attention backends like flash, flex, sage, and xformers, along with auto-detection of common dataset formats. It's built to handle combined image/video datasets, multi-resolution bucketing, and offers memory-efficient precomputation. finetrainers is recommended for use with PyTorch 2.5.1 or above for optimal performance and reproducibility.

ExLlamaV3 is an inference library specifically designed for running Large Language Models (LLMs) locally on modern consumer-class GPUs. Its headline feature is the new EXL3 quantization format, which is based on QTIP from Cornell RelaxML, allowing for efficient model conversion in a single step. The library supports flexible tensor-parallel and expert-parallel inference setups, and provides an OpenAI-compatible server via TabbyAPI for local or remote inference. It also includes features like continuous, dynamic batching, HF Transformers plugin support, speculative decoding, and 2-8 bit cache quantization. ExLlamaV3 aims to make advanced quantization techniques more accessible and less resource-intensive, enabling users to run large models like Llama-3.1-70B with minimal VRAM.

Fara-7B is Microsoft's first agentic small language model (SLM) specifically engineered for computer use. With only 7 billion parameters, it offers an ultra-compact solution for automating multi-step tasks on behalf of users. Unlike traditional chat models, Fara-7B interacts with computer interfaces visually, perceiving webpages and performing actions like scrolling, typing, and clicking directly on predicted coordinates without relying on accessibility trees. This design allows for efficient on-device deployment, reducing latency and enhancing privacy by keeping user data local. Fara-7B completes tasks efficiently, averaging only ~16 steps per task, and achieves state-of-the-art performance within its size class, competing with larger agentic systems. It is trained on 145K trajectories using a novel synthetic data generation pipeline built on the Magentic-One multi-agent framework, and is based on Qwen2.5-VL-7B with supervised fine-tuning.

Food-Recipe-CNN is a deep learning project designed to recognize food images and suggest matching recipes. Utilizing deep convolutional neural networks (CNNs) with Keras, this system can classify food images into 230 distinct categories. The project leverages a large dataset of over 400,000 food images and 300,000 recipes from chefkoch.de. It employs transfer learning with pre-trained CNNs like InceptionV3 and VGG16, alongside feature extraction and dimensionality reduction techniques such as PCA. The goal is to provide a solution for automated recognition of photographed dishes and subsequent recipe retrieval, with a web application called DeepChef in development.

Panicle Tech is a leading software development company offering specialized services in AI/ML solutions, Cloud infrastructure, DevOps, AWS services, and Security integration. They provide expert consultancy to help businesses build scalable startups and robust enterprise solutions. Their expertise spans various technologies including Kubernetes, Docker, TensorFlow, Python, and React, ensuring comprehensive support for modern software development needs. Panicle Tech focuses on delivering cutting-edge technology and strategic guidance to empower clients in navigating complex technological landscapes and achieving their business objectives.

FlexFlow Train is an open-source deep learning framework designed to accelerate distributed deep neural network (DNN) training. It achieves this by automatically searching for and implementing efficient parallelization strategies. The tool helps optimize the training process, reducing the time required for model development and improving overall efficiency. It supports various deep learning models and hardware configurations, making it a versatile solution for researchers and developers working with large-scale DNNs. The project is developed and maintained by teams from several prominent institutions, including CMU, Facebook, Los Alamos National Lab, MIT, Stanford, and UCSD.

open-llms is a comprehensive GitHub repository that serves as a curated list of open Large Language Models (LLMs) explicitly licensed for commercial use, including Apache 2.0, MIT, and OpenRAIL-M. This resource is invaluable for developers, researchers, and businesses looking to integrate open-source LLMs into their applications without licensing concerns. The repository details each model's release date, available checkpoints, associated research papers or blog posts, parameter sizes, context lengths, and specific licenses. It also includes a dedicated section for open LLMs tailored for code generation, offering insights into models like SantaCoder, CodeGen2, and StarCoder. Contributions to the list are welcomed, ensuring it remains up-to-date with the latest commercially viable open LLM releases.

game-datasets is a comprehensive GitHub repository offering a curated list of awesome game datasets and tools specifically designed for artificial intelligence in games. This resource is invaluable for researchers, developers, and enthusiasts working on AI or data mining applications within the digital games domain. The repository categorizes its offerings into APIs for accessing game data, various AI experimentation platforms and competitions, mobile game resources, relevant books, and an extensive collection of game datasets. These datasets cover a wide range of games, from popular titles like League of Legends and Dota 2 to classic board games and even Pokémon. Additionally, it includes related datasets, market research, and miscellaneous resources, making it a central hub for anyone looking to build AI applications or conduct data analysis in gaming.

Gemma is an open-weight Large Language Model (LLM) library developed by Google DeepMind, leveraging research and technology from the Gemini models. This repository offers the implementation of the gemma PyPI package, providing a JAX library for both using and fine-tuning Gemma models. It supports multi-turn, multi-modal conversations and offers various versions of Gemma. The library is designed to run on CPU, GPU, and TPU, with specific RAM recommendations for GPU usage (8GB+ for 2B checkpoint, 24GB+ for 7B checkpoint). Extensive documentation, Colabs, and tutorials are available for sampling, multi-modal fine-tuning, and LoRA.

FunASR is a fundamental end-to-end speech recognition toolkit designed to bridge the gap between academic research and industrial applications. It offers a comprehensive suite of features including speech recognition (ASR), Voice Activity Detection (VAD), Punctuation Restoration, Language Models, Speaker Verification, Speaker Diarization, and multi-talker ASR. The toolkit provides convenient scripts and tutorials for both inference and fine-tuning of pre-trained models. FunASR boasts a vast collection of academic and industrial pre-trained models available on ModelScope and Hugging Face, including the highly accurate and efficient Paraformer-large. Recent updates include support for large models like Fun-ASR-Nano-2512 (31 languages), Whisper-large-v3-turbo, and Qwen-Audio multimodal models, alongside continuous improvements in real-time and offline transcription services, memory optimization, and multi-platform support.

free-llm-api-resources is a comprehensive list of services that provide free access or trial credits for API-based Large Language Model (LLM) usage. This resource is invaluable for developers, researchers, and students looking to experiment with LLMs without initial financial commitment. The list details various providers like OpenRouter, Google AI Studio, NVIDIA NIM, Mistral, HuggingFace, and others, specifying their free tiers, usage limits, and available models. It also includes providers offering trial credits such as Fireworks, Baseten, and AI21. The tool emphasizes legitimate services, explicitly excluding those that reverse-engineer existing chatbots, ensuring users find reliable and ethical resources for their projects.

PromptWizard is an open-source, task-aware, agent-driven framework designed for optimizing prompts used with Large Language Models (LLMs). It features a self-evolving mechanism where the LLM itself generates, critiques, and refines its own prompts and in-context learning examples. This iterative feedback loop ensures continuous improvement in task performance. The framework focuses on holistic optimization by evolving both instructions and examples, generating synthetic, diverse, and task-aware examples. It also supports self-generated Chain of Thought (CoT) steps and offers various scenarios for prompt optimization, including with and without training data, and the generation of synthetic examples. Users can configure hyperparameters and integrate with custom datasets, making it a flexible tool for developers and researchers working with LLMs.

PointNet is a novel deep learning architecture specifically designed for processing point clouds, which are an important type of geometric data structure. Unlike traditional methods that convert point clouds into regular 3D voxel grids or image collections, PointNet directly consumes unordered point sets, respecting their permutation invariance. This approach makes it highly efficient and effective for a range of applications, including object classification, part segmentation, and scene semantic parsing in 3D. Developed by researchers at Stanford University, PointNet is available as an open-source project on GitHub, providing code and data for training classification and part segmentation networks. It has also served as a foundational work for subsequent advancements like PointNet++.

practical-nlp-code is the official GitHub repository for the code accompanying the 'Practical Natural Language Processing' book published by O'Reilly Media. This repository serves as a comprehensive resource for individuals looking to build real-world NLP systems, providing practical code examples and notebooks. It covers various NLP topics across its chapters, including NLP pipelines, text representation, text classification, information extraction, and applications in areas like chatbots, social media, e-commerce, retail, healthcare, finance, and law. The repository is actively maintained, with ongoing development to update notebooks for newer environments like Ubuntu 23 and future migration to TensorFlow 2.x, making it a valuable learning and development tool for those interested in natural language processing.

pysentimiento is an open-source Python toolkit designed for Sentiment Analysis and Social NLP tasks, leveraging Transformer-based models. It offers robust capabilities for sentiment analysis, hate speech detection, irony detection, and emotion analysis across multiple languages including Spanish, English, Italian, and Portuguese. Additionally, it provides NER & POS tagging for Spanish and English, and specialized contextualized hate speech detection and targeted sentiment analysis for Spanish. The library includes a tweet preprocessor optimized for transformer-based models, handling user handles, URLs, repeated characters, laughters, hashtags, and emojis. Developers can easily integrate it into their projects via pip install and utilize its `create_analyzer` function for various tasks.

gobrain is an open-source library written in Go, offering fundamental neural network functionalities. It currently supports Feed Forward and Elman Recurrent Neural Networks, allowing developers to construct and train neural networks within the Go programming environment. The library provides methods for initializing network structures, training with specified patterns, and testing the network's performance. It also includes features for persistence, enabling users to save and load trained networks from files. This makes gobrain a suitable choice for developers and data scientists looking to implement and experiment with basic neural network models in Go for various machine learning and AI development tasks.

R-KV is a novel method for redundancy-aware KV cache compression specifically designed for large language models (LLMs) that rely on chain-of-thought (CoT) or self-reflection for reasoning tasks. It addresses the issue of bloated key-value (KV) caches during inference by ranking tokens on-the-fly for both importance and non-redundancy, retaining only the most informative and diverse ones. This approach allows for significant memory savings, up to 90%, and improved throughput (up to 6.6x) during long CoT generation, often with zero or even negative accuracy loss. R-KV is a plug-and-play, training-free solution that acts as a lightweight wrapper for any autoregressive LLM, making it easy to integrate into existing inference pipelines or RL roll-outs.

qxresearch-event-1 is a GitHub repository providing a hands-on tutorial with over 50 Python applications, each meticulously crafted to be under 10 lines of code. This resource spans a wide array of topics including Machine Learning, Deep Learning, GUI development, Computer Vision, and API creation. Designed for both beginners and experienced developers, the concise nature of each application facilitates easy understanding and modification, making it an ideal platform for learning and experimenting with Python. The repository also offers video explanations for each project on the @qxresearch YouTube channel, enhancing the learning experience and allowing users to quickly grasp and customize the code. It fosters a community for Python enthusiasts to connect and stay updated on new projects.