Research & Education

VideoLLM-online is the official implementation of an Online Video Large Language Model for Streaming Video, presented at CVPR 2024. Unlike traditional models that process full videos offline, VideoLLM-online enables real-time interaction within a video stream, allowing it to proactively update responses based on activity changes or assist with next steps. It features a cheap and scalable method for synthesizing streaming data by transforming offline annotations into dialogue data using open-source LLMs. The inference method is parallelized, combining video encoding, LLM forwarding, and response generation asynchronously, achieving high speeds of 10-15 FPS on an A100 GPU for long-form videos up to 10 minutes. The tool is designed for researchers and developers working with streaming video analysis and real-time multimodal AI.

VideoMamba is an innovative open-source state space model designed for efficient video understanding, specifically addressing the dual challenges of local redundancy and global dependencies in video data. It adapts the Mamba architecture to the video domain, overcoming limitations found in existing 3D convolution neural networks and video transformers. Its linear-complexity operator enables efficient long-term modeling, which is crucial for processing high-resolution and extended video content. The tool demonstrates scalability in the visual domain without requiring extensive dataset pretraining, thanks to a novel self-distillation technique. It also exhibits sensitivity for recognizing fine-grained short-term actions, superiority in long-term video understanding, and compatibility with multi-modal contexts, setting a new benchmark for comprehensive video analysis.

nanabanana2.run is an advanced AI image generator built on Google's Gemini 3.1 Flash Image Preview architecture, specializing in producing images with perfect text rendering. It excels at generating accurate mathematical solutions, detailed infographics, and multilingual content with crystal-clear typography. The tool supports up to 4K resolution output and offers features like reference consistency with multiple images, extreme aspect ratios, and Google Search grounding for factual context. Designed for professional applications, it delivers production-ready images for educational materials, technical documentation, and marketing visuals, outperforming other models in text accuracy and world knowledge understanding.

Enveda is a biotechnology company leveraging AI to revolutionize drug discovery. The platform reads and translates nature's hidden chemistry at unprecedented speed and scale, enabling scientists to access the chemical diversity of the natural world for the first time. By identifying and characterizing molecules found in nature, Enveda aims to discover medicines 4X faster than the industry average. This approach addresses the traditional challenges of purifying, identifying, and characterizing natural molecules, which are often time-consuming and costly. Enveda's mission is to accelerate the discovery of better medicines, offering hope to millions globally by harnessing billions of years of evolutionary intelligence.

Healx is a pioneering technology company dedicated to accelerating the discovery and development of treatments for rare diseases. Leveraging advanced AI and other frontier technologies, Healx redevelops, combines, and enhances known compounds to build a robust pipeline of new and effective therapies. The platform rapidly identifies novel drug-disease relationships with the highest chance of success, de-risking the drug discovery process and enabling parallel program execution. This approach aims to address the significant unmet need for treatments, as 90% of the over 10,000 rare diseases currently lack effective therapies. Healx collaborates with partners to progress treatments from prediction to patient.

Geophysical Insights offers Paradise, an AI workbench specifically designed for reservoir characterization in the oil and gas industry. It leverages machine learning to enable geoscientists and reservoir engineers to extract more geological details from seismic and well data than traditional methods. Key capabilities include producing higher resolution interpretations of reservoirs, automatically detecting faults, classifying seismic facies, and integrating faults with stratigraphic analysis for a comprehensive structural understanding. The platform aims to reduce exploration risk and field development costs by providing AI-enabled insights, dramatically decreasing workflow time, and improving the accuracy of results.

CelebAMask HQ Face Parsing is an AI-powered tool available on Hugging Face Spaces designed for detailed facial feature identification. Users can upload a portrait photo, and the application will automatically parse and label various facial components such as skin, eyes, hair, and lips. The output includes a color-coded label image, clearly marking each region, and a blended image that combines the original photo with the labels. This tool is particularly useful for tasks requiring precise segmentation of facial elements, offering a straightforward interface for quick analysis. While the core functionality is free to use on Hugging Face Spaces, advanced compute options and enterprise features are available through Hugging Face's broader pricing plans.

DeepForest is a Python package designed for training and predicting ecological objects within airborne RGB imagery. It comes equipped with pre-built models for tree crown object detection and bird detection, both of which are single-class modules. Users can extend these models for species classification by annotating new data and training custom models. Built on the object detection module from the torchvision package, DeepForest simplifies the process of training detection models. It aims to address the challenge of identifying individual organisms in high-resolution imagery, offering an open-source deep learning solution for crown detection and facilitating the retraining of models for various forest types, sensors, and spatial resolutions.

Alchemy Cloud offers an AI-powered Laboratory Information Management System (LIMS) and Electronic Lab Notebook (ELN) specifically designed for applied science labs. This platform enables chemists, material scientists, and other technical personnel to accelerate R&D, product development, testing, analysis, and quality control. Key features include digital lab automation combining ELN, LIMS, and PLM functionalities, commercial acceleration for faster product development, and scientific AI models that recommend formulations and predict technical properties. Alchemy aims to help scientific teams reduce data search time, speed up product development, and improve conversion rates on samples across various industries like Beauty & Personal Care, Food & Beverage, and Industrial Chemicals.

Agentarium is a powerful Python framework designed for managing and orchestrating AI agents with ease. It offers a flexible and intuitive platform for creating, managing, and coordinating interactions between multiple AI agents within various environments. Key features include advanced agent management, autonomous decision-making capabilities, and a robust checkpoint system for reproducibility. Users can define custom actions, and agents maintain memory of past interactions for contextual responses. The framework also boasts seamless integration with various AI providers through aisuite, an extensible architecture, and is performance-optimized for efficiency and scalability. It's ideal for developers and researchers looking to build and simulate complex AI agent systems.

The Artificial Intelligence in Medicine Lab (BCN-AIM) at the University of Barcelona is dedicated to pioneering AI solutions for personalized medicine. Their mission involves leveraging multi-source big data to develop innovative applications in healthcare. BCN-AIM is actively involved in numerous significant international research projects, focusing on areas such as medical imaging infrastructure, responsible health data sharing, AI for mental health, and future AI development within collaborative initiatives. The lab's work spans various medical domains, aiming to advance the integration of artificial intelligence into clinical practice and research.

Data Revival is an AI-powered platform designed to unlock the hidden value within legacy R&D data. It specializes in transforming unstructured scientific documents, lab notebooks, and complex PDFs into structured, searchable, and machine-ready data. By leveraging AI, Data Revival helps scientific organizations, particularly in chemistry, to recover and centralize valuable research information that might otherwise remain inaccessible. The platform aims to provide actionable R&D insights, making decades of scientific research readily available for analysis and future innovation. This process enhances data accessibility and supports more efficient scientific discovery and development.

Qwen2.5-Math represents a specialized series of large language models from the Qwen2 family, specifically engineered to excel in mathematical problem-solving and research. These models are tailored to handle complex mathematical queries, equations, and theoretical concepts, providing advanced capabilities for users in academic and scientific fields. By focusing on mathematics, Qwen2.5-Math aims to offer more accurate and relevant solutions compared to general-purpose LLMs. The models are accessible through popular platforms like Hugging Face and ModelScope, facilitating integration and experimentation for researchers and developers working on AI-driven mathematical applications.

regl-cnn is an open-source project designed for GPU-accelerated handwritten digit recognition, leveraging Convolutional Neural Networks (CNNs) within WebGL. This tool serves as a practical demonstration of how to implement a CNN directly on the GPU using WebGL, offering insights into high-performance computing for machine learning in web environments. The underlying network was initially trained using TensorFlow, and subsequently, its architecture and functionality were meticulously reimplemented in WebGL to showcase client-side inference capabilities. It is particularly useful for web developers interested in integrating machine learning models into web applications and machine learning enthusiasts looking to understand GPU-accelerated CNNs.

Sciform is an AI consulting firm that specializes in helping companies, organizations, investors, and board members build responsible AI solutions. They leverage in-depth knowledge in applied mathematics, distributed computing, and interdisciplinary collaboration to provide profound support for projects. Sciform aims to create real value for clients and their customers by smoothly realizing complex solutions in Artificial Intelligence, Big Data, Numerics, High Performance Computing, and Quantum Computing. They offer consulting services tailored for both companies/organizations and investors/board members, providing insights and support even without a specific technical background.

Duoverse offers advanced solutions for accelerating simulations by integrating AI, physics, and data to create actionable digital twins. The platform provides services such as bespoke digital twin development, hybrid AI for enhanced simulations, real-time physics simulation plug-ins, and urban systems modeling. These offerings are designed to optimize performance, minimize resource consumption, and drive innovation across various domains including electronics, mobility, and sustainability. Duoverse's approach combines machine learning algorithms with deep physics knowledge to deliver unparalleled insights and predictive capabilities, empowering clients to make informed decisions and achieve a greener future.

simple-neural-network is a Python script designed to illustrate the backpropagation algorithm, a fundamental concept in neural network training. This open-source tool serves as an educational resource for individuals interested in the inner workings of artificial neural networks. It provides a clear, step-by-step example of how neural networks learn by adjusting weights based on error signals. The script is particularly useful for students, AI enthusiasts, and developers who want to gain practical insight into the backpropagation process without needing to build a complex neural network from scratch. Its simplicity makes it an accessible entry point for understanding more advanced machine learning concepts.

SkyRL is a modular, open-source, full-stack reinforcement learning (RL) library specifically designed for large language models (LLMs). It aims to streamline research and development in the field of AI agents by offering a flexible framework for building and training intelligent agents. While the provided website content is a GitHub pricing page for GitHub itself, the tool's description indicates its core purpose is to support advanced AI development. Researchers and developers can leverage SkyRL to experiment with and implement various RL algorithms tailored for LLM applications, fostering innovation in AI agent capabilities and performance.

SqueezeLLM is a post-training quantization framework designed to optimize the serving of large language models (LLMs) through a novel Dense-and-Sparse Quantization method. This approach addresses the significant memory requirements of LLMs by splitting weight matrices into a dense component, which can be heavily quantized without performance loss, and a sparse component that preserves sensitive outlier parts. This allows for serving larger models with a smaller memory footprint, maintaining the same latency, and achieving higher accuracy and quality compared to baseline models. For instance, SqueezeLLM's variant of Vicuna models can operate within 6 GB of memory, surpassing FP16 baseline models in MMLU accuracy despite the latter requiring twice the memory. The framework supports various LLMs including LLaMA, LLaMA-2, Mistral, Vicuna, XGen, and OPT, with options for 3-bit and 4-bit quantization and different sparsity levels.

stock-rnn is an open-source project designed to predict stock market prices using Recurrent Neural Network (RNN) models. It specifically employs multilayer Long Short-Term Memory (LSTM) cells, a type of RNN architecture well-suited for sequential data like stock prices. The tool also offers optional multi-stock embeddings, allowing for more complex analysis across different stocks. Its primary purpose is to serve as a practical demonstration for building and training an RNN model within the Tensorflow framework, providing a hands-on platform for users to experiment with stock price prediction methodologies.

BeyondMath is a generative physics engine that leverages a foundational AI model trained on the fundamental laws of physics. This innovative approach allows for AI-powered physics simulations that are up to 1000x faster than traditional solvers, significantly compressing development cycles. Engineers can explore thousands of designs in real-time, gaining R&D edge and achieving predictive accuracy. The platform transforms engineering workflows across various industries, including automotive, aerospace, energy, defense, semiconductors, construction, telecommunications, and electronics. Users can upload geometry directly, without needing solver-ready meshes, and the AI automatically prepares the model for simulation. It visualizes pressure, velocity, temperature, and other fields in real time, allowing for detailed analysis and integration into existing workflows.

TensorLayerX is a versatile multi-backend AI framework designed for deep learning and reinforcement learning. It supports popular frameworks like TensorFlow, PyTorch, MindSpore, PaddlePaddle, OneFlow, and Jittor, enabling users to run their models on diverse hardware such as Nvidia-GPU, Huawei-Ascend, and Cambricon. Key features include compatibility across various AI chips and platforms, a Model Zoo offering classic and state-of-the-art models for CV, NLP, and RL, and deployment support via ONNX protocol for model export, import, and deployment. Developed by researchers from leading universities, TensorLayerX simplifies the process of defining models and switching between backends with minimal code changes.

Topological is developing physics-based foundation models specifically for CAD optimization, aiming to help hardware teams iterate at the speed of software teams. The technology leverages AI to accelerate engineering workflows, scaling design and optimization processes to identify ideal designs for complex problems while adhering to physical constraints. Its first model, UToP-v1, is a state-of-the-art topology optimization model that understands physics, geometry, and manufacturability. This model can generate highly efficient designs based on physical requirements, boasting less than 5% compliance error and operating 1930 times faster than traditional methods. Topological is reimagining mechanical engineering and computational design through precision spatial AI.

tensor_parallel is a Python library designed to automatically split PyTorch models across multiple GPUs, facilitating both training and inference for large language models (LLMs). This tool allows users to run models that would otherwise exceed the memory capacity of a single GPU, offering potentially linear speedups. It simplifies the process with a single line of code integration and supports memory-efficient dispatch by converting state_dicts. Key features include options for custom parallelism strategies, distributed training with `torch.distributed`, and sharding parameters using the ZeRO-3 algorithm to avoid duplicate parameters. It is particularly useful for quick prototyping on a single machine with multiple GPUs, offering an easier setup compared to more complex distributed training frameworks.