Data & Analytics

prodigy-recipes is an open-source repository offering a diverse collection of recipes designed for Prodigy, Explosion AI's scriptable annotation tool. These recipes facilitate various data annotation tasks across text, images, and other data types, making it a valuable resource for machine learning and natural language processing practitioners. The repository includes specialized recipes for Named Entity Recognition (NER), text classification, terminology bootstrapping, and image annotation, covering tasks from manual labeling to model-in-the-loop active learning. Users can customize these scripts to tailor Prodigy's behavior, such as modifying sorting functions or adding custom filters. While the recipes are similar to those built into Prodigy, they are enhanced with comments and simplifications to serve as a clearer foundation for custom development. A Prodigy license is required to utilize this collection.

Wd14 Tagging Online is an AI-powered tool hosted on Hugging Face Spaces, designed for comprehensive image tagging and rating. It allows users to upload images and receive detailed tags, which are crucial for data labeling in AI model training and various research applications. The platform offers flexibility through adjustable settings, enabling users to fine-tune the confidence threshold for tags and select preferred output formatting options. This customization ensures that the generated tags meet specific project requirements, making it a valuable resource for professionals involved in image analysis and machine learning development.

segmenteverygrain is a Python package designed for instance segmentation of grains and grain-like objects in images, particularly useful in geomorphology and sedimentary geology for determining grain size and shape. It leverages Meta's SAM 2.1 (Segment Anything Model 2.1) for high-quality outlines, overcoming SAM's limitations by using a Unet-style convolutional neural network for initial segmentation and prompt generation. The tool includes interactive functions for cleaning up segmentation results, such as deleting, merging, and adding grains, and allows saving QC-d masks to improve the Unet model. It also offers modules for extracting individual grain images, feature extraction using pre-trained CNNs, and clustering for classification tasks.

BERT-NER is an open-source tool leveraging Google's BERT model for named entity recognition (NER), specifically fine-tuned on the CoNLL-2003 dataset. This updated version addresses shortcomings of the original by providing clearer annotations and improved data preprocessing and layer design, making it easier for developers to implement and modify. Users can experiment with different layer designs, such as CRF or Softmax, to optimize performance. The repository includes all necessary files, such as BERT model components, data directories, and evaluation scripts, along with detailed instructions for usage. It offers strong performance metrics on the CoNLL-2003 test set, including high accuracy, precision, recall, and F1 scores for various entity types like LOC, MISC, ORG, and PER.

Image Captioning with BLIP is an AI tool hosted on Hugging Face Spaces, designed to generate descriptive captions for uploaded images. Leveraging the BLIP model, it analyzes the visual content of an image to produce relevant textual descriptions. A key feature of this tool is the ability to guide the caption generation process by providing a starting text, allowing users to influence the output. This makes it particularly useful for tasks requiring specific types of captions or for refining the AI's understanding of the desired context. The tool is accessible via a web interface, making it easy to use for various applications.

Damselfly is a server-based Digital Photograph Management system designed to efficiently manage and search extremely large, folder-based collections of images. It leverages powerful Machine Learning for facial detection, face recognition, and object detection, enabling users to quickly identify and tag subjects across their photo library. The system supports a wide range of image formats, including RAW files, and offers full-text search, advanced filtering options, and a fast keyword tagging workflow with non-destructive EXIF data updates. Damselfly also includes a desktop client for closer integration with local file systems, allowing for easy syncing and editing workflows, and supports multi-user environments with role-based entitlements.

Music Genre Classifier is an AI-powered tool hosted on Hugging Face Spaces, designed to analyze and classify the genre of music tracks. Users can upload short MP3 files, ideally under 15 seconds, and choose from various pre-trained models. The tool processes the audio by converting it into visual spectrograms, which are then fed into a neural network for analysis. It provides the most likely genre classification, making it useful for music analysis, data labeling, and potentially for building music recommendation systems. This web-based application offers a straightforward interface for quick genre identification.

GFNet is an innovative AI tool that introduces Global Filter Networks for image classification, detailed in NeurIPS 2021 and T-PAMI. This transformer-style architecture efficiently learns long-term spatial dependencies by operating in the frequency domain, leveraging a 2D discrete Fourier transform, element-wise multiplication with learnable global filters, and a 2D inverse Fourier transform. It replaces traditional self-attention layers, offering a conceptually simple yet computationally efficient solution, especially for high-resolution feature maps. The PyTorch implementation is open-source, built upon pytorch-image-models and DeiT, and includes pre-trained models on ImageNet, along with scripts for evaluation, training, and transfer learning.

Tika Data specializes in providing data annotation services, crucial for fueling the AI age. The company is dedicated to innovation within the data annotation field, aiming to meet the increasing demands of AI applications. Tika Data supports the development of robust AI models by offering high-quality data labeling and preparation services. Their focus is on ensuring that AI systems have access to accurately labeled and well-prepared datasets, which are fundamental for effective machine learning and AI model training. This commitment to quality data annotation helps businesses and developers build more reliable and efficient AI solutions.

GPTeacher is a comprehensive collection of modular datasets, meticulously generated by GPT-4, designed to facilitate various AI training and development tasks. The collection includes several distinct datasets: General-Instruct, Roleplay-Instruct, Code-Instruct, and Toolformer. The General-Instruct dataset, comprising approximately 20,000 examples, focuses on diverse tasks such as Chain of Thought Reasoning, Logic Puzzles, and Wordplay. The Roleplay-Instruct dataset, now in its V2 (Supplemental) version, is 2.5 times larger than the original and features simulated conversations for character role-playing. The Code-Instruct dataset offers around 5,350 code task instructions across various programming languages. Additionally, the Toolformer dataset is designed for training models to use predefined tools like search, Python, and Wikipedia. All datasets are formatted to be compliant with Alpaca's dataset structure, including instruction, input, and output fields, making them easy to integrate into existing fine-tuning processes.

LabelLLM is an innovative, open-source platform dedicated to optimizing the data annotation process crucial for Large Language Model (LLM) development. It is engineered to be a powerful tool for independent developers and small to medium-sized research teams, significantly improving annotation efficiency. The platform provides comprehensive task management solutions, offering real-time monitoring of annotation progress and quality control to ensure data integrity. LabelLLM supports a wide range of data modalities, including audio, images, and video, allowing for complex annotation projects on a single unified platform. Its flexible framework includes customizable task-specific tools and AI-assisted annotation features like pre-annotation loading, which users can refine for enhanced accuracy and efficiency.

OmDet is an open-source project providing OmDet-Turbo, a fast transformer-based open-vocabulary object detection model. It excels in real-time detection scenarios while maintaining high performance. A key innovation is the Efficient Fusion Head, which reduces computational burden and inference time. OmDet-Turbo-Base achieves state-of-the-art zero-shot performance on ODinW and OVDEval datasets, with impressive AP scores of 30.1 and 26.86 respectively. It also boasts a rapid inference speed of 100.2 FPS on an A100 GPU for the COCO val2017 dataset. The project offers installation instructions, local inference capabilities, and the option to run as an API server, making it versatile for various applications.

prm800k is an open-source dataset and accompanying tools, released by OpenAI, that provides 800,000 step-level correctness labels for large language model (LLM) solutions to mathematical problems from the MATH dataset. This resource is crucial for researchers and developers aiming to enhance the mathematical reasoning capabilities of AI models through process supervision. The repository includes raw labels, instructions for labelers, Python grading logic for answer correctness, and non-standard MATH train/test splits. It also contains scored samples used to evaluate large-scale ORM and PRM models, making it a comprehensive resource for advancing AI in mathematics.

SAMv2 Mask Generator is an AI-powered tool available as a Hugging Face Space by lightly-ai, designed for image segmentation tasks. Users can upload any image and interactively define objects of interest by drawing bounding boxes around them. The tool then automatically generates precise segmentation masks, highlighting the selected objects within the image. This functionality is particularly useful for various computer vision applications, including object detection, image analysis, and data labeling, providing a straightforward method to isolate and analyze specific elements within visual data. It offers a practical solution for researchers, developers, and data annotators working with image datasets.

synthetic-data-kit is a powerful open-source tool developed by Meta Llama for generating high-quality synthetic datasets specifically designed to fine-tune Large Language Models. It streamlines the often complex process of data preparation, allowing users to create reasoning traces, QA pairs, and summaries from various input formats. The tool features a modular 4-command CLI flow: ingest, create, curate, and save-as, enabling users to process individual files or entire directories. It supports different LLM backends like vLLM or external API endpoints and can convert curated data into various fine-tuning formats such as Alpaca, OpenAI fine-tuning format, and ChatML. Additionally, it handles multimodal data, extracting both text and images, and offers intelligent chunking for large documents to maintain context and quality.

The synthetic-personality-dataset offers a high-fidelity collection of 10,000 synthetic records designed to simulate the behavioral and social patterns of introverted and extroverted individuals. Generated using Syncora.ai's synthetic data engine, this dataset ensures zero privacy risk while preserving real-world behavioral distributions. It is ideal for researchers, data scientists, and AI developers focused on personality prediction, behavioral modeling, machine learning experiments, and social science research. The dataset includes features like time spent alone, social event attendance, social media posting habits, and a personality target label, making it suitable for various analytical and ML use cases without compromising privacy or ethical concerns.

Epinote offers a comprehensive suite of services designed to plug people into workflows, specializing in data annotation, data collection, and project support. It enables businesses to save resources by delegating labor-intensive projects and simple tasks to its network of freelancers. The platform provides efficient data annotation for AI & ML, data collection, and various project support functions. Epinote helps companies accelerate go-to-market strategies, streamline back-office operations, and prepare data for AI initiatives. It offers tailored projects with bespoke workflows and supports various departments including sales, marketing, customer support, data, operations, HR, finance, and project management, aiming to double efficiency by replacing manual processes with a mix of technology and on-demand workforce.

CelebAMask HQ Face Parsing is an AI-powered tool available on Hugging Face Spaces designed for detailed facial feature identification. Users can upload a portrait photo, and the application will automatically parse and label various facial components such as skin, eyes, hair, and lips. The output includes a color-coded label image, clearly marking each region, and a blended image that combines the original photo with the labels. This tool is particularly useful for tasks requiring precise segmentation of facial elements, offering a straightforward interface for quick analysis. While the core functionality is free to use on Hugging Face Spaces, advanced compute options and enterprise features are available through Hugging Face's broader pricing plans.

awesome-data-labeling is a comprehensive, curated list of open-source and commercial tools designed for data labeling across multiple modalities. It categorizes tools for image, video, text, audio, time series, 3D, and LiDAR data annotation, making it a valuable resource for data scientists and machine learning engineers. The list includes popular tools like CVAT for computer vision, Labelbox for general data annotation, and YEDDA for text span annotation. It serves as a central hub for discovering tools that facilitate the creation of high-quality datasets for training machine learning models, offering solutions for various annotation tasks from bounding boxes to semantic segmentation.

MediaViz AI provides powerful AI-driven insights for media management and analysis, offering tools for tagging, photo curation, and personalized media solutions. It's designed to "see" images the way humans do, offering scalable content perception solutions, image recognition technology, and personalized insights for businesses. The platform helps automate tasks like photo selection, auto-labeling, caption generation, and object classification. MediaViz AI offers flexible solutions through ready-to-use AI Apps for smaller teams and AI as a Service for enterprise-level integration, catering to diverse needs from wedding photographers to large digital asset management systems. It aims to save time, enhance user experience, and boost conversions.

Shield AI specializes in developing and deploying mission autonomy solutions for defense applications. Their core offering, Hivemind Enterprise, is a comprehensive AI-powered developer platform designed to accelerate the development, evaluation, testing, and deployment of mission autonomy. Key solutions include ViDAR, a hardware and software package for unrivaled surveillance day or night, and Tracker, AI-powered software for superior object detection in videos, including a C-UAS version for drone hunting. They also develop AI-piloted aircraft like the X-BAT VTOL fighter jet and the V-BAT for ISR and targeting. Shield AI aims to protect service members and civilians with intelligent systems, focusing on autonomy in contested and degraded environments where traditional communication and manual control falter.

Studio Automated is an innovative AI video production tool that leverages world-class AI to redefine sports broadcasting. It provides automated broadcast solutions for 15 different sports, supporting various production cameras including PTZ, panoramic, and portable setups. The platform excels in automated event detection and highlight creation, ensuring no latency with both cloud and server-based installations. Key products include Virtual Director Regular, Virtual Director PRO, and Virtual Director Portable for AI video production, alongside AI Data Insights tools like Virtual Coach Assistant and Data Solutions, which offer real-time insights to enrich productions and improve team performance. Studio Automated aims to connect one billion people to their favorite sports matches globally through its partner model.

Florence2 + SAM2 is an advanced AI tool designed for precise image and video segmentation. It leverages the capabilities of Florence2 and SAM2 models to enable users to upload visual content and define objects of interest through text prompts. The application then highlights these specified objects, providing detailed segmentations, captions, or descriptions. This makes it particularly useful for tasks requiring accurate object identification and isolation within complex visual data. While the tool is currently experiencing a runtime error, its intended functionality targets AI researchers, developers, and professionals in image processing who need robust segmentation capabilities.

OpenCUA is a comprehensive open-source framework designed for scaling Computer-Use Agent (CUA) data and foundation models. It features AgentNet, the first large-scale computer-use task dataset spanning multiple operating systems and applications, and AgentNetTool, an annotation infrastructure for capturing human computer-use demonstrations. The framework also includes AgentNetBench, an offline evaluator for benchmarking model-predicted actions, and OpenCUA Models, end-to-end computer-use foundation models with strong planning and grounding capabilities. Notably, OpenCUA-72B achieves state-of-the-art performance on OSWorld-Verified, making it ideal for developers and researchers working on advanced AI agents.