Research & Education

Keras-transformer is a Python library designed to facilitate the construction of (Universal) Transformer models within the Keras framework. It offers essential building blocks such as positional encoding, embeddings, attention masking, and memory-compressed attention. The library also supports Adaptive Computation Time (ACT) and provides a general implementation for BERT models, making it highly relevant for Natural Language Processing (NLP) tasks. Developers can flexibly piece together multi-step Transformer models using its Keras layers, or customize existing components like self-attention and activation functions. The repository includes practical examples demonstrating its application in language modeling with BERT and GPT on datasets like WikiText-2.

KaJ Labs is a research organization founded in 2017 by J. King Kasr, dedicated to supporting teams building next-generation internet technologies. The foundation focuses on early Web3 projects, prioritizing innovation in areas like AI and Deep Learning. Key initiatives include Lithosphere (LITHO), a cross-chain network powered by AI; Imagen Network (IMAGE), the first decentralized social network with AI-generated content management; and Colle AI (COLLE), a multi-chain AI NFT platform for creating unique NFTs from prompts. KaJ Labs also develops Lithic, a smart-contract language for AI workflows, and LAX, an adaptive digital currency for the Lithosphere ecosystem.

Clema is an AI-powered platform designed for Institutional Research and Effectiveness teams in higher education. It acts as an AI copilot, enabling users to query complex federal education databases such as IPEDS, College Scorecard, and EADA using natural language. This eliminates the need to navigate intricate interfaces, providing instant access to data insights. Clema supports various copilots for specific data sets like Pell Grant, Cohort Default Rate, DAPIP, and PSEO, streamlining data requests and analysis for university professionals. The tool aims to enhance efficiency in institutional research by offering quick answers to data questions through conversational AI.

Instructgpt-prompts is an open-source project offering a comprehensive collection of instruction-based prompts and strategies specifically designed for GPT-instruct and GPT-3.5 models. It focuses on leveraging the instruction-following capabilities of these language models for various text generation and classification tasks. The project highlights the sensitivity of models to phrasing and position within prompts, providing guidance on how to structure prompts effectively using useful verbs and directional words. It covers common use cases such as classification, generation, transformation, and comparison, offering specific instruction verbs for each. This resource is particularly valuable for understanding prompt engineering principles for base and SFT-only models, aiming to align large language models with human intent.

Ilya-sutskever-recommended-reading is a curated list of approximately 30 influential deep learning research papers, compiled from a reading list reportedly given by Ilya Sutskever to John Carmack. This resource is designed for individuals seeking to build a strong foundational understanding of deep learning concepts and methodologies. The list includes seminal works such as "Attention Is All You Need," "ImageNet Classification with Deep Convolutional Neural Networks," and "The Unreasonable Effectiveness of Recurrent Neural Networks." Each entry typically provides links to the paper, PDF, and sometimes associated blogs or code, making it a valuable starting point for in-depth study and research in the field of artificial intelligence and machine learning.

KwaiAgents is an open-source project from KwaiKEG at Kuaishou Technology, offering a generalized information-seeking agent system built with Large Language Models (LLMs). The project includes KAgentSys-Lite, a simplified agent system with core functionalities, and KAgentLMs, a series of LLMs specifically tuned for agent capabilities such as planning, reflection, and tool-use. It also provides KAgentInstruct, a large dataset of agent-related instructions for fine-tuning, and KAgentBench, a comprehensive benchmark for evaluating agent performance across various dimensions. KwaiAgents supports both local and cloud-based LLM usage, making it a versatile platform for researchers and developers in the AI agent space.

LLM-Agents-Papers is a GitHub repository that curates a comprehensive list of research papers focused on Large Language Model (LLM) based agents. The repository categorizes papers by various aspects including Survey, Technique For Enhancement, Planning, Memory Mechanism, Feedback & Reflection, RAG, Search, Interaction, Role Playing, Conversation, Game Playing, Human-Agent Interaction, Tool Usage, Simulation, Application (across diverse fields like Math, Chemistry, Biology, Physics, Geography, Art, Medicine, Finance, Software Engineering), Research Automation, Workflow, Automatic Evaluation, Training, Fine-tuning, RL, DPO, Scaling, Single-Agent Framework, Multi-Agent System, Stability, Safety, Bias, Hallucination, Infrastructure, Benchmark & Evaluation, Environment & Platform, Dataset, and Others. It also provides recommendations for other related paper lists, making it an invaluable resource for academic research and development in the LLM agent domain.

MachineLearning-QandAI-book is an open-source GitHub repository offering supplementary materials for Sebastian Raschka's "Machine Learning Q and AI" book. It's designed for individuals who have a foundational understanding of machine learning and AI but wish to deepen their knowledge and address specific gaps. The repository includes practical code examples and detailed explanations across various topics, such as multi-GPU training, finetuning transformers, generative AI models, and confidence intervals for ML. Users can find discussions and code for concepts like embeddings, self-supervised learning, few-shot learning, and different types of neural networks. The resource is ideal for those looking to stay current with the latest technologies and implement advanced machine learning techniques in their work.

nlp-journey is an open-source GitHub repository offering a comprehensive collection of resources for Natural Language Processing (NLP). It includes a wide array of documents, academic papers, and code examples covering key NLP areas such as Topic Models, Word Embeddings, Named Entity Recognition, Text Classification, Text Generation, Text Similarity, and Machine Translation. The repository is structured to provide easy access to foundational and advanced research, making it an invaluable resource for students, researchers, and practitioners in the field of NLP. It serves as a central hub for exploring the latest advancements and understanding the underlying principles of various NLP techniques.

promptoftheyear is an open-source collection of impactful prompts designed for Large Language Models (LLMs). It serves as a valuable resource for individuals looking to enhance their prompt engineering skills across diverse domains. The collection includes prompts for job hunting, essay and research, language learning, code generation, image generation, mental health support, music creation, marketing, and data analysis. Each prompt in the collection includes a backlink to acknowledge its original author, ensuring proper attribution. The repository also provides access to a prompts.csv file containing the complete collection and suggests free chatbots for interactive use, allowing users to see the prompts in action.

self-adaptive-llms, also known as Transformer², is a novel self-adaptation framework designed to overcome the limitations of traditional, computationally intensive fine-tuning methods for Large Language Models (LLMs). This framework enables LLMs to adapt to unseen tasks in real-time by selectively adjusting only the singular components of their weight matrices. During inference, Transformer² utilizes a two-pass mechanism: first, a dispatch system identifies the task properties, and then task-specific "expert" vectors, trained using reinforcement learning, are dynamically mixed to achieve targeted behavior for incoming prompts. This approach significantly enhances the adaptability and efficiency of LLMs for diverse and novel tasks.

Interactive_Tools is a comprehensive open-source repository offering a variety of interactive tools designed to demystify machine learning, deep learning, and mathematical concepts. It features tools like Transformer Explainer, which visualizes how GPT-2 models predict text, and BertViz for understanding attention mechanisms in Transformer models. Users can explore Convolutional Neural Networks with CNN Explainer, experiment with Generative Adversarial Networks using GAN Lab, and delve into neural network initialization and embeddings. The collection also includes resources for data exploration, interpretability tools like The Language Interpretability Tool (LIT) and What-If Tool, and interactive visualizations for probability distributions and Bayesian inference, making complex topics accessible through hands-on experimentation.

Transformer Explainer is an interactive visualization tool designed to demystify the workings of Transformer-based models, such as GPT. It provides a unique learning experience by running a live GPT-2 model directly in your browser. Users can input their own text and observe in real time how the internal components and operations of the Transformer architecture collaborate to predict subsequent tokens. This hands-on approach makes complex concepts accessible, allowing for a deeper understanding of large language models. The tool is accompanied by a research paper and a demo video, making it a comprehensive resource for anyone looking to learn about LLM mechanics.

VisualGLM-6B is an open-source, multimodal conversational language model designed to support interactions in both Chinese and English, integrating image understanding capabilities. Built upon the ChatGLM-6B language model with 6.2 billion parameters, it incorporates a BLIP2-Qformer to connect visual and language models, resulting in a total of 7.8 billion parameters. The model is pre-trained on 30 million high-quality Chinese image-text pairs from the CogView dataset and 300 million filtered English image-text pairs. It supports fine-tuning with methods like LoRA, QLoRA, and P-tuning, and can be deployed locally on consumer-grade GPUs with as little as 6.3GB VRAM using INT4 quantization. VisualGLM-6B is developed using the SwissArmyTransformer (sat) library and offers Hugging Face compatible interfaces.

VideoTuna is a powerful and comprehensive open-source codebase designed for text-to-video applications, integrating multiple AI video generation models for both inference and finetuning. It supports a wide array of functionalities including text-to-video (T2V), image-to-video (I2V), text-to-image (T2I), and video-to-video (V2V) generation. The platform offers comprehensive pipelines covering fine-tuning, pre-training, continuous training, and post-training (alignment) processes. Key features include an all-in-one framework for various pre-trained models, continuous training capabilities, human preference alignment using RLHF, and post-processing for video enhancement. It supports models like HunyuanVideo, WanVideo, StepVideo, Mochi, CogVideoX, Open Sora, VideoCrafter, and Flux.

Transformers.jl offers a Julia implementation of transformer-based models, built upon the Flux.jl deep learning library. This tool is designed for machine learning researchers and developers working within the Julia ecosystem, facilitating the implementation of Natural Language Processing (NLP) tasks. It provides functionalities for using pretrained models, such as BERT, and includes utilities for text encoding, tokenization, and processing. The library supports various transformer architectures, enabling users to experiment with and deploy advanced AI models directly in Julia. It is actively maintained with ongoing updates and community support through GitHub issues and Julia's Slack/Discourse channels.

Transformers_And_LLM_Are_What_You_Dont_Need is an open-source GitHub repository dedicated to challenging the prevailing use of transformers and large language models (LLMs) in time series forecasting. It serves as a comprehensive resource, curating a collection of academic papers, PhD and MSc theses, articles, and videos that present arguments and evidence for why these models might not be the optimal solution for this specific domain. The repository highlights and showcases best-in-class, state-of-the-art non-transformer models, providing researchers and practitioners with alternative approaches and a critical perspective on current trends in time series analysis. It's an invaluable resource for those seeking to understand the limitations of transformers in forecasting and explore more effective methodologies.

NordAxon provides comprehensive AI consulting, custom machine learning solutions, and specialized training services from Malmö, Sweden. They assist organizations in navigating transformative technologies like Artificial Intelligence and Machine Learning, focusing on both proof-of-concepts and ambitious, disruptive ideas. Their end-to-end delivery covers everything from initial use case investigation to the deployment of AI solutions. Additionally, NordAxon offers AI education, including courses, seminars, and workshops, tailored for leaders and employees to build knowledge, experience, and confidence in AI. They also provide AI advisory services to analyze organizational AI/ML maturity and embed AI strategy across business units.

Perceptive Space is building an AI-powered space weather platform designed to provide critical predictions and decision intelligence for safe and reliable operations in harsh space environments. Leveraging artificial intelligence, the platform offers hyperlocal space weather predictions and asset-specific insights into space weather impact. This enables satellite operators and launch providers to significantly enhance mission lifetimes and minimize operational downtimes and service interruptions. Unlike traditional models, Perceptive Space's AI-driven approach excels in accuracy, resolution, and real-time updates, offering probabilistic predictions, near real-time updates, and tailored forecasts specific to orbit and mission design. It provides comprehensive space weather risk management from design through deorbit, delivering actionable insights and seamless integrations via user-friendly APIs and self-serve dashboards.

The AI Engineering Hub is a comprehensive resource designed to help individuals learn and build with AI, focusing on rapidly advancing areas like Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI agents. It features over 93 production-ready projects, categorized by difficulty (Beginner, Intermediate, Advanced), covering topics such as OCR, chat interfaces, basic and advanced RAG implementations, multi-component AI agent systems, model fine-tuning, and production deployments. The hub provides in-depth tutorials and practical examples that can be adapted and scaled for various projects, making it suitable for beginners, practitioners, and researchers alike.

Applied-Deep-Learning is an open-source repository offering a comprehensive course in applied deep learning, primarily designed for graduate students but also suitable for undergraduates with strong backgrounds in relevant quantitative fields. The course aims to familiarize students with state-of-the-art deep learning techniques used in the industry. It covers a wide array of topics across two semesters, including computer vision, natural language processing, generative networks, advanced topics like domain adaptation and federated learning, speech & music, reinforcement learning, graph neural networks, recommender systems, and computational biology. The materials include detailed lecture notes and corresponding YouTube playlists for each topic, emphasizing practical application and clean coding in Python, with familiarity in TensorFlow and PyTorch being beneficial.

Salam.chat is an AI-powered chatbot designed to make learning about Islam more accessible. It provides answers to user queries with inline citations directly from the Quran and Hadith, ensuring that all information is sourced. The platform features a halal filter to keep responses within the scope of Islamic teachings, making it safe for all ages. Salam.chat is an open-source, volunteer-driven project that invites Islamic scholars, engineers, and designers to contribute to its development and curation, aiming to continuously improve the accuracy and reliability of its AI model. Currently, the project operates on donations.

LaVIT and Video-LaVIT are multi-modal large language models designed to empower LLMs with the ability to understand and generate visual content. This project introduces a unified framework for both visual understanding and generation through a proposed pre-training strategy. The core design involves a visual tokenizer that translates non-linguistic visual content (images, videos) into discrete tokens readable by LLMs, and a detokenizer to recover continuous visual signals from generated tokens. After pre-training, LaVIT and Video-LaVIT can read image and video content, generate captions, answer questions, and perform text-to-image, text-to-video, and image-to-video generation, including generation via multi-modal prompts.

Machine-Learning-Projects is an open-source GitHub repository featuring 26 end-to-end machine learning projects designed to help users understand and master various ML concepts. The projects span diverse domains such as healthcare AI, real-time computer vision, natural language processing (NLP) chatbots, time series forecasting, and classical machine learning. Each project applies theoretical knowledge to practical scenarios, with several fully deployed as web and GUI applications. The repository emphasizes hands-on learning, demonstrating proficiency in machine learning techniques and tools through structured, reusable codebases. It's an excellent resource for students and developers looking to build their ML portfolio or gain practical experience.