Shypd.aiThe largest open index of AI toolsllm.c is an Open Source AI Frameworks & Infra tool that enables LLM training in simple, raw C/CUDA. It focuses on pretraining and reproducing GPT-2 and GPT-3 miniseries, offering a faster alternative to PyTorch.
Trending
Also listed in
About
llm.c is an open-source project designed for training Large Language Models (LLMs) using simple, raw C/CUDA, aiming to provide a lightweight and efficient alternative to frameworks like PyTorch. The project's primary focus is on pretraining, specifically reproducing the GPT-2 and GPT-3 miniseries. It includes a parallel PyTorch reference implementation (a tweaked nanoGPT) for comparison, and currently boasts a performance edge over PyTorch Nightly. The repository offers a clean, ~1,000-line CPU fp32 implementation in C, alongside bleeding-edge CUDA code. It supports single and multi-GPU training, multi-node training, and integrates with libraries like cuBLAS, cuBLASLt, CUTLASS, and cuDNN for optimized performance. The project also serves an educational purpose, providing documented kernels and tutorials for understanding LLM layer implementations.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
