Shypd.aiThe largest open index of AI toolsBitBLAS is an open-source library that supports mixed-precision matrix multiplications on GPUs, especially for quantized LLM deployment. It enables efficient low-precision deep learning computing through hardware-aware tensor transformation.
Trending
About
BitBLAS is an open-source library designed to facilitate efficient mixed-precision DNN model deployment on GPUs. It specializes in mixed-precision BLAS operations, particularly for $W_{wdtype}A_{adtype}$ quantization in large language models (LLMs). Key features include high-performance matrix multiplication for both GEMV and GEMM, supporting various mixed-precision types like FP16xFP8/FP4/INT4/2/1 and INT8xINT4/2/1. BitBLAS also offers auto-tensorization for TensorCore-like hardware instructions and provides integrations with popular frameworks such as PyTorch, GPTQModel, AutoGPTQ, vLLM, and BitNet-b1.58. Based on techniques from the "Ladder" paper, it allows for customizing mixed-precision DNN operations via a flexible DSL (TIR Script).
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
