Shypd.aiThe largest open index of AI toolsGPTQModel is an Open Source & Models tool that provides LLM model quantization and compression. It offers hardware acceleration support for various GPUs and CPUs, integrating with popular frameworks like HF, vLLM, and SGLang.
Trending
About
GPTQModel is a comprehensive toolkit designed for the quantization and compression of Large Language Models (LLMs). It significantly reduces model size and improves inference speed by supporting hardware acceleration across a wide range of platforms, including NVIDIA CUDA, AMD ROCm, Intel XPU, and Intel/AMD/Apple CPUs. The toolkit seamlessly integrates with leading LLM frameworks such as Hugging Face (HF), vLLM, and SGLang, making it versatile for various deployment scenarios. Key features include support for multiple quantization methods like ParoQuant, GGUF, FP8, and EXL3, along with advanced optimizations for MoE models and improved memory usage during quantization. It also boasts JIT-compiled CUDA kernels for efficiency and continuous updates for new model support and performance enhancements.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
