Hqq

Visit Tool

hqq is an open-source implementation of Half-Quadratic Quantization (HQQ) that reduces the size of large machine learning models. It accelerates inference and optimizes memory usage without requiring calibration data.

Claim this tool

1View

At a glance

Pricing

Open Source

Free tier

Yes

API

Skill level

Technical

About

What is hqq?

hqq provides an official implementation of Half-Quadratic Quantization (HQQ) and HQQ+, designed to efficiently quantize large machine learning models. It supports various bit-widths (8, 4, 3, 2, 1 bits) and can be applied to different model types, including LLMs and Vision models. A key differentiator is its ability to quantize models quickly without the need for calibration data, making it a fast and accurate solution. The tool is compatible with optimized CUDA/Triton kernels for faster inference and supports PEFT training. Users can customize quantization settings for different layers and integrate with popular frameworks like Hugging Face Transformers and vLLM, offering flexibility for both research and deployment.

Best used for

Ideal for developers and data scientists who need to optimize large language models for faster inference, reduce memory footprint, and enable efficient deployment. Especially valuable for those working with Hugging Face Transformers or vLLM who require quick and accurate model quantization without calibration data.

Common actions

quantize machine learning models

optimize model inference

reduce model memory

train quantized models

low-code/no-codecollaborationworkflowsautomated workflowdeepfakeopen-source"AI Agents"github copilotface swapping

Capabilities

Key features

Half-Quadratic Quantization (HQQ)
HQQ+ for lower bits
8,4,3,2,1-bit quantization
No calibration data needed
Fast model quantization
Optimized inference backends
PEFT training compatible

Target Audience

developerdata scientist

Integrations

hugging-face-transformersvllm

Pricing & Plans

Open Source

Free

FAQs

What bit-widths does HQQ support for quantization?

HQQ supports a wide range of bit-widths for quantization, including 8, 4, 3, 2, and 1 bits. This flexibility allows users to choose the optimal balance between model size, performance, and accuracy for their specific applications.

Does HQQ require calibration data for quantization?

No, HQQ is designed to be a fast and accurate model quantizer that skips the need for calibration data. This significantly simplifies the quantization process and reduces the time required to optimize models.

Is HQQ compatible with PEFT training?

Yes, HQQ is compatible with PEFT (Parameter-Efficient Fine-Tuning) training. It integrates directly with Hugging Face's PEFT library, allowing users to fine-tune quantized models efficiently.

What are the recommended quantization settings for HQQ?

For a good balance of quality, VRAM usage, and speed, the recommended starting settings are nbits=4, group_size=64, and axis=1. Users can adjust these parameters for better results or specific performance needs.

Trending

Subcategories trending in Coding & Development

Code Assistants DevOps & Infrastructure No-Code / Low-Code Testing & QA Backend & APIs Prompt Engineering

Trending

Explore

Browse AI tools by category

Content & Design Productivity & Business Coding & Development AI Agents & Automation Research & Education Wellness & Lifestyle Career Development Marketing & Growth Data & Analytics Customer Support & CX Finance E-commerce