Shypd.aiThe largest open index of AI toolsqlora is an Open Source & Models tool that enables efficient finetuning of quantized large language models (LLMs). It significantly reduces memory usage, allowing finetuning of 65B parameter models on a single 48GB GPU.
Trending
Also listed in
About
qlora is an open-source project designed to democratize access to LLM research by providing an efficient finetuning approach for quantized large language models. It integrates with Hugging Face's PEFT and transformers libraries and utilizes bitsandbytes for quantization. Key innovations include 4-bit NormalFloat (NF4) data type, Double Quantization to reduce memory footprint, and Paged Optimizers to manage memory spikes. This allows for finetuning large models, such as a 65B parameter model, on a single 48GB GPU while preserving full 16-bit finetuning task performance. The project also releases the Guanaco model family, which outperforms previous openly released models on the Vicuna benchmark.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
