Shypd.aiThe largest open index of AI toolsLLM-Shearing is an open-source tool for accelerating language model pre-training through structured pruning. It provides base models and instruction-tuned models for efficient development.
Trending
About
LLM-Shearing is an open-source tool developed by Princeton NLP for accelerating language model pre-training through structured pruning. It offers base models like Sheared-LLaMA-1.3B and Sheared-Pythia-160m, as well as pruned models without continued pre-training and instruction-tuned models such as Sheared-LLaMA-1.3B-ShareGPT. The tool provides a codebase for pruning and continued pre-training algorithms, demonstrating that pruning strong base models is a cost-effective way to achieve powerful small-scale language models. It includes detailed instructions for installation, data preparation, model conversion, and sample scripts for pruning and continued pre-training, built upon MosaicML's Composer package.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
