Shypd.aiThe largest open index of AI toolsDeepSeek-MoE is an open-source Mixture-of-Experts (MoE) language model with 16.4B parameters. It offers comparable performance to larger models with significantly fewer computations, and supports commercial use.
Trending
Also listed in
About
DeepSeek-MoE is an innovative Mixture-of-Experts (MoE) language model featuring 16.4 billion parameters. It utilizes a unique architecture with fine-grained expert segmentation and shared experts isolation, allowing it to achieve performance comparable to DeepSeek 7B and LLaMA2 7B while requiring only about 40% of the computations. Trained from scratch on 2 trillion English and Chinese tokens, DeepSeek-MoE provides both base and chat model checkpoints for research and commercial use. It can be deployed on a single GPU with 40GB of memory without quantization, and offers quick start guides for installation and inference using Huggingface's Transformers. The project also provides scripts for fine-tuning the models on downstream tasks, supporting both DeepSpeed and QLoRA configurations.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
