Shypd.aiThe largest open index of AI toolsOpenMoE is an open-source Mixture-of-Experts (MoE) Large Language Model project. It provides a family of open-sourced MoE LLMs for research and development, with models available on Huggingface.
Trending
About
OpenMoE is an open-source project dedicated to fostering the Mixture-of-Experts (MoE) Large Language Model community. Initiated in summer 2023, it offers a family of MoE LLMs, including intermediate checkpoints and fully trained models like OpenMoE-8B and OpenMoE-8B-Chat. The project emphasizes full transparency, sharing training data, strategies, model architecture, and weights. It supports inference with PyTorch via ColossalAI and provides tutorials for both TPU and GPU environments. OpenMoE models are evaluated on benchmarks like MT-Bench and BigBench-Lite, demonstrating competitive performance. The project encourages contributions to advance open-source MoE research.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
