Shypd.aiThe largest open index of AI toolsLarge Concept Models (LCM) is an open-source AI tool for language modeling in a sentence representation space. It provides official implementations and experiments for building and finetuning models.
Trending
About
Large Concept Models (LCM) is an open-source project by Facebook AI Research, offering official implementations and experimental setups for language modeling within a sentence representation space. It operates on explicit higher-level semantic representations, termed "concepts," which are language- and modality-agnostic. The current work defines a concept as a sentence, utilizing the SONAR embedding space that supports up to 200 languages for text and 57 for speech. The LCM is a sequence-to-sequence model in the concept space, trained for auto-regressive sentence prediction. It explores approaches like MSE regression and diffusion-based generation, with models up to 1.6 billion parameters trained on 1.3 trillion tokens. The repository includes recipes for reproducing training and finetuning of both MSE and Two-tower diffusion LCMs.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
