Shypd.aiThe largest open index of AI toolsLLM2Vec is an open-source coding and development tool that converts decoder-only Large Language Models (LLMs) into powerful text encoders. It provides a simple recipe for enabling bidirectional attention and training with masked next token prediction and unsupervised contrastive learning.
Trending
About
LLM2Vec is an open-source project that offers a straightforward method to transform decoder-only Large Language Models (LLMs) into highly effective text encoders. This process involves three key steps: enabling bidirectional attention, training with masked next token prediction (MNTP), and applying unsupervised contrastive learning (SimCSE). The resulting models can be further fine-tuned to achieve state-of-the-art performance in various natural language processing tasks. LLM2Vec supports popular LLMs like Meta-Llama-3, Mistral-7B, and Llama-2-7B, and provides pre-trained checkpoints for both unsupervised and supervised variants. It includes scripts for training, evaluation on benchmarks like MTEB, and integration with HuggingFace models, making it a valuable resource for researchers and developers working with text embeddings.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
