Yarn

Visit Tool

yarn is an open-source tool for efficient context window extension of large language models. It provides code and data for the YaRN method, including Llama 2 and Mistral fine-tuned variants with extended context windows.

Claim this tool

1View

At a glance

Pricing

Open Source

Free tier

Yes

API

Skill level

Technical

About

What is yarn?

YaRN (Yet another RoPE-scaling method) is an open-source project providing code and data for efficiently extending the context window of large language models. It offers fine-tuned variants of popular models like Llama 2 and Mistral, with context window lengths up to 128K. The project emphasizes open science, publishing all code and data necessary to reproduce the results presented in their ICLR 2024 paper. Developers can access these models on Hugging Face under the Llama 2 license. The repository includes scripts for training and evaluation, supporting DeepSpeed acceleration for efficient model training. This tool is ideal for researchers and developers looking to experiment with or implement long-context LLMs.

Best used for

Ideal for developers and data scientists who need to extend the context window of large language models, fine-tune existing models for longer contexts, and reproduce research results. Especially valuable for those working on applications requiring extensive contextual understanding or experimenting with advanced LLM architectures.

Common actions

extend LLM context

fine-tune language models

reproduce AI research

develop long-context applications

deepfakeworkflowsautomated workflowopen-sourcelow-code/no-codecollaborationgithub copilot"AI Agents"face swapping

Capabilities

Key features

Context window extension
Llama 2 fine-tuned models
Mistral fine-tuned models
SOLAR fine-tuned models
Reproducible training code
Evaluation scripts

Target Audience

developerdata scientist

Integrations

Not yet documented

Pricing & Plans

Open Source

Free

FAQs

What is the YaRN method?

YaRN (Yet another RoPE-scaling method) is a technique designed to efficiently extend the context window of large language models. It allows models to process and understand significantly longer sequences of text than their original design, improving performance on long-context tasks.

Which large language models are supported by YaRN?

YaRN provides fine-tuned variants for several popular large language models, including Llama 2 (7B, 13B, 70B), Mistral (7B), and SOLAR (10.7B). These models are available with extended context window lengths, such as 32K, 64K, and 128K.

Where can I access the fine-tuned YaRN models?

The fine-tuned YaRN models are published and available on Hugging Face. They are released under the Llama 2 license, allowing researchers and developers to download and utilize them for their projects and experiments.

Trending

Subcategories trending in Coding & Development

Code Assistants DevOps & Infrastructure No-Code / Low-Code Testing & QA Backend & APIs Prompt Engineering

Trending

Also listed in

This tool also appears in

AI Agents & Automation › AI Frameworks & Infra

Explore

Browse AI tools by category

Content & Design Productivity & Business Coding & Development AI Agents & Automation Research & Education Wellness & Lifestyle Career Development Marketing & Growth Data & Analytics Customer Support & CX Finance E-commerce