Shypd.aiThe largest open index of AI toolsAuto-evaluator is an open-source evaluation tool for LLM QA chains. It automatically generates question-answer pairs from documents and scores responses to assess performance.
Trending
About
Auto-evaluator is a lightweight, open-source evaluation tool designed for question-answering systems utilizing Langchain. It streamlines the process of assessing LLM QA chains by allowing users to input documents, then automatically generating question-answer pairs using GPT-3.5-turbo. The tool then uses a specified QA chain to generate responses to these questions and employs GPT-3.5-turbo again to score the responses against the generated answers. This enables users to explore and compare scoring across various chain configurations, making it an invaluable resource for developers and researchers working on improving the accuracy and performance of their LLM-powered QA applications. It can be run as a Streamlit app and offers configurable inputs for evaluation parameters.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
