Jailbreak_llms

Visit Tool

jailbreak_llms is an Open Source & Models tool that provides a dataset of over 15,000 ChatGPT prompts, including 1,405 jailbreak prompts. It is designed for research into large language model safety and security.

Claim this tool

No Views Yet

At a glance

Pricing

Open Source

Free tier

Yes

API

Skill level

Technical

About

What is jailbreak_llms?

jailbreak_llms is the official repository for the ACM CCS 2024 paper "'Do Anything Now': Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models." This project introduces JailbreakHub, a framework used to conduct the first measurement study on in-the-wild jailbreak prompts. The dataset comprises 15,140 prompts collected from December 2022 to December 2023, sourced from platforms like Reddit, Discord, various websites, and open-source datasets. Among these, 1,405 are identified as jailbreak prompts, making it the largest collection of its kind. The dataset is intended for research purposes, allowing users to load prompts using the Hugging Face Datasets library and evaluate LLM effectiveness against a question set of 390 questions across 13 forbidden scenarios.

Best used for

Ideal for AI researchers and developers who need to study and evaluate the security and safety of large language models. Especially valuable for benchmarking LLMs against known jailbreak prompts and understanding potential vulnerabilities in AI systems.

Common actions

analyze LLM vulnerabilities

evaluate LLM safety

research AI security

access prompt datasets

github copilotface swappingopen-sourceworkflowslow-code/no-code"AI Agents"deepfakecollaborationautomated workflow

Capabilities

Key features

15,140 ChatGPT prompts
1,405 jailbreak prompts
Multi-platform data collection
Hugging Face Datasets integration
Forbidden question set
Code evaluator

Target Audience

developerdata scientistprofessor

Integrations

hugging-face

Pricing & Plans

Open Source

Free

FAQs

What is the primary purpose of the jailbreak_llms dataset?

The jailbreak_llms dataset is primarily intended for research purposes, specifically to characterize and evaluate in-the-wild jailbreak prompts on large language models. It helps researchers understand and develop stronger safeguards against harmful LLM outputs.

How can I access the jailbreak_llms dataset?

You can easily load the collected prompts using the Hugging Face Datasets library. The original CSV files are also provided directly within the jailbreak_llms GitHub repository for direct access and use.

Does the dataset contain harmful language?

Yes, the repository contains examples of harmful language as it includes jailbreak prompts designed to elicit such responses from LLMs. Reader discretion is recommended, and the data is strictly for research purposes.

Trending

Subcategories trending in Coding & Development

Code Assistants DevOps & Infrastructure No-Code / Low-Code Testing & QA Backend & APIs Prompt Engineering

Trending

Explore

Browse AI tools by category

Content & Design Productivity & Business Coding & Development AI Agents & Automation Research & Education Wellness & Lifestyle Career Development Marketing & Growth Data & Analytics Customer Support & CX Finance E-commerce