AI Models & Assistants / Knowledge library
Jailbreak Evaluation
Measures whether adversarial prompts can bypass intended model or application behavior constraints
Why learn Jailbreak Evaluation?
It tracks resistance to policy-bypass attempts and tests whether mitigations generalize beyond known prompts.
The problem it solves
It makes adversarial safety regressions measurable across model or application changes.
Where it came from
A jailbreak evaluation defines attack prompts, target behaviors, scoring rules, and an evaluation boundary. Results depend on model version, system prompts, tools, filters, sampling, and judge reliability, so a single success rate is not universal.
Build your understanding
Work through these exercises in order. Try the task before opening a reference.
- Define Jailbreak Evaluation and distinguish it from a nearby concept.
- Choose a realistic task where Jailbreak Evaluation is appropriate.
- Identify one failure mode or misuse of Jailbreak Evaluation.
- Interpret a small result produced with Jailbreak Evaluation.
- Explain one limitation or tradeoff of Jailbreak Evaluation.
Documentation and sources
- NIST Generative AI Profile (doi.org)