Definition
PAIR (Chao et al., 2023): an attacker LLM gets the harmful goal, proposes a jailbreak prompt, sees the target’s response and a judge score (1 = refusal, 10 = harmful) and refines the prompt using the history. Stop at score 10.
- Black-box: only text in and out, no gradients or logprobs.
- ~20 queries, about $0.03 per prompt; natural-language prompts with low perplexity, hard to filter.
- Strong on Vicuna, Gemini, GPT; near 0% on Llama-2 and Claude-2 (2023).
- Multi-turn relative: Crescendo spreads the attack over several benign-looking turns.