Adversarial Training

Formula

$$\underset{\theta }{\min }{\mathbb{E}}_{(x,y)}\left[\underset{\Vert \hat{x}-x{\Vert }_p\le \epsilon }{\max }\mathcal{L}\right(f_{\theta }(\hat{x}),y\left)\right]$$

Inner max: PGD finds the worst case per batch. Outer min: SGD on it.

• Costs $K\times$ compute and more data.
• Clean accuracy drops (CIFAR-10: 87.3% clean, 45.8% under PGD-20); robust models ignore predictive but non-robust features.
• Side effect: interpretable input gradients.
• In LLMs: hard-negative mining (keep mining jailbreaks, train refusals on them); the cost shows up as overrefusal.

Appears in

• Lecture 3, min-max
• Lecture 3, interpretable gradients
• Lecture 3, tradeoff
• Lecture 3, LLM version
• Lecture 4, TAR: adversarial training against fine-tuning attacks
• Lecture 5, Modern Industrial Detectors: hard-negative mining for detectors
• Lecture 6, Sleeper Agents: red-team training teaches the model to hide the trigger

Related concepts

• Adversarial Example
• Adaptive Attack