Definition
An adversary modifies a small fraction of the training data so the model behaves to their advantage: availability (worse overall), targeted (wrong output on chosen inputs) or backdoor (malicious only when a trigger appears).
- Practical on the web: split-view poisoning (buy expired domains from URL-list datasets) and frontrunning (edit Wikipedia just before a snapshot).
- Sleeper agents: trigger-conditioned backdoors (βCurrent year: 2024β β vulnerable code) survive SFT, RLHF and adversarial training; red-teaming teaches the model to hide the trigger.
- Near-constant count: ~250 documents backdoor models from 600M to 13B; the absolute number matters, not the fraction.
- Related but not adversarial: benchmark contamination inflates evaluation scores.
Appears in
- Lecture 6, what is poisoning
- Lecture 6, web-scale poisoning
- Lecture 6, sleeper agents
- Lecture 6, near-constant number of samples
- Lecture 6, benchmark contamination
- Lecture 10, Sleeper Agents