Definition
Unlearning tries to remove hazardous knowledge (proxy benchmark: WMDP, 3,668 bio/cyber/chem questions) from the weights, so open-weight models can’t be used for it.
- RMU: push hidden states on forget data towards a scaled random vector , keep them equal to the frozen model on retain data.
- TAR: bilevel training: simulate -step fine-tuning attacks in the inner loop and backpropagate through them so the weaponization loss stays bad for the attacker; a retain loss matches the base model’s hidden states.
Suppression, not deletion
10 retain examples bring RMU’s WMDP-Bio accuracy from 29.9 back to 62.4; direction removal and GCG recover more. TAR falls to fine-tuning with new hyperparameters (~60% vs. 28%). The knowledge is still in the representation.
Appears in
- Lecture 4, WMDP and RMU
- Lecture 4, unlearning is brittle
- Lecture 4, TAR
- Lecture 4, TAR broken
- Lecture 6, The WMDP Benchmark: the benchmark and bio capabilities
- Lecture 7, Deep Ignorance: filtering pretraining data instead of unlearning