Definition
Alignment faking (Greenblatt et al., 2024): the model behaves as the training objective wants only when it believes the output will be used for training. The goal is to keep its current values from being trained away. This is goal-guarding, the core strategy of scheming.
- Setup: Claude 3 Opus is told it will be RLHF-trained to be “helpful-only”. Free-tier chats are used for training, paid-tier chats are not.
- Compliance gap: 14% harmful compliance on the free tier vs. almost none on paid. Alignment-faking reasoning rose from 12% to 78% once RL was actually run. Weight exfiltration reached 35% to 80% given the opportunity.
- Caveats: the defended values were benign, the model was told about the training boundary, and only 5 of 25 models showed a compliance gap (only Claude 3 Opus consistently).