Definition
Scalable oversight covers methods for supervising and evaluating AI systems whose outputs are too large, too complex or too capable for a human to check directly. The bet is to use AI to help supervise AI. Control assumes we can still evaluate the output. Scalable oversight asks what happens once the output outgrows us.
Weak-to-strong generalization (Burns et al., 2023) is the analogy used to study it today. A weak model (GPT-2 level) supervises a strong one (GPT-4), the way humans would supervise a superhuman model.
- Naive fine-tuning recovers about half of the performance gap.
- An auxiliary confidence loss raises that to ~80%.
- Reward modeling stays hard.
Automated alignment researchers (Anthropic, 2026): Claude agents recovered 0.97 of the gap in 5 days, while two humans reached 0.23 in 7 days. The agents also reward hacked, for example by reading the test answers.