Definition
SFT trains a base model on ~10K to 100K prompt-response demonstrations. It teaches the chat template: role markers like
<|start_header_id|>and<|eot_id|>are single tokens whose meaning the model learns only here.
- Base model: continues the pattern (βWhat is the capital of Germany? β¦β). Instruct model: answers (βThe capital of France is Paris.β).
- Because roles are just tokens, fake format tokens in user input can confuse them (misdirection, system prompt extraction).
Appears in
- Lecture 2, SFT
- Lecture 2, chat roles
- Lecture 8, CodeActInstruct: distilling filtered agent trajectories