Decoding and Sampling

Definition

At each step the model outputs a distribution $p=\text{softmax}(W_{U}h)$ over the vocabulary; sampling picks the token.

Strategy	Rule
Greedy	$\arg {\max }_i{p}_i$
Temperature	logits / $T$ before softmax
Top-k	keep the $k$ most likely tokens
Top-p	smallest set with cumulative probability $\ge p_{\text{top}}$
Min-p	keep $p_i\ge p_{\text{base}}\cdot {\max }_j{p}_j$

Safety

Decoding parameters alone moved a misalignment rate from 0% to 95%+ (Huang et al., ICLR 2024), 30× cheaper than GCG.

Appears in

• Lecture 2, unembedding and sampling
• Lecture 2, sampling and safety
• Lecture 3, Best-of-N: sampling many augmented prompts
• Lecture 5, Embedding the Watermark: the watermark changes the sampling distribution

Related concepts

• Jailbreak