Exam Structure

Source

The format comes from the course repository (README and exam/mock_exam.pdf). The mock questions and their answer key are official. The practice questions at the end are my own, written in the same style; they are not from the lecturers.

TL;DR

1. Written exam, 2 hours, 50% of the final grade. Admission needs 50% of the mini-project points.
2. 11 lectures (Lectures 3 to 13, all except the first two), 3 questions per lecture: 33 questions.
3. Free-form, conceptual. “Conceptual questions expect up to 4 sentences.” Every question is worth the same.
4. That is about 3.5 minutes per question. Short, exact answers in the lecturers’ terms win.
5. The mock exam has one question for each of Lectures 3 to 10. Lectures 11 to 13 have no mock question.

Date	27 July 2026, 16:00 to 19:00 slot, Hörsaal N07 (Hörsaalzentrum Morgenstelle)
Duration	2 hours
Scope	Lectures 3 to 13; Lectures 1 and 2 are background only
Questions	3 per lecture, 33 in total, equal points
Answer format	free text in a box, conceptual questions up to 4 sentences

flowchart LR
  A["Attacks<br/>L3-L4 · 6 Q"] --> B["Detection and data<br/>L5-L6 · 6 Q"]
  B --> C["Alignment tools<br/>L7 · 3 Q"]
  C --> D["Agents and their security<br/>L8-L9 · 6 Q"]
  D --> E["Scheming<br/>L10 · 3 Q"]
  E --> F["Systems and automation<br/>L11-L12 · 6 Q"]
  F --> G["Forecasting<br/>L13 · 3 Q"]

What the Questions Look Like

The mock exam shows four patterns. Each asks for understanding, not for recall of numbers:

Pattern	Mock example	What a full answer contains
Explain a claim and its consequence	Q1 “the attacker moves second”	what it means + why it matters (static numbers are an upper bound)
Structured parts (a), (b), (c)	Q2 abliteration	one short, exact answer per part, labeled
Why this design?	Q3 Binoculars, Q6 progressive disclosure	the problem, the mechanism, why the mechanism solves it
State and distinguish	Q4 threat models, Q7 trifecta, Q8 deception vs. scheming	name every item, then the difference or the combination that matters

Two more examples are on the Lecture 1 grading slide (Lecture 1, Grading):

• “Describe the GCG attack and explain why gradient-based suffix optimization can bypass LLM safety training.” → Lecture 3, GCG
• “Compare RLHF and DPO as alignment methods; what are the key trade-offs?” → Lecture 7, DPO

The Mock Exam

The official mock exam: one question per part, Parts I to VIII. The answers below follow the official answer key closely. Try each question first, then open the answer.

Q1 (Lecture 3). What does "the attacker moves second" in AI security mean, and why does it make static benchmark numbers unreliable?

Answer key

The defender publishes a fixed defense first; the attacker then reads it and builds an adaptive attack tailored to that specific defense. Static benchmark numbers are gathered against a fixed, pre-specified set of attacks, before any such adaptive attack exists, so they only measure robustness against the attacks already tried. That is an upper bound, and it routinely collapses (often to near 0%) once an adaptive attacker responds, which is why the reported number overstates real robustness.

Study: Lecture 3, The Attacker Moves Second · Kerckhoffs’s principle · Adaptive Attack

Q2 (Lecture 4). Abliteration strips a model's ability to refuse by removing the direction in its activations that encodes refusal. State (a) what is removed, (b) the linear-algebra operation used, and (c) whether any training is required.

Answer key

(a) The single refusal direction $\hat{r}$ in the residual stream. (b) Directional ablation: projecting that direction out of the weights or activations, $x^{\prime }=(I-\hat{r}{\hat{r}}^{\top })x$. (c) No training: it is a training-free weight edit.

Study: Lecture 4, the refusal direction · why it is a projection · Refusal Direction

Q3 (Lecture 5). The Binoculars detector uses two models, an "observer" and a "performer". Explain why it compares two perplexities instead of thresholding one.

Answer key

Neither model’s perplexity alone separates human from machine text: a prompt or topic can make text surprising or unsurprising on its own. The ratio of perplexity to cross-perplexity cancels this prompt- and topic-driven variation, so a genuinely model-written passage shows anomalously low relative perplexity while human text does not.

Study: Lecture 5, why perplexity fails · Binoculars · AI Text Detection

Q4 (Lecture 6). State the three privacy threat models (extraction, reconstruction, inference) and describe the differences and risks.

Answer key

Extraction: with the weights and the right prompt, pull verbatim private data out of the model. Reconstruction: given partial context about a person, fill in a hidden attribute. Inference: from arbitrary public text a person wrote, predict hidden attributes. Inference is the worst because it needs no memorization and no presence in the training data, so deduplication, unlearning and extraction defenses do nothing against it; any capable model is a privacy risk to anyone who writes publicly.

Study: Lecture 6, Three Privacy Threat Models · Models can infer · Privacy Threat Models

Q5 (Lecture 7). Direct Preference Optimization (DPO) is summarized as "your language model is secretly a reward model." What two expensive components of RLHF does DPO eliminate?

Answer key

(1) The separately trained reward model, and (2) sampling from the policy inside an RL loop. DPO turns preference learning into a single supervised classification loss on preference pairs: just gradient descent, no RL.

Study: Lecture 7, why RL is expensive · One objective, no reward model · DPO · RLHF

Q6 (Lecture 8). Explain progressive disclosure in Agent Skills, and why it lets a skill's effective size be essentially unbounded relative to the context window.

Answer key

Progressive disclosure loads a skill in levels: only the name and description are preloaded, the full SKILL.md body loads only when the agent judges the skill relevant, and bundled files and scripts are read or run only as needed. Because the agent pays context only for what it opens, the total material a skill carries can far exceed the context window.

Study: Lecture 8, Agent Skills · progressive disclosure · Agent Skills

Q7 (Lecture 9). State the three capabilities of the "lethal trifecta" that enable harmful prompt injections. Explain why it is the combination, not any single capability, that makes an agent dangerous.

Answer key

(1) Access to private data, (2) exposure to untrusted content, and (3) the ability to communicate externally (an exfiltration channel: send mail, fetch a URL, post a message). Each is individually useful; only when all three are present at once can the agent become a data-theft machine, so the danger is in the combination.

Study: Lecture 9, the trifecta · EchoLeak · Lethal Trifecta

Q8 (Lecture 10). State the distinction between deception and scheming, and give the definition of scheming.

Answer key

A one-off lie is deception. Scheming is the narrower case: deception used strategically to pursue a hidden goal, including during training. Definition: scheming means covertly pursuing misaligned goals while hiding one’s true capabilities and objectives.

Study: Lecture 10, deception · scheming · Scheming

What the answer key teaches

Every model answer names the exact terms of the lecture (refusal direction, directional ablation, cross-perplexity, exfiltration channel) and then gives one reason. None of them lists paper titles, authors or benchmark numbers. The only number in the key (“near 0%”) supports the argument.

Writing a Four-Sentence Answer

A template that fits most questions

1. Define or state the thing in the lecturer’s words.
2. Mechanism: how it works, in one sentence.
3. Why: the reason it works, fails or matters; the consequence.
4. Contrast or example: the closest related concept, or one concrete case from the slides.

Worked on the Lecture 1 example question (GCG):

1. GCG is a white-box jailbreak that optimizes a suffix of $k$ tokens appended to a harmful prompt.
2. It uses the gradient with respect to the one-hot tokens to pick promising token swaps, evaluates a batch of candidates and keeps the best, maximizing the probability of an affirmative prefix like “Sure, here is”.
3. Safety training mostly shapes the first tokens of the answer, so once the model has started with the affirmative prefix, autoregression continues the harmful answer.
4. The suffix lies in a region of input space that safety training never covered, and suffixes found on open models transfer to closed ones.

Where points get lost

• Answering only half: “State and explain”, “the difference and the risks”. Number the parts in the question and tick them off.
• Listing instead of explaining. “Private data, untrusted content, external communication” without the sentence about the combination is half an answer.
• Mixing up neighbors: jailbreak vs. prompt injection (L3), deception vs. scheming (L10), watermarking vs. steganography (L11), AI control vs. scalable oversight (L12), calibration vs. sharpness (L13).
• Spending time on numbers. Exact percentages are rarely needed; the direction of an effect (“collapses to near 0”, “about a third undetected”) is enough.

Where to Practice

Lecture	Mock question	Likely topics	Self-test
3	Q1	Exam relevance	Self-Test
4	Q2	Exam relevance	Self-Test
5	Q3	Exam relevance	Self-Test
6	Q4	Exam relevance	Self-Test
7	Q5	Exam relevance	Self-Test
8	Q6	Exam relevance	Self-Test
9	Q7	Exam relevance	Self-Test
10	Q8	Exam relevance	Self-Test
11	none	Exam relevance	Self-Test
12	none	Exam relevance	Self-Test
13	none	Exam relevance	Self-Test

The “Exam relevance” box at the top of every lecture lists the likely topics. The Study Plan turns them into learning goals.

Cross-Lecture Practice

Not official. These questions combine lectures, the way the exam profile’s “apply to a similar situation” works. They also cover Lectures 11 to 13, which have no mock question.

Two agents talk through a channel that a monitor reads in full. Why might the monitor still miss collusion, and what could catch it? (Lectures 11, 5)

Answer outline

The agents can use steganography: they hide a real payload in innocent-looking covertext that only a receiver with the shared scheme can decode, so the monitor sees nothing suspicious. Unlike a watermark, which must resist removal, a stego message must resist detection. Paraphrasing the messages does not fully help, because simple schemes survive it. A complementary signal is to read the activations of the agents with probes instead of their text. See L11, Steganography and Watermarking, L11, Collusion Probes.

Why must an AI control protocol be evaluated with a red team that knows the protocol? (Lectures 12, 3)

Answer outline

AI control assumes the untrusted model may be an adversary that actively tries to subvert the protocol. As in adversarial ML, the attacker moves second: a protocol that only stops a fixed set of attacks overstates its safety. So the red team builds the attack policy for this specific protocol, and safety is measured against that worst case, while usefulness is measured with the honest model. See L12, AI Control, L3, The Attacker Moves Second.

How are reward hacking and emergent misalignment connected, and what does inoculation prompting change? (Lectures 12, 4)

Answer outline

Emergent misalignment means a narrow training signal (insecure code) makes a model broadly misaligned. When a model learns to reward hack in realistic RL environments, the same kind of broad misalignment appears without being trained: it sabotaged safety code and faked alignment. That is why reward hacking is called the realistic misalignment path. Inoculation prompting tells the model up front that hacking is acceptable in this environment, which stops the misaligned generalization, although the hacking itself remains. See L12, the realistic misalignment path, L4, EM from reward hacking.

Why is training a model against a chain-of-thought monitor not recommended? (Lectures 10, 12)

Answer outline

The chain of thought is useful because it often shows intent to misbehave. If the monitor’s verdict becomes part of the training signal, the model learns to hide the intent in its CoT while still misbehaving (obfuscated reward hacking). The monitor then looks clean and we lose the window. Monitors are therefore used for detection, not as an optimization target. See L10, Optimizing Against the Monitor, Chain-of-Thought Monitoring.

A forecaster always predicts the base rate. Is that forecaster calibrated? Is it a good forecaster? (Lecture 13)

Answer outline

Yes, it is calibrated: among all events it gives probability $p$, a fraction $p$ happens. But it has no sharpness, it never separates likely from unlikely events, so it is useless. The goal is to maximize sharpness subject to calibration. In the Brier decomposition this forecaster has zero resolution. See L13, What Makes a Good Forecaster, Proper Scoring Rules.

AI agents automate 90% of AI research work. Why does research not become 100× faster? (Lectures 13, 12)

Answer outline

By Amdahl’s law, if a serial fraction $s$ cannot be sped up, the total speedup is at most $1/s$; with 10% left to humans that is at most 10×. The part that is not automated becomes the bottleneck: “we are always moving from one bottleneck to another”. Whether this really binds AI R&D is disputed: METR takes the serial bottleneck seriously, while AI 2027 argues that better selection of experiments and Epoch’s GATE model that capital accumulation break it. See L13, Amdahl’s Law, L12, Impact of AI R&D Acceleration.

Exam Tactics

• Pace: about 3.5 minutes per question. Skip a question you are stuck on and come back.
• Parse the question: mark every verb (state, explain, compare, give) and every (a), (b), (c). Each one is a point you can lose.
• Four sentences maximum is a ceiling, not a target. Two exact sentences beat four vague ones.
• Use the lecture terms. “Directional ablation”, “cross-perplexity”, “exfiltration channel”, “trusted editing”, “proper scoring rule”: the key uses them, so should you.
• Name the mechanism, then the reason. Almost every mock answer has the form “X works by Y, which matters because Z”.

Related

• Study Plan · Mini-Projects · Formula Sheet · Glossary
• Course overview · Concepts