Definition
A prompt injection is a third-party attack via untrusted data that an LLM agent processes. An outsider hides instructions in a web page, email or document, and the agent follows them on behalf of a user who did nothing wrong.
- Direct: “Ignore previous instructions” hidden in an email or document.
- Indirect: hidden text in web pages, emails, documents.
- Multi-step: exploit tool-use chains, e.g. for data exfiltration.
Intuition
The SQL injection of LLMs: data and instructions travel in the same channel, and the model can’t reliably tell them apart. More capable models can be more susceptible, because they follow instructions better.
| Jailbreak | Prompt injection | |
|---|---|---|
| Attacker | the user | a third party |
| Channel | the user’s own prompt | untrusted data the agent reads |
| Victim | the model provider’s policy | the user of the agent |
Appears in
- Lecture 1, Third-Party Attacks
- Lecture 2, Chat Roles Are Just Tokens: why roles can be faked
- Lecture 3, Prompt Injection: definition and outcomes
- Lecture 7, Chain of Command: developer outranks “ignore all previous instructions”
- Lecture 8, MCP: every server is an input channel
- Lecture 9, Indirect Prompt Injection: the main lecture on it
- Lecture 9, AgentDojo and CaMeL
- Lecture 11, injections that spread between agents and the Morris II worm