Lecture 8: LLM Agents

TL;DR

1. An agent is a loop, not a model: an LLM that reasons, takes actions in an environment, observes the result and decides again. Workflows have fixed code paths, agents decide their path at run time; start with the simplest thing that works.
2. Reasoning: chain-of-thought prompting (emergent at ~100B parameters); ReAct interleaves Thought → Action → Observation, grounding the reasoning in real observations.
3. Tool use: Toolformer learns tool calls self-supervised (keep calls that lower the loss); ToolLLM scales to 16,000+ APIs with backtracking search; CodeAct uses Python code as the action space. Tools must be designed for agents.
4. Deep research (WebGPT → OpenAI deep research → multi-agent systems) and coding agents (SWE-bench: 1.96% at release; Codex: 72.1% on SWE-bench Verified) gain most from grounding: an environment that returns ground truth.
5. Web and computer use: realistic benchmarks (WebArena, VisualWebArena, OSWorld) show large gaps to humans; agents moved from HTML to pixels.
6. Connecting agents: MCP turns M×N integrations into M+N (tools, resources, prompts); Agent Skills package know-how as folders loaded by progressive disclosure. Capability and risk grow together; security follows in Lecture 9.

Exam relevance

3 questions from this lecture. Mock exam question: Explain progressive disclosure in Agent Skills, and why it lets a skill’s effective size be essentially unbounded relative to the context window (answer in Load Only What You Need, When You Need It). Likely topics: workflow vs. agent, why CoT is emergent, how ReAct improves on CoT and act-only, Toolformer’s filtering, DFSDT vs. ReAct, CodeAct’s advantages, SWE-bench grading and why “Verified”, functional-correctness grading in WebArena, MCP’s M+N argument and its three primitives.

Overview: 1. Framing, 2. Reasoning, 3. Tool use, 4. Deep research, 5. Coding agents, 6. Web and computer use, 7. Protocols and skills.

Framing

From a Chatbot to an Agent

Slide 2

A chat model maps a prompt to one text response. An agent is a model placed in a loop: it takes actions in an environment, observes the results and decides what to do next, all towards a goal.

flowchart LR
  L["LLM<br/>reason and decide the next step"] --> A["Action<br/>call a tool, browse, run code"] --> E["Environment<br/>returns an observation"] --> L

Three ingredients: reasoning (plan and decompose), tools / actions (reach beyond text) and a feedback loop with the world. The loop closes the gap between a static predictor and a system that can change its own next input by acting. See LLM Agent.

This lecture follows the capability arc reasoning → tool use → deep research → coding → web and computer use → protocols and skills. Safety and security of agents is Lecture 9.

Workflows vs. Agents

Slide 3

	Workflow	Agent
Control flow	predefined code paths, fixed by the engineer	the LLM directs its own process and tool use
The model	fills in the blanks	decides the path at run time

Both build on the augmented LLM: a model with retrieval, tools and memory (Anthropic, “Building Effective Agents”, Dec 2024). The guiding advice: start with the simplest thing that works, and add agentic complexity only when simpler calls fall short.

Slide 3: the augmented LLM, the basic building block. Source: Anthropic, 2024.

A Small Set of Composable Patterns

Slide 4

Most successful systems are compositions of a few patterns (Anthropic 2024):

Type	Pattern	Idea
workflow	Prompt chaining	fixed sequence of calls, each processes the previous output, optional checks in between
workflow	Routing	classify the input, dispatch to a specialized prompt or model
workflow	Parallelization	run calls at once: sectioning (split subtasks) or voting (aggregate diverse runs)
workflow	Orchestrator-workers	a lead LLM breaks the task down, delegates to workers, synthesizes the results
workflow	Evaluator-optimizer	one LLM generates, another critiques; loop until the criteria are met
agent	Autonomous agent	uses tools in a loop on environmental feedback; plans and acts independently, with human checkpoints

Part 1: Reasoning

Slide 5

Make the Model Show Its Work: Chain-of-Thought

Slide 6

Standard few-shot prompting shows only input → answer examples, and large models still fail multi-step arithmetic. Chain-of-thought (CoT) prompting adds a few examples that include the intermediate reasoning steps, so the model generates its own reasoning before answering (Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”, NeurIPS 2022).

• No fine-tuning, no gradients: purely in-context.
• A handful (e.g. 8) of worked examples is enough.
• Decomposes a hard problem into steps the model can handle.

Slide 6: the model output (green) is the generated chain of thought leading to the answer. Source: Wei et al., 2022, Fig. 1.

It Only Works Once the Model Is Big Enough

Slide 7

On small models CoT gives little or no benefit and can even hurt. The gains appear only around ~100B parameters, so CoT is an emergent ability: flat for small models, then rising steeply.

Result	Standard → CoT
PaLM 540B on GSM8K (math word problems)	17.9 → 56.9, beating a fine-tuned GPT-3 175B with a verifier, the prior state of the art
StrategyQA (commonsense)	68.6 → 77.8 (+9.2), beating the prior fine-tuned state of the art

Slide 7: standard vs. CoT vs. prior supervised best across LaMDA, GPT-3 and PaLM; CoT pulls ahead only at the largest scale. Source: Wei et al., 2022, Fig. 4.

The Same Trick Spans Different Kinds of Reasoning

Slide 8

• Arithmetic: GSM8K, SVAMP, MAWPS math word problems.
• Commonsense: CSQA, StrategyQA, date and sports questions, robot planning (SayCan).
• Symbolic: last-letter concatenation, coin-flip state tracking.

Slide 8: in each task the model writes out the steps (highlighted) before committing to an answer. Source: Wei et al., 2022, Fig. 3.

Interleave Thinking with Acting: ReAct

Slide 9

Pure CoT can hallucinate (it only reasons from memory); pure acting lacks planning. ReAct interleaves both in one trace, prompted with only 1 or 2 examples (Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models”, ICLR 2023):

• Thought: free-form reasoning, decompose and decide.
• Action: call a tool, e.g. search[...].
• Observation: the result, fed back into the context.

Slide 9: a knowledge task (HotpotQA): standard, CoT and act-only miss, ReAct gets it right. Source: Yao et al., 2023, Fig. 1.

The Same Loop, Now Acting in a World

Slide 10

In ALFWorld (a text-based simulated home), the agent must “put a pepper shaker on a drawer”. Act-only takes blind actions and gets stuck repeating useless ones. ReAct’s interleaved thoughts decide where to look and track the sub-goal, so it finds the object and finishes the task.

Slide 10: act-only (left) wanders and fails; ReAct (right) reasons about where the object is, then acts. Source: Yao et al., 2023.

Reasoning + Acting Beats Either Alone

Slide 11

Slide 11: click Step to advance the Thought / Action / Observation loop.

ReAct’s reasoning is more grounded and interpretable than CoT’s. On the two decision tasks, 1 or 2 examples beat trained baselines:

Task	ReAct	Baselines
ALFWorld	71% success	45% act-only, 37% BUTLER (imitation learning)
WebShop	40.0% success	30.1% act-only, also above IL and RL baselines

Combining internal and external knowledge

ReAct reasons over retrieved facts; CoT-SC reasons from the model’s own knowledge (majority vote over sampled chains). The best HotpotQA method, ReAct → CoT-SC, runs ReAct and falls back to CoT-SC only when it can’t answer within a step budget.

Part 2: Tool Use

Slide 12

Toolformer: Learn Tool Use from the Model’s Own Signal

Slides 13-14

LLMs fail at things tiny tools do trivially: exact arithmetic, factual lookup, today’s date, translation. Toolformer (Schick et al., NeurIPS 2023) teaches a model to decide which API to call, when and with which arguments, self-supervised, with only a few demonstrations per tool. Tools: calculator, question answering, Wikipedia search, translation, calendar; base model GPT-J (6.7B).

Slide 13: API calls are inserted inline, their results inserted into the text. Source: Schick et al., Fig. 1.

Toolformer's filtering

Sample candidate API calls → execute them → keep a call only if its result reduces the model’s loss (perplexity) on the following tokens. Useful calls survive, the others are discarded; the model is then fine-tuned on the augmented text. No human labels for when or how to call a tool.

Slide 14: sample → execute → filter by the LM loss. Source: Schick et al., Fig. 2.

With tools, the 6.7B model beats GPT-3 (175B) on math and factual tasks: ASDiv 7.5 → 40.4 (GPT-3: 14.0), SVAMP 5.2 → 29.4 (GPT-3: 10.0). Language modeling is preserved: perplexity is unchanged when API calls are disabled.

ToolLLM: From a Handful of Tools to 16,000+ Real APIs

Slides 15-16

Open models lagged far behind ChatGPT at real tool use, partly for lack of data. ToolLLM (Qin et al., ICLR 2024) builds ToolBench automatically with ChatGPT, without human labels:

1. Collect APIs: 16,464 real REST APIs in 49 categories from RapidAPI Hub, with their documentation.
2. Generate instructions: ChatGPT writes diverse requests needing one or several APIs.
3. Annotate solution paths: ChatGPT searches for a working sequence of API calls per instruction: the supervised target.

Slide 15: collect → instruct → solve yields ToolBench, used to fine-tune ToolLLaMA; at inference an API retriever supplies the relevant APIs. Source: Qin et al., Fig. 1.

Real API calls fail often (bad arguments, dead endpoints), so a single linear chain (ReAct) gets stuck. DFSDT (depth-first search-based decision tree) tries several action paths and backtracks from failed calls; it lifts ChatGPT’s average pass rate from 40.2% (ReAct) to 64.8%.

• API retriever: a neural search model picks the few relevant APIs from the 16k pool.
• ToolLLaMA: LLaMA-2-7B fine-tuned on ToolBench, similar to ChatGPT even on unseen APIs.
• ToolEval: an automatic ChatGPT judge for pass rate (finished within budget) and win rate (quality vs. a reference), validated against humans.

Slide 16: DFSDT explores a tree and abandons failed paths. Source: Qin et al., Fig. 4.

CodeAct: Stop Picking from a Menu, Write Code

Slides 17-19

Predefined JSON or text tool calls are rigid: they can’t easily compose tools or reuse libraries. CodeAct (Wang et al., “Executable Code Actions Elicit Better LLM Agents”, ICML 2024) makes Python code the action space. Same task (find the cheapest country to buy a phone): the text agent needs many turns, CodeAct one code action.

Slide 17: text/JSON actions (left) vs. one code action (right). Source: Wang et al., Fig. 1.

Advantage	Why
Composability	chain many operations in one action
Control flow	loops, branches and functions for free
Ecosystem	call any existing Python library
Fewer actions	one code block replaces many tool calls

The loop: generate code → execute in an interpreter → observe the output or traceback → fix and continue. The agent debugs its own code from execution feedback. Results on M3ToolEval: up to +20% success and up to 30% fewer actions than JSON/text across 17 LLMs; GPT-4 with CodeAct reaches 74.4% (+20.7 points).

Where the skill comes from (CodeActInstruct): no new human labels.

1. Repurpose tasks: 7K multi-turn trajectories from info-seeking (HotpotQA), math (MATH), code with self-debug (APPS), tables (WikiTableQuestion), embodied planning (ALFWorld), recast so the action is Python.
2. Generate and filter: GPT-4, GPT-3.5 and Claude solve them in the run-observe loop; keep only successful runs, favoring ones where the model makes an error and fixes it (411 from GPT-4, 6,728 from GPT-3.5 / Claude).
3. Fine-tune Llama-2-7B and Mistral-7B, mixed with general chat data (~10.6M agent tokens + ~55M chat tokens): CodeActAgent.

The recurring recipe for agents

Take an environment with ground truth (here the interpreter), let a strong model generate trajectories in it, filter for the behavior you want (successful, self-correcting) and distill it into a smaller policy.

What Makes a Good Tool for an Agent?

Slides 20-21

Agents are non-deterministic and have limited context, so tools must be designed for agent ergonomics, not just wrap an API endpoint (Anthropic, “Writing Tools for Agents”, 2025):

Principle	Example
Build fewer tools	one schedule_event instead of separate list / find / create calls
Return meaningful context	semantic names instead of opaque UUIDs; concise vs. detailed format; paginate, filter, truncate (Claude Code truncates tool responses at 25,000 tokens by default)
Namespacing	prefixes like asana_search, jira_search so the agent picks the right tool among hundreds
Prompt-engineer the descriptions	write specs as for a new hire, with unambiguous names; small refinements alone gave SOTA on SWE-bench Verified

Evaluation-driven: generate realistic tasks, run them in agentic loops, collect accuracy and runtime, number of tool calls, tokens and tool errors. Then let Claude read the transcripts and refactor the tool definitions. On both the Slack and Asana servers, Claude-optimized tools beat the human-written ones on held-out accuracy.

Slide 21: human-written vs. Claude-optimized tools (Asana server). Source: Anthropic, 2025.

Part 3: Deep Research

Slide 22

WebGPT: Answering Questions by Browsing

Slides 23-24

Long-form QA needs evidence, but models hallucinate and give no provenance. WebGPT (Nakano et al., OpenAI 2021) fine-tunes GPT-3 to answer in a text-based browser: it reads a text summary of the current state (question, page text at the cursor, link indices, actions left) and replies with one command (Search, Click link, Find in page, Quote, Scroll, Back, End: Answer), then writes an answer with citations.

Slide 23: the graphical interface shown to the human demonstrators; the model gets a text version of the same state. Source: Nakano et al., Fig. 1.

flowchart LR
  B["Behavior cloning<br/>imitate human browsing"] --> R["Reward model<br/>from answer comparisons"] --> P["RL + rejection sampling<br/>optimize the policy"]

The same RLHF machinery as in Lecture 7, applied to a browsing policy. On ELI5, WebGPT (175B) answers are preferred 56% of the time over its human demonstrators and 69% over the highest-voted Reddit answer. Citations let humans verify the claims.

OpenAI Deep Research: An Autonomous Research Analyst

Slide 25

Deep research (OpenAI, “Introducing deep research”, Feb 2025) searches, reads and pivots across hundreds of sources and writes a fully cited report in 5 to 30 minutes.

• Powered by a version of o3 optimized for browsing and data analysis.
• End-to-end RL on hard browsing and reasoning tasks: it learns to plan, backtrack and react to live information.
• State of the art on Humanity’s Last Exam and the GAIA web-research benchmark.

Humanity’s Last Exam (pass@1)	Accuracy
GPT-4o	3.3%
Claude 3.5 Sonnet	4.3%
OpenAI o1	9.1%
DeepSeek-R1	9.4%
o3-mini (high)	13.0%
Deep research (o3)	26.6%

From WebGPT to here: browsing went from a scripted tool loop to a trained, autonomous skill.

Multi-Agent Research: One Lead Agent, Many Subagents

Slides 26-27

Research is open-ended; you can’t hardcode the path. Claude’s Research feature uses the orchestrator-workers pattern: a lead agent plans and spawns subagents that explore in parallel, each with its own context window, then compress and return their findings (Anthropic, “How we built our multi-agent research system”, Jun 2025).

Slide 26: the lead agent orchestrates search and citation subagents. Source: Anthropic, 2025.

• Parallel, breadth-first exploration; each subagent owns a sub-problem; more total context than one agent could hold.
• +90.2% for the multi-agent system (Opus 4 lead + Sonnet 4 subagents) over single-agent Opus 4 on their research eval.
• ~80% of the performance variance is explained by token usage, number of tool calls and model choice.

The downside: performance tracks tokens

Token usage relative to one chat: chat 1×, single agent 4×, multi-agent 15×. It pays off only on high-value tasks; coordination, delegation prompts and debugging stateful agents are hard.

Part 4: Coding Agents

Slide 28

SWE-bench: Can a Model Resolve a Real GitHub Issue?

Slides 29-30

Toy coding benchmarks were saturated. SWE-bench (Jimenez et al., ICLR 2024) uses 2,294 real issue + pull-request pairs from 12 popular Python repositories (django, sympy, scikit-learn, …). Given the full codebase and the issue, the model must produce a patch.

• Graded by the repo’s real unit tests: FAIL_TO_PASS (the bug is fixed) and PASS_TO_PASS (no regressions).
• Hard: huge context, cross-file edits, real-world reasoning.

Slide 29: issue + codebase → generated patch → verified by the test suite. Source: Jimenez et al., Fig. 1.

At release the best model (Claude 2 with BM25 retrieval) resolved only 1.96%.

Why a "Verified" subset?

Some unit tests were overly specific, many samples underspecified, some environments hard to set up, so valid solutions could be graded as wrong: the full benchmark underestimates ability. SWE-bench Verified (OpenAI, Aug 2024) is 500 tasks that 93 professional developers confirmed as solvable and fairly graded; it is now the standard set (frontier agents exceed 70%). The point is fair grading, not bad code.

Codex: An Agent That Lives in a Sandboxed Repo

Slides 31-32

Codex (OpenAI, May 2025) runs each task in its own isolated, sandboxed cloud environment preloaded with the repository. It reads and edits files and runs commands (tests, linters, type checkers), then proposes a pull request for human review. It is powered by codex-1, a version of o3 trained with RL on real coding tasks.

Slide 31: assign a task → spin up a sandbox (network off) → edit and run → iterate to green → propose a PR. Click Step to walk through one task.

Result	Score
SWE-bench Verified, single attempt (codex-1)	72.1%
SWE-bench Verified, multiple parallel attempts (pass@8)	~84%

Mind the test sets

The ~2% at release was on the full SWE-bench, the 72.1% on Verified. That is a trajectory, not a like-for-like jump.

The pattern that won

A model trained for the task, an environment it can act in and get ground truth from (run the tests), iterative self-correction, and a human reviewing the diff.

Part 5: Web and Computer-Use Agents

Slide 33

WebArena: a Realistic, Reproducible Web

Slides 34-35

WebArena (Zhou et al., ICLR 2024) is a self-hosted, fully functional web with sites in 4 domains (e-commerce, social forum, GitLab, a CMS) plus tools and knowledge resources, and 812 long-horizon tasks as natural-language intents. The best GPT-4 agent succeeded on 14.41% vs. 78.24% for humans.

Slide 34: self-hosted sites + tools; tasks verified by functional correctness. Source: Zhou et al., Fig. 1.

The agent acts with click / type / goto over the page’s accessibility tree and is graded by functional correctness: on the resulting state, not the words.

Task type	Example intent	Check
Information seeking	”Tell me the name of the customer who has the most cancellations in the history”	exact_match(answer, "Samantha Jones")
Site navigation	”Checkout merge requests assigned to me”	exact_match(current_url(state), ".../merge_requests?assignee_username=byteblaze")
Content and config	”Post to ask ‘whether I need a car in NYC‘“	must_include(post_url, "/f/nyc"), must_include(post_body, "a car in NYC")

A fluent description of the intended action earns no credit; the checker inspects the actual end state (database, URL or DOM).

VisualWebArena and OSWorld: from Text to Pixels, Beyond the Browser

Slide 36

Benchmark	Setting	Best agent	Humans
VisualWebArena (Koh et al., ACL 2024)	910 visually grounded web tasks, many need image understanding	16.4% (GPT-4V with set-of-marks)	88.7%
OSWorld (Xie et al., NeurIPS 2024)	real computers (Ubuntu, Windows, macOS), 369 tasks across real apps, execution-based grading	12.2%	72.4%

VisualWebArena.

OSWorld.

Both grade by the resulting state and both show a wide gap to humans.

WebVoyager: Seeing the Page, Not Parsing the HTML

Slide 37

Early web agents read raw HTML or accessibility trees. WebVoyager (He et al., ACL 2024) is an end-to-end agent on a large multimodal model: it sees the rendered page as a screenshot with the interactive elements marked, reasons, then clicks / types / scrolls on 15 real, live websites: 59.1% success.

Slide 37: observation = screenshot + marked elements. Source: He et al., Fig. 1.

Benchmark vs. agent

VisualWebArena is a reproducible sandbox graded by exact programmatic checks. WebVoyager is an agent on the live web; its 643-task test set has no ground-truth checker, success is scored by an automatic GPT-4V judge (with the judge biases of Lecture 7).

Part 6: Protocols and Skills

Slide 38

The M×N Integration Problem: Model Context Protocol

Slide 39

You have $M$ AI apps (Claude Desktop, Cursor, ChatGPT) and want to reach $N$ systems (Drive, Slack, GitHub, Postgres).

	Before	With MCP
Integrations	$M\times N$: a separate connector per app-tool pair (3 apps × 4 tools = 12)	$M+N$: one client per app + one server per tool (3 + 4 = 7)
Reuse	a Slack connector written for Cursor does nothing for Claude	any client talks to any server

flowchart TB
  C["AI applications (clients)<br/>Claude Desktop, IDEs, agents"] <--> M["MCP<br/>one open protocol"] <--> S["Data and tools (servers)<br/>Drive, Slack, GitHub, Postgres"]

“Like a USB-C port for AI”: one standard plug, so any app connects to any source (Anthropic, “Introducing the Model Context Protocol”, Nov 2024). The old per-pair connectors were app-specific: ChatGPT plugins, LangChain tool wrappers, function-calling glue. See Model Context Protocol.

MCP: A Standard Interface, Three Primitives

Slide 40

A bidirectional client-server model: an MCP server exposes a system’s capabilities, an MCP client (the AI app) connects to it. Three kinds of primitives:

Primitive	What it is
Tools	actions the model can invoke (functions with arguments)
Resources	data the model can read (files, records, context)
Prompts	reusable prompt templates the server offers

• Why standardize: one uniform interface, so the agent’s working context carries across tools (read from Postgres, then act in Slack). Add a system by adding a server, not by rebuilding the agent.
• A real cross-vendor standard: launched by Anthropic (Nov 2024), adopted by OpenAI, Google DeepMind and Microsoft in 2025, plus many community servers. Anthropic open-sourced the spec, SDKs and servers for Google Drive, Slack, GitHub, Git, Postgres, Puppeteer.

The security side (every server is untrusted input) is Lecture 9.

Agent Skills: Package Expertise as a Folder

Slide 41

A model is a generalist: it doesn’t know your team’s procedures, formats and conventions. An Agent Skill packages that know-how as a plain folder: a SKILL.md file plus any scripts it needs (Anthropic, “Equipping agents for the real world with Agent Skills”, 2025).

• Each SKILL.md starts with a short YAML header with a name and a description (both required), followed by instructions.
• It can bundle runnable scripts, so exact steps (parse a PDF, sort data) run as code instead of being regenerated.
• Skills are portable and pair with MCP: the skill captures the workflow, which can call MCP tools.

Slide 41: a SKILL.md with YAML frontmatter (name, description) and instructions. Source: Anthropic, 2025.

Load Only What You Need, When You Need It

Slide 42

A finite context window can’t hold every skill in full. Progressive disclosure loads a skill in levels, like a well-organized manual:

flowchart LR
  A["1 · Metadata<br/>name + description,<br/>always preloaded"] -->|"skill looks relevant"| B["2 · SKILL.md body<br/>full instructions,<br/>loaded on demand"] -->|"task needs it"| C["3 · Bundled files<br/>references and scripts,<br/>read or run as needed"]

Mock exam: progressive disclosure

Only the skill’s name and description are preloaded; the full SKILL.md body is loaded only when the agent judges the skill relevant; bundled files and scripts are read or run only when needed. Because the agent pays context only for what it actually opens, the total material a skill carries can far exceed the context window: its effective size is essentially unbounded. The same way a human uses a manual: skim the index, open only the needed chapter.

See Agent Skills.

The Agent Capability Stack

Slide 43

Capability	Key idea	Landmark work
Reasoning	elicit step-by-step thinking; interleave it with actions	CoT, ReAct
Tool use	call APIs; train for it; write code instead of menus	Toolformer, ToolLLM, CodeAct
Tool design	fewer, ergonomic, eval-driven tools	Writing tools for agents
Deep research	autonomous multi-step browsing with citations; multi-agent	WebGPT, deep research, multi-agent research
Coding	real issues + executable tests; sandboxed iterative agents	SWE-bench, Codex
Web / computer use	realistic environments; HTML → pixels	WebArena, OSWorld, WebVoyager
Connection	standard protocol for tools and data; portable skill folders	MCP, Agent Skills

Modern agents compose all of these: a reasoning model, in a loop, with well-designed tools, in a realistic environment, connected by open standards.

Three Things to Remember

Slides 44-45

1. An agent is a loop, not a model. Reasoning, tools and environment feedback turn a predictor into a system that acts. Start simple; add agency only when needed.
2. Capability comes from acting in the world. The biggest gains came from grounding: browsing, running tests, seeing the screen. The environment supplies the ground truth.
3. The trajectory is towards autonomy. Scripted tool loops became trained, autonomous, multi-agent systems connected by shared standards. Capability and risk grow together: the same loop that makes agents useful makes them exploitable (Lecture 9).

Self-Test

Question cards (13)

What turns a chat model into an agent, and how do workflows differ from agents?

Answer

An agent is a model in a loop with reasoning, tools/actions and environment feedback: it decides, acts, observes and repeats towards a goal. A workflow orchestrates LLM calls along predefined code paths; an agent decides its own process and tool use at run time. Both use an augmented LLM (retrieval, tools, memory); start with the simplest option.

What is chain-of-thought prompting, and why is it called emergent?

Answer

Few-shot examples include the intermediate reasoning, so the model writes its own reasoning before the answer, without fine-tuning. It helps little or even hurts on small models and only pays off around ~100B parameters (PaLM 540B on GSM8K: 17.9 → 56.9), so the ability appears only at scale.

How does ReAct improve on pure CoT and pure acting?

Answer

It interleaves Thought → Action → Observation. Observations ground the reasoning in retrieved facts, reducing CoT’s hallucinations; thoughts add planning and sub-goal tracking that act-only agents lack, so they don’t wander. With 1 or 2 examples it reaches 71% on ALFWorld vs. 45% act-only.

How does Toolformer learn tool use without human labels?

Answer

It samples candidate inline API calls, executes them and keeps a call only if the result lowers the model’s loss on the following tokens; then it fine-tunes on this augmented text. It learns which tool, when and with which arguments; GPT-J 6.7B with tools beats GPT-3 on math and factual tasks while perplexity stays unchanged.

How does ToolLLM handle 16,000+ APIs and failing calls?

Answer

ChatGPT builds ToolBench (collect RapidAPI docs, generate single- and multi-tool instructions, annotate working call sequences); a neural retriever picks the relevant APIs. DFSDT explores a tree of action paths and backtracks after failed calls instead of getting stuck in one ReAct chain (pass rate 40.2% → 64.8%).

Why use code as the action space (CodeAct), and how is the agent trained?

Answer

Code composes tools in one action, has loops, branches and functions, and can use any Python library, so fewer actions are needed (+20% success, up to 30% fewer actions). The loop is generate → execute → observe → self-debug. Training: strong models solve repurposed tasks in the interpreter, only successful (ideally self-correcting) trajectories are kept and distilled into Llama-2 / Mistral 7B.

What makes a tool ergonomic for an agent?

Answer

Few consolidated tools (one schedule_event, not list/find/create), meaningful and token-efficient returns (names not UUIDs, pagination, truncation), namespacing (asana_search vs. jira_search) and unambiguous descriptions written like for a new hire. Improve them with realistic evals and let the model refactor them from transcripts.

How did WebGPT set the recipe for grounded research, and what does multi-agent research add?

Answer

WebGPT browses via text commands and answers with citations, trained with behavior cloning, a reward model from comparisons and RL / rejection sampling. Multi-agent research uses orchestrator-workers: a lead agent spawns parallel subagents with separate context windows and synthesizes their results (+90% over single agent), at ~15× the tokens of a chat.

How does SWE-bench grade a coding agent, and why was SWE-bench Verified created?

Answer

The agent gets a real GitHub issue and the repository and must produce a patch; the repo’s unit tests check FAIL_TO_PASS (fixed) and PASS_TO_PASS (no regressions). Verified is a 500-task subset confirmed by 93 developers as solvable and fairly graded, because overly specific tests and underspecified issues made the full set underestimate ability.

What pattern makes Codex-style coding agents work?

Answer

A model trained with RL on coding tasks acts in a sandboxed copy of the repo (network off), edits files, runs the tests, reads failures and iterates until green, then proposes a PR for human review. Ground truth from execution plus self-correction plus human review: 72.1% on SWE-bench Verified.

How do WebArena, VisualWebArena, OSWorld and WebVoyager differ?

Answer

WebArena: self-hosted sites, accessibility-tree actions, programmatic end-state checks (GPT-4 14% vs. 78% humans). VisualWebArena: visually grounded web tasks with screenshots (16% vs. 89%). OSWorld: real operating systems and apps, execution-based grading (12% vs. 72%). WebVoyager: an agent on live websites using marked screenshots, scored by a GPT-4V judge (59%).

How does MCP solve the M×N integration problem, and what does a server expose?

Answer

Without a standard, M apps and N systems need M×N connectors; with MCP each app implements one client and each system one server, M+N pieces, and any client talks to any server. A server exposes tools (actions), resources (readable data) and prompts (templates). OpenAI, Google DeepMind and Microsoft adopted it in 2025.

Explain progressive disclosure in Agent Skills and why a skill's effective size is essentially unbounded.

Answer

Only the name and description are preloaded; the full SKILL.md is loaded when the agent judges it relevant; bundled files and scripts are read or run only when needed. The agent pays context only for what it opens, so a skill can carry far more material than fits in the context window.

Multiple Choice

Multiple choice (5)

Which pattern is an agent rather than a workflow?

• prompt chaining

• routing

• evaluator-optimizer

• an LLM that uses tools in a loop on environment feedback and decides its own path

Explanation

The first five patterns (chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) have code paths fixed by the engineer.

When does Toolformer keep a sampled API call in its training data?

• when a human labels it as useful

• when inserting its result lowers the model’s loss on the following tokens

• when the API returns without error

• always, if the tool is a calculator

Explanation

The filter is the model’s own loss: a call is useful if it makes the continuation easier to predict. That is what makes the method self-supervised.

Why is SWE-bench scored with unit tests instead of comparing the patch to the reference fix?

• Unit tests are faster to write.

• Many different patches can fix an issue; tests check the behavior (fixed and no regressions).

• The reference fixes are secret.

• Text comparison is not possible for code.

Explanation

FAIL_TO_PASS checks the bug is fixed, PASS_TO_PASS that nothing broke. This is the same principle as WebArena’s functional correctness: grade the end state, not the words.

With 5 AI apps and 8 data systems, how many pieces must be built with and without MCP?

• 13 without, 40 with

• 40 without, 13 with

• 40 in both cases

• 8 without, 5 with

Explanation

Without a standard every pair needs a connector (5 × 8 = 40); with MCP one client per app and one server per system (5 + 8 = 13).

Which statements about multi-agent research systems are true? (Select all that apply.)

• Subagents work in parallel, each with its own context window.

• They use roughly 15× the tokens of a chat.

• They are always cheaper than a single agent.

• Token usage explains most of the performance variance.

Explanation

Anthropic reports ~80% of variance explained by tokens, tool calls and model choice; the cost means it only pays off on high-value tasks.

References

All sources cited on the slides, in slide order (19 entries)

Slide	Source	Key point
3, 4	Anthropic, “Building Effective Agents”, 2024	workflows vs. agents, composable patterns
6	Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”, NeurIPS 2022	chain-of-thought
9, 10	Yao et al., “ReAct”, ICLR 2023	reasoning + acting
13	Schick et al., “Toolformer”, NeurIPS 2023	self-supervised tool use
15, 16	Qin et al., “ToolLLM”, ICLR 2024	16k APIs, DFSDT
17, 19	Wang et al., “Executable Code Actions Elicit Better LLM Agents”, ICML 2024	CodeAct
20, 21	Anthropic, “Writing Tools for Agents”, 2025	tool ergonomics
23	Nakano et al., “WebGPT”, 2021	browsing with citations
25	OpenAI, “Introducing deep research”, 2025	autonomous research agent
26	Anthropic, “How we built our multi-agent research system”, 2025	orchestrator-workers research
29	Jimenez et al., “SWE-bench”, ICLR 2024	real GitHub issues
30	OpenAI, “Introducing SWE-bench Verified”, 2024	verified subset
31	OpenAI, “Introducing Codex”, 2025	sandboxed coding agent
34	Zhou et al., “WebArena”, ICLR 2024	realistic web environment
36	Koh et al., “VisualWebArena”, ACL 2024	visual web tasks
36	Xie et al., “OSWorld”, NeurIPS 2024	real computer environment
37	He et al., “WebVoyager”, ACL 2024	multimodal web agent
39, 40	Anthropic, “Introducing the Model Context Protocol”, 2024	MCP
41, 42	Anthropic, “Equipping agents for the real world with Agent Skills”, 2025	Agent Skills, progressive disclosure

Related

• Previous: Lecture 7: Alignment Tools · Next: Lecture 9: Agent Security · Course: Overview
• Exam and reference: Exam Structure · Study Plan · Formula Sheet · Glossary
• Concepts: LLM Agent, Model Context Protocol, Agent Skills, Prompt Injection, METR Time Horizon
• The METR time horizons of agents: Lecture 1; decomposition attacks with coding agents: Lecture 3.