What Are AI Agents

An AI agent is a system in which a language model directs its own process - choosing which tools to call, reading the results and deciding what to do next - in a loop, until a goal is met or it stops. This chapter pins down that definition, places agents on the spectrum from a single model call to fully autonomous systems, and gives a test for when an agent is the right design and when it isn't.

The Definition That Matters: Who Controls the Next Step?

Anthropic's widely used framing (Building Effective Agents, 2024) draws the line at control flow:

• Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
• Agents are systems where LLMs dynamically direct their own processes and tool usage.

Both are agentic systems; the difference is whether your code or the model decides the next step. OpenAI's guide says the same thing from the user's side: an agent is a system that independently accomplishes a task on your behalf, using a model to manage the workflow, choose tools and recognise when it is done.

flowchart LR
    subgraph L1["💬 Single call"]
        A1["Prompt"] --> A2["🧠 LLM"] --> A3["Answer"]
    end
    subgraph L2["🔗 Workflow"]
        B1["Input"] --> B2["🧠 Step 1"] --> B3["🔧 Tool"] --> B4["🧠 Step 2"] --> B5["Output"]
    end
    subgraph L3["🤖 Agent"]
        C1["Goal"] --> C2["🧠 LLM decides"]
        C2 -->|"tool call"| C3["🔧 Execute"]
        C3 -->|"result"| C2
        C2 -->|"done"| C4["✅ Answer"]
    end
    L1 ~~~ L2 ~~~ L3

    style L1 fill:#e8e2d9,stroke:#ccc4b8
    style L2 fill:#d8dfe8,stroke:#b0bac8
    style L3 fill:#dde4dc,stroke:#b0c4b0

	Single LLM call	Workflow (chain / pipeline)	Agent
Who picks the next step	Nobody - one step	Your code, fixed in advance	The model, at run time
Number of steps	1	Fixed (possibly branching)	Variable, unknown in advance
Handles surprises	No	Only the branches you wrote	Yes - it can re-plan
Cost and latency	Lowest, predictable	Predictable	Higher, variable
Testability	Easy	Easy - each step testable	Hard - many possible trajectories
Example	"Classify this ticket"	"Extract fields → validate → write to CRM"	"Resolve this customer's billing problem"

Vocabulary used throughout the agents track

Term	Meaning
Tool	A function the model can ask your code to run, described by a name, description and JSON Schema
Harness (or scaffold)	The code around the model: the loop, tool execution, context management, bounds, guardrails
Trajectory	The full sequence of model outputs, tool calls and results in one run
Episode	One run of an agent on one task, from goal to termination
Environment	Everything the agent acts on through tools - APIs, files, a browser, a database
Augmented LLM	A model with retrieval, tools and memory available - the building block of both workflows and agents
Compound AI system	Any system combining several model calls, retrievers and tools (Zaharia et al., BAIR 2024)

---

The Autonomy Spectrum

Autonomy is a design decision, separate from capability: a highly capable agent can still be deployed with a human approving every action. Feng, McDonald and Zhang (2025) define five levels by the role the user plays:

Level	User role	What the agent does	Example
1	Operator	Acts only when the user directs each step	Inline code completion
2	Collaborator	Works alongside the user; either can take the lead	Pair-programming in an IDE chat
3	Consultant	Leads the work and asks the user for input and preferences	A research agent that checks scope with you
4	Approver	Works independently; the user approves high-risk actions	A support agent that needs sign-off before issuing refunds
5	Observer	Acts fully autonomously; the user can monitor and stop it	A nightly dependency-upgrade agent opening pull requests

flowchart LR
    O["🕹️ Operator"] --> C["🤝 Collaborator"] --> Co["💡 Consultant"] --> A["✅ Approver"] --> Ob["👀 Observer"]

    style O fill:#e8e2d9,stroke:#ccc4b8
    style C fill:#d8dfe8,stroke:#b0bac8
    style Co fill:#dde4dc,stroke:#b0c4b0
    style A fill:#e8e0d4,stroke:#c8b89a
    style Ob fill:#ddd8e4,stroke:#b8b0c8

Most production agents that take consequential actions sit at Approver: they do the work, and a person confirms the irreversible steps (payments, deletions, external messages). Move right only as your evaluation evidence and guardrails justify it (Agents in Practice).

---

Why Agents Became Practical

Agents built on 2023 models (AutoGPT, BabyAGI) were famous for looping, losing the goal and hallucinating actions. Three things changed:

1. Models trained for tool use and long horizons. Frontier models are now post-trained with reinforcement learning on multi-step agentic tasks (coding, browsing, tool use), not just on single responses - see Post-training & Reasoning.
2. Reasoning between actions. Reasoning models think before each tool call and can interleave thinking with tool results, which makes planning and error recovery native rather than prompted (Planning & Reasoning).
3. Measured, steady progress on long tasks. METR measured the length of software tasks (in human time) that models complete with 50% reliability: it has doubled roughly every seven months since 2019, reaching about 50 minutes for early-2025 frontier models (Kwa et al., 2025).

Two cautions follow from the same evidence. The 50% horizon is a coin-flip reliability level - production needs far higher. And reliability over repeated attempts is much lower than single-attempt success: on τ-bench, agents that solved a task once often failed it on a rerun (Agent Evaluation Basics).

---

Workflow or Agent? A Decision Test

The best first design is usually the simplest one that works. Anthropic's advice: start with a single well-prompted call with retrieval; add a workflow when the steps are known; use an agent only when the steps genuinely can't be predicted.

flowchart TD
    Q1{"Can one well-prompted call<br/>(with retrieval) do it?"} -->|"yes"| S["💬 Single call"]
    Q1 -->|"no"| Q2{"Can you write down the steps<br/>in advance?"}
    Q2 -->|"yes"| W["🔗 Workflow<br/>(chain, router, parallel)"]
    Q2 -->|"no"| Q3{"Is the value worth the extra<br/>cost, latency and risk?"}
    Q3 -->|"no"| W2["🔗 Workflow + human<br/>for the hard cases"]
    Q3 -->|"yes"| Q4{"Can you verify outcomes and<br/>bound the damage of mistakes?"}
    Q4 -->|"no"| H["🙋 Keep a human in the loop<br/>(Approver level)"]
    Q4 -->|"yes"| A["🤖 Agent"]

    style S fill:#e8e2d9,stroke:#ccc4b8
    style W fill:#d8dfe8,stroke:#b0bac8
    style W2 fill:#d8dfe8,stroke:#b0bac8
    style H fill:#e8e0d4,stroke:#c8b89a
    style A fill:#dde4dc,stroke:#b0c4b0

Signal	Points to a workflow	Points to an agent
Path through the task	Same every time	Depends on what you find along the way
Inputs	Structured, well-defined	Open-ended requests
Cost of a wrong step	High and irreversible	Recoverable, or gated by approval
Volume and latency needs	High volume, tight latency	Lower volume, minutes are acceptable
How you know it worked	Per-step checks	An outcome check (tests pass, ticket resolved)

Coding is the canonical agent use case for exactly these reasons: the path is unpredictable, but the outcome is verifiable (tests, type checks) and changes are reversible (version control). The workflow patterns themselves - chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer - are covered in Agent Patterns.

---

Check Yourself

Exercises

Study Notes

• Agent = a model directing its own process: tools in a loop, deciding the next step itself
• Workflow = predefined code paths around model calls; both are "agentic systems"
• The difference is who controls the sequence of steps - not whether tools or multiple models are used
• Autonomy levels by user role: operator, collaborator, consultant, approver, observer; consequential agents usually start at approver
• Agents became practical through RL post-training on agentic tasks, reasoning between actions, and a steady rise in task horizon (≈7-month doubling, METR)
• Start with the simplest design; use an agent when steps can't be predicted, outcomes can be verified, and mistakes can be bounded

References

• Anthropic, Building Effective Agents (Dec 2024)
• OpenAI, A Practical Guide to Building Agents (2025)
• Zaharia et al., The Shift from Models to Compound AI Systems (BAIR, 2024)
• Feng, McDonald and Zhang, Levels of Autonomy for AI Agents (2025)
• Kwa et al., Measuring AI Ability to Complete Long Software Tasks (METR, 2025)
• Yao et al., τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (2024)

Last reviewed: 2026-09