What makes a system agentic
A single model call with a prompt is not an agent. Three properties together are:
- Tools — the model can invoke functions that read or change the world.
- A loop — it observes results and decides what to do next.
- Control over termination — the model, not the code path, decides when it is done.
┌────────────────────────────────────────────┐
│ observe → plan → call tool → observe ... │ ← model decides
└────────────────────────────────────────────┘ ← when to exitWhen not to use an agent
Worth stating first, because agents are frequently reached for where a workflow would do the job better, cheaper and more reliably.
| If the task is | Use | Because |
|---|---|---|
| Fixed sequence of known steps | A workflow | Deterministic, testable, cheaper |
| One retrieval then one answer | Plain RAG | No loop needed |
| Branching on a small known set of conditions | A router plus workflows | The model classifies; code executes |
| Genuinely open-ended, variable step count | An agent | This is what agents are for |
The honest test: can you draw the flowchart? If you can, build the flowchart. An agent that rediscovers a fixed procedure on every request is paying model tokens for something a switch statement does perfectly.
Tool design is the real work
Agent quality tracks tool quality far more closely than it tracks model choice. Tools are the model's API, and the same rules apply as for any API consumed by an unreliable client.
{
"name": "get_charging_sessions",
"description": "Retrieve charging sessions for one site within a date range. Returns at most 100 sessions, newest first. Use get_session_detail for a single session.",
"parameters": {
"type": "object",
"properties": {
"site_id": { "type": "string", "description": "Site identifier, e.g. LOC-ND-014" },
"date_from": { "type": "string", "format": "date-time" },
"date_to": { "type": "string", "format": "date-time" },
"status": { "type": "string", "enum": ["ACTIVE","COMPLETED","INVALID"] }
},
"required": ["site_id", "date_from", "date_to"]
}
}- Describe when to use it, not just what it does. “Use
get_session_detailfor a single session” prevents a whole class of wrong call. - Constrain with enums and formats. Every free-text parameter is a hallucinated value waiting to happen.
- State limits in the description. A model that knows it gets 100 rows will paginate rather than assume completeness.
- Keep the toolset small. Selection accuracy degrades as the tool count rises; beyond roughly 15–20, split into specialised agents.
- Return errors as usable text. “No site with ID LOC-ND-999. Did you mean LOC-ND-014?” lets the agent recover; a stack trace does not.
Orchestration patterns
| Pattern | Shape | Fits |
|---|---|---|
| Single agent | One model, one toolset, one loop | Most problems — start here |
| Router | Classify, then dispatch to a specialist | Distinct request categories |
| Parallel fan-out | Independent subtasks concurrently, then merge | Research, multi-source gathering |
| Supervisor / worker | A planner delegates to specialists | Genuinely complex multi-domain work |
| Reflection | Generate, critique, revise | Quality-critical output where a second pass pays |
Start with a single agent. Multi-agent architectures are attractive on a whiteboard and expensive in practice: every handoff is a context transfer where information is lost, every additional agent multiplies token cost, and debugging a failure across five agents is genuinely hard. Adopt them when a single agent has measurably failed, not in anticipation.
The router pattern is the exception worth reaching for early. Classifying a request and dispatching to a narrow, well-tooled handler is cheap, testable, and keeps each toolset small.
Bounding the loop
Every agent needs hard limits enforced in code, not requested in the prompt. A prompt asking the model to be efficient is a suggestion; a step counter is a guarantee.
| Guardrail | Bounds | On breach |
|---|---|---|
| Max steps | Runaway loops | Terminate and return partial results with an explanation |
| Token budget | Cost per request | Terminate; alert if breached often |
| Wall-clock timeout | Latency | Terminate; return what is available |
| Tool call quota | Hammering a downstream system | Terminate; the downstream owner will thank you |
| Repetition detection | Same call with same arguments | Break the loop — a strong signal of being stuck |
| Approval gate | Irreversible actions | Pause for a human |
State and memory
| Scope | Holds | Where |
|---|---|---|
| Working | The current loop's steps and observations | In the context window |
| Session | This conversation | Cache or session store |
| Long-term | Facts about a user or account | Database, retrieved deliberately |
| Knowledge | Documents and reference material | Vector index — see RAG |
The failure mode here is treating the context window as memory. It fills, and when it does, the earliest content — usually the original instructions and the user's actual question — is what gets truncated. The agent then continues confidently on a task it has partly forgotten.
Manage this explicitly: summarise older steps rather than dropping them, keep the system prompt and original request pinned, and store durable facts outside the window so they can be retrieved rather than carried.
Observability
An agent trace is not a stack trace. When something goes wrong the question is usually “why did it decide that?”, and answering it requires having recorded the decision.
Log per step: the model's stated reasoning, the tool selected, the arguments, the raw result, tokens consumed, and latency. Correlate all of it under one request ID.
The metrics that actually predict problems:
- Steps per request — the distribution, not the mean. A growing tail means the agent is getting stuck more often.
- Tool error rate — rising values usually mean malformed arguments, which means a tool description needs work.
- Guardrail breach rate — how often limits fire. Non-zero is fine; rising is a warning.
- Cost per resolved request — the number that decides whether the system survives a budget review.
Production lessons
- If you can draw the flowchart, build the flowchart. Agents are for genuinely open-ended work.
- Enforce limits in code. Prompt-level requests for restraint are not controls.
- Separate read from write and gate the writes.
- Invest in tool descriptions. They matter more than model choice.
- Keep toolsets under ~15–20 before splitting into specialists.
- Start single-agent. Add agents against evidence, not anticipation.
- Return errors the model can act on.
- Trace every step under one ID — the same correlation discipline that makes protocol integrations debuggable.
- Detect repetition and break. Identical repeated calls mean stuck, not thorough.
Frequently asked questions
Three properties together: the model can call tools that read or change the world, it runs in a loop observing results and deciding what to do next, and it controls when the loop terminates. That last property is what creates the risk profile, because cost and behaviour are decided at runtime.
When you can draw the flowchart. A fixed sequence of known steps should be a workflow — deterministic, testable and cheaper. An agent that rediscovers a fixed procedure on every request is paying model tokens for what a switch statement does perfectly.
Enforce limits in code rather than requesting restraint in the prompt: maximum steps, token budget, wall-clock timeout, tool call quota, and repetition detection that breaks when the same call is made with the same arguments. Prompt-level guidance is a suggestion; a counter is a guarantee.
Usually not at first. Every handoff loses context, every extra agent multiplies token cost, and debugging across several agents is genuinely hard. Start with a single agent and adopt multi-agent patterns when a single agent has measurably failed. The router pattern is the exception worth using early.
Roughly 15 to 20 at most. Tool selection accuracy degrades as the count rises, so beyond that point split the work into specialised agents each with a smaller, focused toolset.