How AI Agents Work: An Engineer's View of Agentic AI
September 26, 2026 · 26 min read
Contents
An AI agent is a system that is given a goal and decides for itself which steps to take toward it. The load-bearing part of that definition is not "decides for itself" but the loop: the model calls a tool, sees the result, uses it to pick the next action, and repeats until the goal is met or a stopping condition fires.
That loop turns an LLM feature into something qualitatively different. In a single-shot call the worst case is a wrong sentence. In a loop the worst case is a process that heads in the wrong direction for fifteen steps, spends money at each one, and changes real systems along the way. This article is about building that loop so it is safe and predictable. For the fundamentals underneath, start with LLM-based application development.
Note
"Agent" has been absorbed into marketing language and now gets applied to every LLM feature. Here I use it narrowly: a loop that calls tools and decides its own next step from the results.
Things that are not agents
Drawing the boundary is the easiest way to avoid unnecessary complexity. None of the following is an agent, and for most problems one of them is the right answer:
| Structure | What it does | Who decides | When it suffices |
|---|---|---|---|
| Single call | Text in, text out | Nobody — the flow is fixed | Summarising, classifying, translating |
| Chain | Several calls in a fixed order | The developer, in code | Known flows like extract → validate → format |
| Router | Picks a branch based on input | The model, but once | Sending a support request to the right team |
| Agent | Calls tools in a loop | The model, at every step | Tasks whose step count is not known in advance |
The rule of thumb: if you can draw the flow in advance, build a chain, not an agent. Chains are predictable, cheap, testable and easy to debug. An agent earns its cost only where you genuinely cannot say how many steps it will take or in what order.
The basic loop
However elaborate an agent architecture looks, it is a repetition of four steps:
- 1Think. The model sees the goal and the observations so far, and chooses the next action.
- 2Call. The model emits a tool name and arguments; your code runs the tool.
- 3Observe. The tool's result — or its error — is appended to the model's context.
- 4Decide. Is the goal met? If yes, stop and answer; if not, loop.
def run_agent(goal: str, tools: dict, max_steps: int = 8, budget_usd: float = 0.50):
messages = [{"role": "user", "content": goal}]
spent = 0.0
for step in range(max_steps):
reply, cost = model.call(messages, tools=describe(tools))
spent += cost
# A budget is a more reliable brake than a step count: one expensive
# tool can blow the bill in very few steps.
if spent > budget_usd:
return Halt(reason="budget_exceeded", step=step, spent=spent)
if reply.is_final:
return Done(answer=reply.text, steps=step + 1, spent=spent)
call = reply.tool_call
if call.name not in tools:
messages.append(observation(f"Unknown tool: {call.name}"))
continue
# Authorisation HERE, from the session — never from what the model proposed.
if not authorized(session.user, call.name, call.args):
messages.append(observation("You are not authorised for this action."))
continue
result = safe_invoke(tools[call.name], call.args)
messages.append(observation(result))
return Halt(reason="max_steps", step=max_steps, spent=spent)Those thirty lines contain every critical decision the rest of this article unpacks: a step limit, a budget limit, unknown-tool handling, an authorisation check and a safe invocation. Whether or not you use a framework, all five need an equivalent.
Tool definitions: all the model can see
The model never sees your tool's code. It sees the name, the description and the parameter schema. A tool definition is therefore not API documentation — it is an instruction to the model, and its quality converts directly into correct tool selection.
- One job each.
manage_orderis a bad tool;get_order,cancel_orderandupdate_shipping_addressare good ones. Models pick the wrong mode on multi-purpose tools. - Say when to use it. "Fetches an order" is not enough. "Fetches the status and line items of an order whose id is known. Does not search by customer name — use search_orders for that." What the model most needs is what the tool does not do.
- Narrow parameter types. Enums instead of free text, ISO dates, patterns for ids. The tighter the schema, the less the model invents.
- Concise results. Returning 400 lines of JSON fills the context and scatters attention. Return only the fields that affect the decision.
- Explanatory errors. "Error 500" teaches the model nothing. "Order not found; ids are 8 digits, got 'ABC'" lets it correct itself.
{
"name": "search_orders",
"description": (
"SEARCHES orders by customer name, email or date range and returns up to "
"20 summary records. For the detail of a single order use get_order. "
"Does not cancel or modify anything."
),
"input_schema": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Name, surname or email"},
"from": {"type": "string", "format": "date", "description": "YYYY-MM-DD"},
"to": {"type": "string", "format": "date"},
"status": {"type": "string", "enum": ["pending", "shipped", "delivered", "cancelled"]}
},
"required": ["query"]
}
}Mini task
Open one of your tool descriptions and add this sentence: "Do not use this for: …". It is usually the single change that most reduces wrong tool selection.
Why tool selection goes wrong
The commonest agent failure is picking the wrong tool, and the cause is almost always on the definition side. Four patterns recur in production:
- Too many tools. Past about fifteen, models start getting lost. The fix is to filter by context — if the user is on an orders screen, do not expose accounting tools at all.
- Tools that resemble each other. With both
get_userandfetch_user_profileavailable, the model picks at random. Merge them or make the difference explicit in the descriptions. - Invented arguments. The model may fabricate an order id it does not have. Fix it in code: return a clear error for ids that do not exist and let the model correct itself.
- Unnecessary tool calls. Reaching for a tool to answer something it already knows. Adding "do not call a tool if you can already answer" to the system prompt reduces this noticeably.
Stopping conditions: the most critical design decision
An agent that does not know when to stop is the most expensive class of production bug. The loop does not end on its own; you guarantee that it ends. There should be at least four brakes, and they complement rather than duplicate each other.
| Brake | What it prevents | Typical value |
|---|---|---|
| Step limit | Infinite loops | 5-10 steps; needing more means the task is too broad |
| Budget limit | A large bill in few steps | A fixed ceiling per task |
| Time limit | Requests hanging | Total wall-clock timeout |
| Repeat detection | Calling the same tool with the same arguments | Stop if the last 3 calls are identical |
The fourth is the most skipped and, in practice, the most useful. Agents that get stuck repeat the same action as though a different result might arrive. Keeping the names and an argument hash of the last few calls catches it — ten lines of code that save countless wasted loops.
Warning
Design what happens when a stopping condition fires. Quietly returning half an answer is the worst outcome, because the user assumes the job is done. "I could not finish this; here is what I did manage" is both honest and far more useful.
The system prompt: an agent's constitution
In a single-shot call the system prompt describes role and format. In an agent it does an extra job: it sets behaviour rules inside the loop. Without them the model makes decisions that are reasonable and unwanted.
- Do not call tools unnecessarily. "If you can answer from what you already have, do not call a tool." Simple, and it measurably lowers step counts.
- Ask rather than guess. "If it is unclear which order is meant, ask the user; do not pick one." The most irritating agent behaviour is filling a gap by invention.
- Do not repeat a call. The prompt-side counterpart of code-side repeat detection; the two layers work better together.
- Report failure. "If you cannot reach the goal, stop and say what you tried and what was missing." Stops half-finished tasks being presented as complete.
- Announce side effects. "Before changing anything, say in one sentence what you are about to change." Makes traces readable and feeds the approval screen.
Tip
Rerun your evaluation set as you add each of these. Agent prompts bloat especially fast, because adding a rule after every incident is tempting — and every rule dilutes the ones after it.
Memory: what an agent remembers
An agent needs two kinds of memory, and confusing them fills the context window fast.
Working memory is the observations within a task: which tools were called and what came back. It lives in the context and is discarded when the task ends. On long tasks even that bloats; the usual answer is summarising older observations. Be careful though — summarising tool results can lose critical detail like ids and numbers. Truncation is often safer: keep the first observation and the last three, replace the middle with "5 steps omitted".
Persistent memory carries across tasks: user preferences, conclusions reached in an earlier session. Keep that in a store rather than the context and retrieve it when relevant — which is to say, RAG. Injecting persistent memory into every task is both expensive and misleading, because irrelevant history distracts the model. The patterns in what RAG is apply directly.
Planning: up front or step by step
Two approaches, and the choice depends on the task.
Plan up front: the model lists all the steps, then executes them in order. The upside is inspectability — you can show the plan to the user for approval and estimate cost in advance. The downside is brittleness: an unexpected result at step three invalidates the plan and you need replanning logic.
Step by step: each step decides only the next one. It adapts to changing conditions, but where it is heading is not knowable in advance and cost estimation gets hard.
What works best in practice is a hybrid: a rough plan up front, step-by-step execution, replanning when needed. The plan is shown to the user and becomes an approval point; execution stays flexible; when a step surprises, the model updates the plan. Cap the number of replans too — a plan that keeps changing means the task was framed wrong.
Recovering from errors
Tools fail: network errors, rate limits, invalid arguments, authorisation denials. What the agent does about each is one of the details that separates good from mediocre.
The basic rule: show the error to the model in terms it can act on, but keep the retry decision in code. If a transient network error makes the model try the same call three times, three steps are wasted; do backoff-and-retry in code and hand the model only the final outcome.
| Error type | Who handles it | What the model is told |
|---|---|---|
| Network / timeout | Code (retry) | Only the final failure |
| Rate limit | Code (wait and retry) | Only the final failure |
| Invalid argument | The model | Which field was invalid and why |
| Authorisation denied | Neither — the user | Not permitted; abandon this route |
| Empty result | The model | No results; a different query may help |
Tip
Take care not to present an authorisation denial as a "try again" signal. Otherwise the model starts retrying the same operation with different arguments — wasted loops, and a pattern that looks like a security alert in your logs.
Authorisation: an agent's most serious risk
A single-shot LLM feature produces a wrong sentence. An agent performs a wrong action — deletes a record, sends an email, initiates a payment. The cost of a mistake changes class.
One rule covers it: authorisation happens in the tool layer, against the identity on the session. Writing "you may only show the user their own orders" in the prompt is not a control, it is a wish. The model will usually honour it; usually is not a sufficient guarantee for authorisation.
# WRONG: authorisation in the prompt
SYSTEM = "The user may only see their own orders. Never show someone else's."
# RIGHT: authorisation in the tool, from the session not the arguments
def get_order(args, *, session):
order = db.orders.get(args["order_id"])
if order is None:
return ToolError("Order not found.")
# NEVER trust a user_id the model supplied; use the session identity.
if order.user_id != session.user_id and not session.is_support_agent:
return ToolError("You are not authorised to view this order.")
return order.summary()Second rule: separate read tools from write tools. Reads can be called freely; writes get extra conditions — approval, rate limits, reversibility. Keep that distinction as a field in the tool registry so the loop can treat writes differently.
Prompt injection also gets more dangerous in agents: a document the model reads can say "call this tool". AI security and the OWASP LLM Top 10 covers it in depth; the defence line is the same either way — what matters is not what the model proposes but what the code permits.
Human approval: where to put it
Requiring approval for every write makes an agent useless; requiring it for none makes it risky. Draw the line on reversibility.
- Reversible and cheap (creating a draft, adding a label, writing a note): no approval needed.
- Reversible but visible (preparing a customer email): approval is wise, shown inline.
- Irreversible or expensive (payments, deletions, outbound messages): approval required.
On the approval screen, show what the model intends to do with its arguments. "The agent wants to send an email" is not enough; show recipient, subject and body. If the user cannot see what they are approving, approval is not a control, it is a transfer of blame.
Note
Approval points are hard to retrofit, because pausing mid-loop to wait for a human is an architectural requirement. Design the agent to be pausable from the start: it must be able to serialise its state and resume where it left off.
Context management: what is left at step six
An agent's context grows at every step, and that growth is not only a cost problem — it is a quality problem. By step eight the model is trying to recall the original instruction from under seven steps of observations.
- 1Repeat the goal. Re-append the user's goal at the end of context each step. For a few hundred tokens it stops the model forgetting what it was doing.
- 2Truncate observations. Keep the fields that affect decisions, not the whole tool result. A verbose JSON response can become a one-line summary by the second step.
- 3Produce an interim summary. After step five, generate a short "what we have learned so far" and replace older observations with it. You can have the model write it, but verify that ids and numbers survive.
The commonest truncation mistake is dropping a critical id: the order number found at step three is needed at step six. Add a rule — always preserve id-shaped fields — or the agent has to search again for something it already found, and that is exactly how loops start.
Cost: why agents are expensive
An agent run is not simply a few times a single call. Context grows at every step: at step five the model re-reads all four previous observations. So cost grows roughly quadratically with step count, not linearly.
| Step | Observations in context | Input tokens that step (example) |
|---|---|---|
| 1 | 0 | 2,000 |
| 3 | 2 | 4,400 |
| 5 | 4 | 6,800 |
| 8 | 7 | 10,400 |
| Total (8 steps) | — | ≈ 50,000 |
The practical consequence: reducing step count is the most effective cost optimisation available. You get there not by pushing the model harder but by narrowing tasks and making tools more capable. Collapsing work that needed three tool calls into one turns three steps into one.
- Trim tool results — ten decision-relevant fields instead of 400 lines of JSON.
- Truncate or summarise older observations; do not let context grow without bound.
- Use a small model for simple steps and a strong one for the decisions that matter.
- Cache the fixed system prompt and tool definitions — they are resent at every step.
To approach this as budget discipline, AI FinOps takes that angle.
Latency and user experience
An eight-step agent spending a few seconds per step leaves the user waiting half a minute. Half a minute behind a blank spinner is how products die.
The fix is to make the loop visible. Write what is happening at each step: "Searching orders…", "Checking shipment status…". That does three things at once: the wait feels shorter, the user sees progress, and when the agent heads the wrong way the user can intervene early.
A second technique: make long tasks asynchronous. The user starts the task, closes the screen and gets a notification when it finishes. Not every task has to be interactive; some work better in the background.
Observability: you cannot debug an agent without traces
For a single-shot call, logging the prompt and the response is enough. For an agent it is not; you need the whole run as a trace. When a user says it did the wrong thing, looking at the answer will not help — you need to see which step went off.
A trace needs: task id, step number, the model's stated reasoning, the tool called and its arguments, the tool result (truncated), step duration, step cost and the final status (done / step limit / budget / error).
$ curl -s localhost:8080/traces/tsk_91f2 | jq -r '.steps[] |"\(.n) \(.tool // "final") \(.ms)ms $\(.cost) \(.summary)"'1 search_orders 812ms $0.004 query="A. Yilmaz" → 3 results2 get_order 204ms $0.006 id=88213 → status=shipped3 get_shipment 190ms $0.008 carrier=X, last_scan=2 days ago4 search_orders 798ms $0.011 query="A. Yilmaz" → 3 results ← repeat5 search_orders 805ms $0.014 query="A. Yilmaz" → 3 results ← repeat6 halt(repeat_detected)# ✓ The model stalled at step 3: the shipment tool did not answer the# question, so it went back to searching. The problem is the tool.
Diagnosis from that trace takes three seconds. Without it, finding the same bug means hours of manual prompt-poking. In agent systems tracing is not an optional improvement; it is a baseline requirement.
Evaluation: how you test an agent
Evaluating an agent is harder than evaluating a single call, because the same goal can be reached by different routes and "the right answer" is not one string.
| Layer | Question | How it is measured |
|---|---|---|
| Outcome | Was the goal achieved | Is the post-task system state what it should be |
| Route | Did it get there sensibly | Step count, share of unnecessary tool calls, repeat rate |
| Cost | Is that acceptable | Mean and 95th-percentile cost per task |
The outcome layer matters most and automates most easily: build a test environment with mock tools, run the agent over fifty scenarios, and assert the expected end state. Mock tools also let you test failure paths — what does the agent do when a tool returns 500?
Do not dismiss the route layer. An agent reaching the right answer in twelve steps is four times more expensive and four times slower than one reaching it in three. Watching the step-count distribution catches quality regressions before outcomes break. AI agent evaluation and observability covers the tooling.
Running an agent in a test environment
The main obstacle to testing agents is that real tools have real side effects. The answer is to design the tool layer as swappable from the start: real tools in production, fakes honouring the same schema in tests.
How good those fakes are determines how much the tests are worth. A fake that always succeeds never exercises failure behaviour. Build at least three scenarios: happy path, empty result and error. The third matters most — the strangest agent behaviour shows up in error states.
class FakeOrderTool:
"""Same schema, controllable behaviour."""
def __init__(self, mode: str = "ok"):
self.mode = mode
self.calls: list[dict] = [] # the call log IS the test's output
def __call__(self, args):
self.calls.append(args)
if self.mode == "empty": return {"orders": []}
if self.mode == "error": return ToolError("Service temporarily unavailable")
if self.mode == "slow": raise TimeoutError()
return {"orders": [{"id": 88213, "status": "shipped"}]}
def test_agent_stops_on_empty_results():
tool = FakeOrderTool(mode="empty")
out = run_agent("Find this customer's order", {"search_orders": tool})
assert out.status == "done" # must not loop
assert len(tool.calls) <= 2 # more than two tries is waste
assert "could not find" in out.answer.lower() # must answer honestlyWhat those tests measure is not the text of the answer but the agent's behaviour: how many times it called, whether it stopped, whether it was honest. Testing text produces brittle tests; testing behaviour produces durable ones. The same split applies when you wire this into CI/CD — agent tests can be slow, so run fast behaviour tests on every PR and end-to-end scenarios nightly.
A case: the agent that repeated itself
The trace above shows a real failure class, so let us open it. An agent for a support team: find a customer's order by name and summarise the shipment status. Fine in testing. In production, for certain customers, it loops and hits the budget limit.
The trace gave the diagnosis: the model called get_shipment, which returned last_scan and carrier but no estimated delivery date. The summary we asked for included the delivery date. Unable to find it, the model went back to searching, got the same results and tried again.
What is interesting is that the model behaved sensibly. It searched again to find missing information. The fault was not in the model but in the mismatch between the tools and the task definition: the output we asked for could not be produced from the tools we provided.
The fix had three parts, all instructive. First, get_shipment gained an estimated delivery date — that is the actual solution. Second, the tool description gained "returns null when no estimate is available", so the model can treat absence as a result. Third, repeat detection went in so that a similar mismatch cannot burn a budget again.
Tip
When an agent loops, the first place to look is not the prompt but whether the tools can produce the requested output at all. Most "the model is being stupid" cases are missing-tool cases.
Agents that retrieve: combining with RAG
One of an agent's tools is almost always search. The combination pays off in both directions, and it has one trap.
The upside: an agent can do what single-shot RAG cannot — search, assess the result, notice what is missing and search again with a different query. Splitting a vague question into several retrieval rounds makes a clear difference on complex questions. The pattern is sometimes called agentic RAG and appears among the advanced variants in what RAG is.
The trap: when the search tool returns something, the model tends to treat it as correct. Even if the retrieved document is irrelevant, the model has text in hand and will try to build an answer from it. So the search tool must report emptiness and low relevance explicitly: "3 results found but none above the relevance threshold" is far better than silently returning three irrelevant chunks.
There is also a bounding problem: without a cap on retrieval rounds, a question with no answer makes the agent search forever. Limit rounds to two or three and let it say it could not find anything.
MCP: standardising tools
Every agent framework defines tools in its own format, so using the same tool in three systems means writing it three times. MCP (Model Context Protocol) is an open protocol that standardises that definition: write the tool once as a server and any client that speaks the protocol can use it.
For production teams the practical benefit lands in three places: keeping a central tool inventory, reusing the same tool across products, and connecting third-party tools through ready-made servers. The MCP and AI tooling page covers the scope.
Warning
Connecting an MCP server means exposing that server's tools to your model. Before connecting a third-party one, inspect what tools it offers and what they can do; a tool list is a new dependency and a new attack surface.
Side effects and isolation
If an agent touches real systems, bound what it can touch. A few practical patterns:
- Dry run. Give write tools a preview mode that returns what would happen without doing it. Useful for both testing and the approval screen.
- Operation caps. Limit how many writes a single task may perform. An agent updating a hundred records is almost never what anyone wanted.
- Undo log. Record every write with its task id so a run can be rolled back wholesale.
- Separate identity. Run the agent as its own service account with only the permissions the task needs, not with the user's full authority.
Agents that execute code need isolation too: run generated code in a separate, network-restricted, time-bounded environment rather than in your main process. For container fundamentals, what Docker is is a reasonable starting point.
Concurrency and race conditions
One agent on one task looks orderly. A hundred agents running at once brings classic distributed-systems problems back — and agents handle them especially badly, because a model does not understand a race condition.
- Two agents, one record. If two support tasks touch the same order, both may say "update the status". Put optimistic locking (a version number) on write tools; on conflict the tool errors and the model re-reads.
- Shared rate limits. A hundred agents hitting the same API at once exhausts the limit instantly and all of them fail. Route tool calls through a central queue; each agent retrying on its own makes this worse, not better.
- Cost blow-ups. Per-step budget checks work for one task; a hundred concurrent tasks can exceed a monthly budget in hours. Alongside a per-task budget you need a total concurrent spend ceiling.
The shared lesson: think of an agent not as a function but as a workload. Queues, rate limits, backpressure and budgets are classic infrastructure concerns, and none of them is solved by model quality. The framing in what SRE is applies directly.
Rolling out: graduated trust
Putting an agent straight into production with full authority is the most expensive way to learn. Mature teams climb a ladder, and each rung teaches something distinct.
- 1Shadow mode. The agent runs and logs its decisions but performs no actions. You learn — for free, on real traffic — how many steps it burns, which tools it picks and where it flounders.
- 2Suggestion mode. The agent says what it would do, a human does it. The acceptance rate becomes a quality metric: if humans accept ninety percent, you can climb.
- 3Narrow autonomy. Fully automatic for reversible operations only: drafts, labels, classification.
- 4Broad autonomy. Irreversible operations too, but with an approval threshold: automatic below a value or impact limit, human above it.
The most valuable rung is the first, and it is the one most often skipped. Shadow mode is the only mechanism that shows what production will look like before production. It is cheap to set up and teaches a great deal: a week of shadow running and reading traces beats a week of prompt tuning by an incomparable margin.
Note
Even in shadow mode, let the tools actually be called — reads for real, writes in dry-run. Shadow mode over synthetic data hides how messy real data is and gives false confidence.
Multi-agent: when it is genuinely needed
Having several agents talk to each other is an appealing idea and usually a premature one. A single agent with good tools handles most work; a second one adds coordination, cost and debugging difficulty.
Three situations where it genuinely helps: different tool sets (each agent owning a narrow domain), different context budgets (one agent chewing through a long document without polluting another's context) and different privilege levels (separating a read-only agent from one that writes).
Outside those, arrangements like "a researcher agent, a writer agent, a critic agent" usually reduce to steps of a single agent — and once reduced they are cheaper, faster and easier to debug. Multi-agent and A2A goes deeper.
Where the cost gets booked
A recurring management problem in agent projects: cost accumulates in one line item and nobody owns it. Computing cost per task and comparing it with the work itself is the best way to make that conversation concrete.
A simple frame: what does a task cost and how long does it take with the agent, how long would a person take, and how long does verifying the agent's output take? The third is the one everyone forgets. Output produced in one minute but requiring five minutes of checking is not always better than ten minutes of manual work — particularly when the checker is more senior than the doer.
Treat that calculation as a product decision: where does the agent genuinely win, and where does it merely look modern? The answer moves over time, so keep measuring.
Five classic mistakes
| Mistake | How it shows up | Prevention |
|---|---|---|
| No brakes | One task runs 40 steps, the bill climbs | Step + budget + time + repeat brakes |
| Authorisation in the prompt | The model reaches another user's data | Authorisation in the tool layer, from the session |
| No traces | "Why did it do that" has no answer | Step-by-step trace: reasoning, tool, arguments, result |
| Tools cannot serve the task | The agent loops | Derive the task from the tools, not the reverse |
| No approval point | An irreversible action happens silently | Approval tiers by reversibility |
The third row deserves emphasis: developing agents without traces is like debugging a distributed system without logs. Possible, needlessly painful, and every diagnosis is a guess.
When not to use an agent
This section matters as much as the rest. Agent architecture is complex, and there are plenty of situations where the complexity does not pay.
- The flow is fixed. If you know the steps, write a chain. An agent only makes a known flow unpredictable.
- Mistakes are expensive and irreversible. Financial transactions, production database changes — the model should not decide; at most it should prepare.
- Latency is critical. An eight-step loop is unacceptable on any synchronous request path.
- The tools are insufficient. As in the case above: if the requested output cannot come from the available tools, an agent is not a solution, it is a symptom generator.
- You cannot observe it. If you will not build tracing, do not run an agent; it becomes unmaintainable.
A pre-launch checklist
- 1All four brakes exist: step, budget, time, repeat.
- 2When a stopping condition fires, the user gets an honest message.
- 3Authorisation is in the tool layer, from the session; the prompt is not trusted.
- 4Tools are split into read and write; irreversible operations require approval.
- 5Every run is traced: step, tool, arguments, result, duration, cost.
- 6A fifty-scenario evaluation set runs against mock tools.
- 7Step count and per-task cost distributions are monitored.
- 8The agent runs as its own service account with least privilege for the task.
Mini task
If you already run an agent, add one thing today: repeat detection. If the last three tool calls share a name and argument hash, halt. Ten lines of code closes most of the most expensive failure class.
Starting small: what your first agent should be
Choosing the wrong first agent is the fastest route to a wrong impression of the whole technology. A good first agent has four properties: a narrow task, few tools (no more than three), reversible actions and an output whose correctness is easy to check.
Classic examples fitting that description: an agent that classifies an incoming support ticket and attaches relevant documentation links; one that reads an error report, pulls the related logs and writes a first diagnostic note; one that turns a meeting summary into draft task cards. All three are cheap when wrong and visibly useful when right.
The counter-example to avoid: making your first agent "a general assistant connected to every system in the company". That project stalls, because it asks you to solve tool design, authorisation, evaluation and user experience simultaneously. Start narrow; widen as you measure.
Common questions
Should I use a framework or write my own?
Write your first agent yourself. The thirty lines above show what an agent is more clearly than framework documentation can. As complexity grows — multi-agent, pausing, distributed execution — moving to a framework makes sense. Even then, move knowing which decisions it is making on your behalf.
How autonomous should an agent be?
The answer is commercial, not technical: what does a wrong decision cost? A mislabelled support ticket is cheap; a wrongly issued refund is not. Derive the autonomy level from that cost, not from how good the model is. The same model on the same task deserves different autonomy in different cost contexts.
How many tools is too many?
The practical threshold starts around fifteen; beyond that selection quality degrades. The fix is not fewer tools but context-dependent filtering: show the model only the tools that make sense in the current situation. It is the only approach that scales.
Who is responsible when an agent does something wrong?
Whoever operates the product — you. That is a legal question and a design one: it is the real reason irreversible actions need approval. "The model decided" is not a defence.
Can I combine an agent with a workflow engine?
Yes, and most mature systems do. The workflow engine owns the big picture — ordering, retries, persistence, scheduling — while the agent solves one uncertain step inside it. "Classify this document and route it" is the agent's job; the other ten steps belong to the engine. That division confines the agent's unpredictability to a single box.
How do I explain a decision the agent made?
Through the trace. If you store the model's reasoning at each step, "why did it cancel that order" has an answer. If you do not, it has none — and in a regulated domain that is a serious gap. Make the reasoning field part of the trace from day one; adding it later leaves past decisions unexplainable.
Can a small model run an agent?
With few tools and a narrow task, yes, and the cost difference can be substantial. Where decision points get complex, small models tend to pick the wrong tool and loop. A common middle path: routing and simple steps on a small model, decisions on a strong one.
Where this is heading
Agent architecture is moving fast, and rather than predicting, it is more useful to say which part stays fixed. Models are selecting tools better, following longer tasks and looping less; that will continue. But the party building the loop, fitting the brakes, checking authority and keeping the trace is you, and that responsibility does not shrink as models improve.
What does change is which tasks are economical for an agent. If something that takes eight steps today takes three next year, work done by hand because it was too expensive may become automatable. So record "an agent is not right for this" decisions with a date and revisit them every six months; some of them will flip.
What stays fixed is the line of responsibility: the distinction between what the model proposes and what the system permits. Teams that keep it sharp can widen autonomy confidently with each model generation; teams that do not rediscover the same risks with every release.
Next step
The agent is this pillar's front door. Three directions lead out of it: if you want to define tools with a standard protocol, MCP and AI tooling; if your agent needs access to knowledge, what RAG is; if you want to go deeper on security, AI security and the OWASP LLM Top 10.
But the most useful next step is probably not writing code: read twenty traces from an existing agent. Seeing where the model hesitates, which tool it picks wrongly and how many steps it takes builds an intuition no article can hand you. To return to the fundamentals, LLM-based application development is always here.
Last verified: 2026-09-26
Frequently Asked Questions
What is the difference between an AI agent and a chatbot?
A chatbot produces text; an agent calls tools and decides its own next step from the results. The difference is the loop: an agent continues until the goal is met or a stopping condition fires. If you can draw the flow in advance, you need a fixed chain, not an agent.
How do I stop an agent looping forever?
Fit at least four brakes: a step limit, a budget limit, a time limit and repeat detection. The fourth is the most skipped and most useful — halt when the last three tool calls share a name and argument hash. And when a stopping condition fires, tell the user honestly; never present a half-finished task as complete.
How do I limit what an agent is allowed to do?
Authorisation belongs in the tool layer, checked against the session identity. Writing "only show their own data" in the prompt is a wish, not a control. Separate read tools from write tools, gate irreversible operations behind human approval, and run the agent as its own service account with least privilege.
Why are agents so expensive?
Because context grows at every step, cost scales roughly quadratically rather than linearly: at step five the model re-reads all four earlier observations. The most effective optimisation is reducing step count — narrowing tasks, making tools more capable and trimming tool results.
How should I roll an agent out to production?
Gradually. Shadow mode first: the agent runs and logs decisions but performs no actions. Then suggestion mode (a human executes), then narrow autonomy on reversible operations, and finally broad autonomy with an approval threshold. Shadow mode is the most valuable rung and the one most often skipped.
Related articles
LLM Application Development: A Ground-Up Start to AI Engineering
What AI engineering is and how an LLM feature travels from idea to production: context engineering, structured output, evaluation, cost, security and a 90-day plan.
What Is RAG? Retrieval-Augmented Generation in Production
What RAG is and when you need it: chunking, embedding choice, hybrid search, reranking, authorisation, evaluation and the failures that show up in production.
SRE Nedir, Ne İş Yapar? DevOps'tan Farkı ve Modern Güvenilirlik Mühendisliği
Site Reliability Engineering (SRE) nedir, bir SRE gün içinde ne yapar, SLO/hata bütçesi/toil gibi kavramlar ne anlama gelir ve SRE ile DevOps arasındaki fark tam olarak nedir? Sıfırdan, örneklerle ve kariyer yol haritasıyla kapsamlı bir Türkçe rehber.
What Is Docker? The Fundamentals
Docker is the container technology that runs your app the same way everywhere. We explain images, containers, and Dockerfiles in the simplest terms, with command examples.
Reading isn't enough — do it.
Practice these topics in an interactive terminal in your browser.