How to Build AI Agents: A Practical Guide & Best Practices

An AI agent is an LLM that decides its own steps — it plans, calls tools, looks at the result, and adapts until the job is done. That autonomy is powerful, but it also makes agents slower, more...

Updated for Orrery 0.6.0 (October 2026). Every practice here is applied in Orrery, an open-source on-call agent that investigates before it acts. The Building AI Agents for DevOps series shows each one in code, most recently streaming the agent's work and opening the platform without opening the gates.

An AI agent is an LLM that decides its own steps — it plans, calls tools, looks at the result, and adapts until the job is done. That autonomy is powerful, but it also makes agents slower, more expensive, and less predictable than plain code. The whole craft of building good agents is knowing where to spend that autonomy and where to take it away.

This is a short, framework-agnostic guide to the decisions that matter. No specific stack, no product — just the patterns that hold up in production.

1. Workflow or agent? Decide this first

Before writing a line of code, figure out how much autonomy the task actually needs. The honest default is the simplest thing that works.

• One LLM call + a tool handles a surprising amount — classification, extraction, a single lookup-and-answer.

• A workflow (your code orchestrates a fixed sequence of LLM calls — chain, route, parallelize) covers anything where you already know the steps.

• A true agent earns its keep only when the steps can't be known in advance and the task is valuable enough to justify the cost.

In practice, the best systems are hybrids: a deterministic skeleton you control, with autonomous LLM steps inside each phase. Let the model reason where reasoning is needed; keep everything else in code.

2. Decompose — but not too much

One giant agent with 40 tools picks the wrong tool and confuses its own parameters. The fix is to split by domain: one orchestrator that reads intent and routes, plus a few specialists, each with a tight tool set and its own prompt.

But splitting has a cost too — every handoff between agents loses context. Naive over-splitting measurably degrades reasoning. The guidance that holds up:

Add a sub-agent only when it pays for the context it loses.

3. Design tools for the model, not for yourself

Tools are how the agent touches the real world, and they are the single biggest lever on reliability. A tool is just a function with a clear name, a docstring the model reads, and typed arguments. Two rules matter most:

Return structure, not dumps. Always include a status field so the model knows whether the call worked, and truncate large output — a 10,000-line log pasted into context is the fastest way to derail an agent.

def get_service_health(name: str) -> dict:
    """Return the health of a service. Use before diagnosing an incident."""
    try:
        data = client.check(name)
        return {"status": "ok", "healthy": data.up, "replicas": data.replicas}
    except ServiceError as e:
        return {"status": "error", "message": str(e)}   # never raise into the model

Put deterministic logic in code, not in the prompt. Verdicts, validation, budget checks, "is this done?" — these should be tool results the model reads, not judgments you ask the model to make. Models are great at deciding what to do and bad at being a reliable state machine.

4. Context is your scarcest resource

Context bloat is the number-one cause of both cost and flakiness. The model pays — in money and attention — for every token you put in front of it. Guard the window:

• Truncate tool output and return summaries, not raw data.

• Give the agent a scratchpad (a working file or state field) instead of carrying everything in the conversation.

• Inject the facts it would otherwise invent — the working directory, today's date, available resources. Models hallucinate paths and names when you don't.

Structure prompts with named, tagged sections so they stay readable and the model can be pointed at them precisely:

<role>         Who the agent is and what it's responsible for.
<workflow>     The ordered steps to follow.
<constraints>  Hard limits: forbidden actions, "ambiguous is a valid answer."

5. Route by complexity — don't pay top-tier for everything

Not every step needs your most capable model. Define tiers and resolve the actual model from config, never hardcoded in the agent:

A common, effective pattern: a cheap model gathers context, an advanced model acts on it. Because the model is just config, swapping or upgrading one becomes a single change — re-run your evals before and after, never swap blind.

6. Guard the exits and keep a human at the boundary

An autonomous agent will, given the chance, declare success it can't prove or loop forever. So the limits live outside the model, in code:

• Code-verified exits — the "done" status is something your code checks, not something the model asserts.

• Budgets outside the loop — cap iterations and spend in code; running out should write a clear failure, not a fake success.

• Human at the boundary — for anything irreversible, the agent proposes (a draft, a diff, a request) and a person approves. Confirm before acting outward.

• Fail fast at boot — check every credential and connection before the first expensive LLM call.

And treat all external content as untrusted. Agents read web pages, files, and logs — any of which can contain instructions trying to hijack them. State explicitly, up front, that data is data and is never to be obeyed as a command. This is your first line of defense against prompt injection.

7. Evaluate behavior — don't unit-test it

You can't assert exact string equality on an LLM's output; phrasing varies run to run. Instead, build evalsets — collections of realistic scenarios scored on what actually matters:

• Did it call the right tools, in a sensible order? (trajectory)

• Is the final answer correct and grounded? (often judged by another model against a rubric)

• What did it cost?

Run the evalset whenever you change a prompt, tool, or model. It's the only way to know an "improvement" didn't quietly break something — and it turns the build loop from vibes into measurement.

The short version

Build the smallest agent that does something real, measure it, and add autonomy only where it pays off. That single discipline separates agents that ship from demos that don't.