The Agent Gateway: One Turn Pipeline Behind Every Surface

Most writing about agent architecture stops at the agent — the prompt, the tools, the orchestration graph. That's the part that demos well. But once an agent is something people actually use, the...

Updated for Orrery 0.6.0 (October 2026). The gateway gained three jobs in 0.6.0. It streams turns as they happen (Part 16). It sets whom downstream calls act for, from the transport's verified identity only. And it can swap its root agent live when an admin edits the catalog (Part 17). The shape described here did not change, which is the point of having it.

Most writing about agent architecture stops at the agent — the prompt, the tools, the orchestration graph. That's the part that demos well. But once an agent is something people actually use, the demo is maybe a fifth of the code, and almost none of the incidents.

The rest is exposition: the layer between a message arriving from somewhere and a turn running against a model. I run one agent that answers in Slack, in Google Chat, in a web console, in a terminal, and on a cron schedule with no human present. Five surfaces. For a while, each one had its own copy of the same five steps.

This post is about the layer that replaced them, why it's the only place certain problems can be solved, and the two that turned out to be hardest: human approval and horizontal scale.

The N×M Problem

Every transport, independently, did the same five things: build a runner, work out who is talking, map this conversation to a session, run the turn, and walk the event stream to assemble a reply. None of that is channel-specific. What is channel-specific is narrow — how to decode an inbound payload, and how to render a reply as JSON, as Slack mrkdwn, as a Chat card.

The cost of duplicating the other 90% isn't the line count. It's that a fix lands in one place, and nothing tells you about the other four — as the two red rows above show. Reply extraction has to skip the model's thinking parts or planner output leaks into user-facing text; the filter got written once, in the transport that hit the bug first. And a destructive tool reached over HTTP bypassed the confirmation gate entirely, because the gate had been wired into the chat path and the HTTP path built its own runner.

That second one is the shape of the whole problem. When a security control lives in the transport, "which transports have it?" is a question you can only answer by reading all of them.

The Pattern: Ports and Adapters, Named for Agents

The fix is unglamorous and old — hexagonal architecture, applied to the turn. One AgentGateway owns the pipeline; each surface supplies only what is genuinely its own.

Four types carry it. Two are data:

@dataclass
class InboundMessage:
    text: str
    user_id: str            # verified id — JWT subject, Slack user, Chat email
    conversation_key: str   # opaque, channel-defined: "{channel}:{thread}"
    channel: str            # "http" | "slack" | "google_chat" | "cli"
    state_delta: dict       # identity + per-turn context, applied by the gateway
    raw: Any = None

Two are protocols — ChannelAdapter (parse / deliver) and SessionResolver (resolve). And the whole flow is three lines:

async def dispatch(self, adapter: ChannelAdapter, raw: Any) -> Any:
    msg = await adapter.parse(raw)
    if msg is None:
        return None                       # bot's own message, an edit, a retry
    reply = await self.run(msg, on_event=getattr(adapter, "on_event", None))
    return await adapter.deliver(reply, msg)

Two details in there are worth more than they look.

parse may return None. Every chat transport needs to ignore some payloads — its own messages, edits, reactions, joins. Making "ignore" a first-class return value keeps that decision in the adapter, where the channel semantics live, instead of as an early return buried in a handler that also runs agents.

on_event is optional and duck-typed. Google Chat renders a progress card that updates while the agent works; Slack and HTTP don't. Rather than force every adapter to implement a streaming hook, the gateway looks for one. Surfaces that want progressive output get it; surfaces that don't, don't mention it.

Identity Belongs to the Gateway, Not the Agent

Here's the first thing that cannot be solved anywhere else.

Authorization needs a trustworthy answer to "who is this?" In an agent runtime, the tempting place to put that answer is session state — it's already there, tools can read it, the model can see it. It is also exactly the wrong place, for a reason that takes one sentence to state and is easy to miss:

Session state is not a trust boundary. Tools write to it, and tool output is attacker-reachable text.

A pod annotation, a log line, a Kubernetes event, an Elasticsearch document — all of it arrives with a tool result's authority, and any of it can be authored by someone who is not your user. If user_role: admin in session state is sufficient to be an admin, then anything that can influence a write to state is a privilege escalation.

So the gateway is the only writer of identity, and it marks its writes:

delta = dict(state_delta or {})
delta.setdefault(ACTOR_STATE_KEY, user_id)   # the verified sender, this turn

Roles go in through a set_user_role() helper that also stamps _role_set_by_server; a role that appears in state without that stamp is ignored and the request is forced down to viewer. Same defence for the autonomy level — a bare value in state is dropped with a warning rather than promoting a read-only deployment to one that can delete things.

Note the mechanism: identity is applied as a state delta on the run call, not by mutating the session. The transport hands the gateway a fact about this turn; the gateway hands it to the runtime; nobody reaches into the session and writes. That's what makes "the server set this" a checkable property instead of a convention.

It also fixes something subtler. In a shared Slack thread, three people talk to one agent. Stamp the actor per turn and the agent knows who it's answering right now — so it acts for the current sender, and never reports its own service account as "you".

Confirmation: The Hard One

This is the section I'd keep if I had to cut the rest.

The requirement sounds trivial: before the agent restarts a deployment, a human says yes. The standard implementation — the one most frameworks hand you, and the one I shipped first — is model-mediated, and it ends with the gate treating "the model called the tool a second time" as evidence that a human approved. Those are not the same event. The model decides to re-call; whether a human said anything, and what they said, is something the gate never observes.

The fix is to make the gateway a witness — to record the human decision at the boundary where a verified person actually spoke, and require the gate to match against it:

if self.verified_confirmation:
    delta[CONFIRMATION_STRICT_STATE_KEY] = True
    decision = classify_decision(text)          # approve / deny / None
    delta[CONFIRMATION_DECISION_STATE_KEY] = (
        {"decision": decision, "by": user_id, "timestamp": time.time()}
        if decision else None
    )

Four properties fall out of those seven lines. The first two are readable off the diagram: the decision has an author, so a colleague who wandered into the thread cannot approve your deployment restart (denial stays open to anyone — stopping an action is not a privilege); and the words are deliberate, because in a chat thread "ok" and "yes" are ambient noise that people say to each other.

The other two are the ones I'd have got wrong without being bitten.

The decision is rewritten every single turn — including to None. This looks like defensive noise and isn't. If a stale "approve" is allowed to sit in state, it authorizes the next guarded action the model chooses, which the human never saw. Clearing on every non-decision turn means a decision is only ever valid for the turn it was spoken on.

A decision cannot authorize an action that didn't exist yet. The gate's final check is one comparison:

and decided_at >= pending.created_at

Without it, "approve" said before any pending existed — for something else, for nothing at all — would authorize whatever guarded call the model makes next, because the args-hash it's matched against belongs to a pending raised on that very turn. The human must have been able to see the action before approving it. That's the property, and it's one >=.

Where the pending lives (and why not in the session)

One more piece, and it took a production regression to learn. Guarded tools are routinely reached through a sub-agent — an AgentTool call — whose session is throwaway. A pending confirmation written into session state on the request turn is simply gone by the approval turn. The handshake worked in direct calls and silently failed through delegation, which is the harder half of the system to test and the more common half in production.

So pendings live in a dedicated store keyed by an action_id, scoped by whatever the transport can verify: the requester's id for HTTP and CLI, the thread or space for the chat bots. Two flows over one surface — single-phase, where the decision arrives on the same turn that retries the tool (one atomic check-and-remove), and two-phase, where the decision is a separate transport event (a card click, a thread reply) that marks the entry approved for the model's retry to consume.

Scaling: The Gateway Is Where the State Concentrates

Push every shared concern into one layer and you've also collected every piece of state that stops you running two replicas. That's not a flaw in the pattern — it's the pattern doing its job, making the constraint visible in one file instead of five. But you have to then go and deal with it.

Two of those rows carry an idea the table can only gesture at.

The pending-confirmation store's failure mode is benign-looking, which is why it survives a load test. A pending raised on pod A is invisible to the pod that receives the approval; the user approves, and the agent just asks again. Nothing errors. The Postgres backend fixes it by putting the one-shot guarantee on the database — consume and pop resolve their target row inside a single transaction (a DELETE … RETURNING), so two racing replicas cannot both consume the same approval and run the tool twice. Traffic here is human-scale, a handful of single-row statements around a decision that takes seconds, so a synchronous engine inline in the callback path costs nothing measurable.

Idempotency's whole design is one asymmetry. Pub/Sub, SQS, and most webhook retry policies deliver at least once. For a read-only agent a duplicate is a wasted LLM call; for an agent that can restart a deployment, a duplicate is a second restart. The guard runs at the transport edge, before any agent code:

if not await store.claim(event_id):
    return  # already processed or in flight — ack and drop
try:
    await gateway.dispatch(adapter, event)
except Exception:
    await store.release(event_id)   # failed — let redelivery retry it
    raise
# success: keep the claim until TTL, so genuine duplicates stay blocked

Success keeps the claim; failure releases it. That asymmetry is the design — a redelivery after a crash does retry the work, while a redelivery after a success does not repeat it. And when the event source gives you no stable id, derive one: a hash of space, thread, creation time, and text is stable across redeliveries of the same logical event and distinct for genuinely different ones.

The session-map row hides a bug worth naming outright. The map must be keyed by (user_id, key), not key alone. The runtime scopes every session by (app, user, session), so handing user B a session id created for user A doesn't share the conversation — it resolves to nothing and quietly creates a fresh empty session behind the same id. Keying by conversation alone never shared history between participants; it only hid the fact that each one has their own. Sessions in a shared thread are per-participant. The key should say so.

Security: What the Gateway Owns, and What It Doesn't

The gateway is not the security layer. It's the layer that establishes the facts the security layer needs, and getting that split right matters more than any individual control.

The line is: the gateway answers questions only the transport can answer. Everything downstream of that is a cross-cutting concern that belongs in the runtime's plugin chain, where it applies once and applies everywhere — including to agents written later by someone who never read this article.

One nuance the diagram flattens. RBAC answers who; the autonomy level answers what this process is allowed to do at all. They're orthogonal, and you want both — an admin running a scheduled sweep should still be unable to delete anything, because the sweep is read-only by construction, not because the person is untrusted.

And the exception in the amber band is the one that keeps costing me. The confirmation gate cannot be a plugin: it has to work inside sub-agent sessions and in bare developer tooling, neither of which goes through plugin registration. So it's wired per agent — and an agent added without that line keeps RBAC and silently loses confirmation. It has happened twice. The only durable answer I've found is a test that walks the agent tree and fails the build on an ungated agent. Extend the walker; never the exception list.

When You Don't Need This

Honestly: one surface, one replica, read-only tools, one user. Then the gateway is ceremony, and a handler that builds a runner and returns a string is the right amount of code.

The pattern starts paying at the second surface, and it pays a lot at the first mutating tool. If I were starting again I'd still write the direct version first — but I'd extract the gateway the day a second transport appeared, not the day the second transport's copy of the confirmation logic turned out to be missing.

The Takeaway

• Exposition is architecture. The path from "a message arrived" to "a turn ran" holds identity, approval, sessions, and idempotency. Duplicate it per transport and you've duplicated your security posture per transport.

• Only the boundary can establish who spoke. Everything downstream — RBAC, autonomy, audit — is consuming a fact that must be produced exactly once, by the only layer that can verify it.

• A model re-calling a tool is not a human saying yes. If your approval flow can't name who approved, when, and which action they could see at the time, it isn't an approval flow.

• Centralizing state doesn't create the scaling problem, it reveals it. One in-memory dict is the difference between a system that runs one replica and one that runs ten. Much better to have it in one file.

The agent is the part everyone writes about. The gateway is the part that decides whether it survives contact with a second user.

Source on GitHub — the gateway is core/orrery_core/serving/gateway.py, the confirmation gate security/guardrails.py, the pending store security/confirmation_store.py.