Building AI Agents for DevOps Part 5: Sub-Agents vs AgentTool

In Part 4, I added Prometheus metrics. The platform now monitors itself. But while reviewing the architecture, I noticed a design flaw hiding in plain sight: I was using the same delegation pattern...

In Part 4, I added Prometheus metrics. The platform now monitors itself. But while reviewing the architecture, I noticed a design flaw hiding in plain sight: I was using the same delegation pattern for two fundamentally different routing modes.

This post fixes that — by applying the right Google ADK pattern to each situation.

The Problem: One Pattern, Two Jobs

Here's how the devops-assistant orchestrator looked before this change:

root_agent = create_agent(
    name="devops_assistant",
    tools=[],
    sub_agents=[
        incident_triage_agent,   # deterministic pipeline
        kafka_agent,             # LLM picks this
        k8s_agent,               # LLM picks this
        observability_agent,     # LLM picks this
        docker_agent,            # LLM picks this
        journal_agent,           # LLM picks this
    ],
)

Six agents, all in sub_agents. But they serve two different purposes:

• incident_triage_agent — A deterministic SequentialAgent pipeline. It always runs the same steps: parallel health checks, then summarize, then save. The orchestrator controls execution order.

• The other five — Specialist agents. The LLM reads their descriptions and picks the right one based on the user's query. This is dynamic routing, not fixed orchestration.

I was using sub-agents for both. Google's ADK has a better pattern for the second case: AgentTool.

Sub-Agents vs AgentTool: The Decision

The ADK offers two ways to compose agents. The key question: who decides the routing?

Sub-agents

Agents listed in a parent's sub_agents parameter. Used with SequentialAgent, ParallelAgent, and LoopAgent for deterministic workflows. All sub-agents share the same session state — earlier agents write results via output_key, later agents read them.

This is the right pattern here. The execution order is predetermined. The LLM doesn't decide which health checker to run — all four run in parallel, every time.

AgentTool

An agent wrapped in AgentTool() and placed in the parent's tools list. The parent LLM decides whether and when to invoke it, just like calling a function.

User: "what's the consumer lag on the orders topic?"

Root LLM thinks: this is a Kafka question
Root LLM calls: AgentTool(kafka_health_agent)
                 → kafka_health_agent runs get_consumer_lag("orders")
                 → returns result to root

This is the right pattern for specialist agents. The LLM routes based on intent, not a fixed sequence.

The Differences That Matter

The single parent constraint is worth highlighting. In ADK, an agent instance can only be added as a sub_agent once. If I ever wanted kafka_agent used in another orchestrator, the old approach would throw an error. AgentTool has no such limitation.

The Refactoring

Three files changed. Here's the core of it.

1. Widen the factory's type hint

The create_agent() factory accepted Sequence[Callable] for tools. AgentTool extends BaseTool, not Callable, so I widened the type:

# core/ai_agents_core/base.py
from google.adk.tools.base_tool import BaseTool

def create_agent(
    *,
    tools: Sequence[Callable[..., Any] | BaseTool],  # was: Sequence[Callable]
    ...
)

And re-exported AgentTool from the core package so agents can import it cleanly:

# core/ai_agents_core/__init__.py
from google.adk.tools.agent_tool import AgentTool as AgentTool

2. Refactor the orchestrator

# agents/devops-assistant/devops_assistant/agent.py
from ai_agents_core import AgentTool, create_agent

root_agent = create_agent(
    name="devops_assistant",
    tools=[
        AgentTool(agent=kafka_agent),        # LLM-routed
        AgentTool(agent=k8s_agent),          # LLM-routed
        AgentTool(agent=observability_agent), # LLM-routed
        AgentTool(agent=docker_agent),        # LLM-routed
        AgentTool(agent=journal_agent),       # LLM-routed
    ],
    sub_agents=[
        incident_triage_agent,  # deterministic workflow — stays as sub-agent
    ],
)

The separation is now self-documenting. sub_agents = fixed workflows. tools = LLM-routed specialists.

3. Update the instruction

The root agent's instruction now explicitly distinguishes the two modes:

## Structured Workflows (sub-agents)
- incident_triage_agent: Runs a comprehensive health check across
  Kafka, K8s, Docker, and Observability in parallel, then summarizes
  and saves a report.

## Specialist Tools (agent tools)
Call these tools for targeted queries on individual systems:
- kafka_health_agent: Kafka cluster health, topics, consumer groups, lag.
- k8s_health_agent: Kubernetes cluster info, nodes, pods, deployments.
- observability_agent: Prometheus metrics/alerts, Loki logs.
- docker_agent: Docker containers, logs, stats, compose status.
- ops_journal_agent: Notes, past findings, session activity.

The Updated Architecture

How to Choose: A Quick Checklist

Next time you're composing agents in ADK, ask yourself:

If you're mixing both patterns in one orchestrator — as I am — that's fine. Use sub_agents for your workflows and tools with AgentTool for your specialists. The two patterns compose naturally.

What Didn't Change

From the user's perspective: nothing. The same queries hit the same agents and return the same results. The change is structural — correct semantics, better reusability, cleaner separation of concerns.

All 308 tests pass unchanged. The refactoring touched the composition layer, not the tool implementations.

What's Next

With the architecture cleaned up, Part 6 takes a security pass over every tool — input validation, a confirmation-bypass fix, secret redaction, and role trust. Verifying that the LLM actually routes to the right specialist comes in Part 8 with agent evals.

The Series

Building AI Agents for DevOps is a 12-part series; every part is linked below.

Source code on GitHub. Architecture decision: ADR-002. Google's reference: Sub-agents vs agents-as-tools.