Agents in the Open Part 1: A Sandbox on the Public Internet

For seventeen parts, Building AI Agents for DevOps described an agent that ran on my machine. This series is about running it where other people can reach it. The first step was the Orrery Lab:...

For seventeen parts, Building AI Agents for DevOps described an agent that ran on my machine. This series is about running it where other people can reach it. The first step was the Orrery Lab: lab.tirraflow.com, a live Orrery that investigates a real, seeded incident, hosted on the same VPS as this blog.

Availability (October 2026). The Orrery Lab is live but not open to readers yet: only Tirraflow admins can sign in while I watch its cost and behaviour. I will update this note when it opens. To try the same incident today, run make install && make demo from the Orrery repository with your own model API key.

An agent that can call tools is not a website. Put one on a public URL and four questions need an answer that doesn't depend on the model behaving:

What can it reach? Anything on its network is one tool call away.
What can it change? A prompt that says "read only" is not a control.
Is it the code I reviewed? The image could have been replaced between release and deploy.
Can it hurt the blog? Same host, same nginx, same Keycloak.

The rule for every answer: enforce the boundary outside the agent. Each one below holds even if the model is fooled, the code has a bug, or a tool does something nobody expected.

Orrery Lab network: Cloudflare to nginx, two internal networks for the lab, and one egress proxy that only allows api.anthropic.com

What It Can Reach: Networks With No Way Out

Every lab service sits on Docker networks marked internal: true, which have no route to the internet:

lab-core holds the incident: Kafka, Prometheus, Alertmanager and the seeder that creates consumer lag.
lab-front holds what the outside world touches: Orrery, nginx (to serve it), Keycloak (whose signing keys verify tokens) and the egress proxy.

The blog's Postgres and API are on neither. From inside the Orrery container they don't even resolve. Keycloak is the one blog service the lab can talk to, because Orrery needs its signing keys (nginx is there too, and refuses anything not from Cloudflare). The lab's only way out is a Squid proxy, the one container also attached to an ordinary network, and it allows exactly one destination:

acl llm_api dstdomain api.anthropic.com
acl ssl_port port 443
http_access deny !ssl_port
http_access allow CONNECT llm_api ssl_port
http_access deny all

Orrery finds it through HTTPS_PROXY; ignoring that variable gains nothing, because there is no other route.

The proxy also showed me what default-deny is for. The first answers took several seconds longer than they should have. The proxy log showed why: LiteLLM was downloading its model price list from GitHub on first use, being refused, and retrying three times. Without the proxy I would never have known the dependency phoned home. One environment variable (LITELLM_LOCAL_MODEL_COST_MAP=True) makes it use its bundled copy.

What It Can Change: Nothing, Enforced Three Times

The lab is read-only, and that is enforced at three independent layers:

Autonomy level L2. Orrery's gates refuse every tool labelled @confirm or @destructive, for every role. This is the fail-closed level from Part 15.
Specialists switched off at startup. Docker, Kubernetes and Elasticsearch are excluded by ORRERY_DISABLED_SPECIALISTS, so they are never routed to or swept, and their tools aren't offered to the agents. An admin in the console can switch more specialists off, but never switch back on one the environment excluded.
Nothing to change. No Docker socket is mounted, the root filesystem is read-only, every Linux capability is dropped, and the container has 1 GB of memory and 1.5 CPUs.

Any one of these alone would stop a restart; three means a bug in one doesn't matter.

Is It the Code I Reviewed: Verify, Then Pin

Orrery's release workflow signs each image with Cosign, keyless, through GitHub's OIDC. The lab's deploy workflow checks the signature before it starts anything, and it checks who signed: the release workflow, at that exact tag.

cosign verify \
  --certificate-identity "https://github.com/BAHALLA/orrery/.github/workflows/release.yml@refs/tags/v${ORRERY_VERSION}" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  "$DIGEST"
# Pinned by digest for this run, so what was verified is what runs.
export ORRERY_IMAGE="$DIGEST"

Verifying a tag and then running the tag leaves a window in which the tag can move; running the verified digest closes it.

Can It Hurt the Blog: Make the Lab Optional

The lab shares nginx with the blog, so a lab problem must never become a blog problem:

nginx resolves the lab per request, not at startup. A static upstream makes nginx refuse to start when the orrery container is missing, and that would take the blog down with it. When the lab is down, readers get a short 503 page that points to the articles.
It is a separate Compose profile, started and stopped only by its own manual workflow. A blog deploy never touches it, and the two workflows share one queue so they never run on the VPS at the same time.
Rate limits at both ends: nginx limits requests per IP, and Orrery allows five chat messages per minute per user.
Spend has a ceiling. The model is Claude Haiku 5.5. The API key lives only in a GitHub secret and a lab.env file on the VPS (written with umask 077), and the real cap is the key's workspace spend limit.

The Pattern

Each control here is something the agent can't talk its way past: a network with no route, a gate that reads a label, a signature tied to a workflow, a proxy with one allowed host. The agent's own instructions still say it is read-only, but nothing depends on that.

One question is still open: who may use it. The lab signs people in with the blog's own Keycloak, and that turned out to have a trap that could have made every blog admin an Orrery admin. That's Part 2.