Agent Sandbox Egress: Troubleshooting Guide

What this covers

You run agent-generated code in a sandbox. The sandbox has a filesystem and a process boundary. Something about the network still bothers you, or an incident already happened. Each section below is one symptom, the cause it usually points to, a check you can run today, and the fix.

The threat model behind all five: the agent’s input is untrusted. Anything the model reads — a web page, a dependency README, an issue body — can carry instructions, and prompt injection has no reliable fix at the model layer. Simon Willison’s running catalogue of prompt injection attacks and failed mitigations is the best single place to watch that assumption stay true. The OWASP Top 10 for LLM Applications covers the same ground as a checklist. Treat the sandbox as the control, not the model’s judgement.

Symptom: strong isolation, and data still left the box

The sandbox had a read-only root, a fresh container per task, no persistent volume, no shared kernel with anything that matters. Data still reached a host you did not choose.

Likely cause. Isolation of the filesystem and process without any egress control still leaves a path for data to leave, because a single outbound request is enough to exfiltrate. Every containment property you bought applies to what the workload can touch locally. None of it constrains what the workload can send. A base64 blob in a query string, a DNS lookup of <secret>.attacker.example, a webhook POST — one request, and the isolation tier you paid for was irrelevant to the outcome. The asymmetry is the whole problem: containment is measured in layers, exfiltration is measured in one packet.

Check. Run a real task in the sandbox and capture every outbound connection at the host or network layer, not from inside. On Linux, conntrack -L on the host, or a tcpdump on the sandbox’s veth interface, both work. Count the distinct destination IPs and ports. Then compare that list against the hosts the task actually needed — usually your model provider, a package registry, and your own API. If the captured list is longer than the needed list, you have no egress control, whatever your isolation tier says.

Fix. Add an egress policy before you add another isolation layer. Concretely: put the sandbox on its own network namespace with a default-drop OUTPUT/FORWARD policy, and route what remains through a proxy you control. This is the same reasoning behind choosing serverless or long-running hosts for agent workloads — the network shape of the host decides how easy this is to enforce, so decide it before you scale the fleet.

Symptom: the security review keeps asking for a stronger sandbox

You are being asked to move from containers to microVMs, or from microVMs to dedicated hardware, and the work is quarters, not days.

Likely cause. The risk being described is data leaving, not code escaping, and the proposed remedy addresses the wrong axis. Default-deny with an explicit allowlist of hosts the task genuinely needs is far cheaper to implement than upgrading the isolation tier, and removes more risk. A stronger boundary raises the cost of a kernel escape — a real but rare and expensive attack. An egress allowlist eliminates the common one: untrusted content telling the agent to send your data somewhere. You get the larger reduction from the smaller project.

Check. Write down, for one real task class, the exact list of hostnames it needs. If you cannot produce that list in an afternoon, that is itself the finding — you do not know what your agents talk to, and no isolation tier fixes that. If you can produce it, and it is under about a dozen entries, an allowlist is a configuration change and the isolation upgrade is a roadmap item.

Fix. Default-deny outbound, then allowlist by hostname at a forward proxy the sandbox must use (no direct route out, HTTP_PROXY is not enough — see the last section). Keep the allowlist per task class, not global; a documentation-summarising agent and a dependency-upgrading agent need different registries. Log every denied connection: the deny log is how the allowlist gets accurate, and it doubles as a signal that something is trying to reach where it should not. Feed those denials into whatever you already use for attributing agent cost and latency to the work that caused it so a policy change shows up next to the run that triggered it.

Symptom: HTTP is locked down and data is still moving

The proxy allowlist is in place and enforced. Traffic analysis shows no unexpected HTTP or HTTPS destinations. Something is still off — a canary token fired, or resolver logs show volume that no task explains.

Likely cause. DNS resolution is a covert channel in its own right, so an egress policy that filters HTTP but leaves DNS open is incomplete. The mechanism: the sandboxed process does not need a reply to leak. It encodes data into a subdomain label and resolves <payload>.exfil.example. Your resolver dutifully walks the delegation chain and hands the query to the attacker’s authoritative nameserver, which reads the payload and returns NXDOMAIN. No connection was ever made to a blocked host. Bandwidth is poor — a couple of hundred bytes per query — but an API key is 40 characters.

Check. Pull your resolver’s query log for the sandbox’s source IPs over a day. Group by registrable domain and sort by distinct-subdomain count. A normal workload resolves a small set of names repeatedly. Exfiltration looks like one domain with hundreds or thousands of unique labels under it, often long and high-entropy. This check is worth running even if you believe DNS is closed, because it is the cheapest of the five and it fails loudly.

Fix. Do not give the sandbox a route to arbitrary resolvers. Block outbound UDP and TCP on port 53 in the same default-drop policy, block DoH and DoT (443 to known resolver endpoints, 853 generally) and force name resolution through a resolver you run that answers only for names on the allowlist and refuses everything else. Once resolution is constrained to the allowlist, the covert channel closes with it, which is why the two controls belong in the same change rather than sequenced.

Symptom: the leak was small but the damage was not

What left the sandbox was one token. The consequence was access to production data the sandboxed task had no business touching.

Likely cause. Credentials reachable from inside the sandbox set the real blast radius, so scoping and short expiry matter more than the strength of the boundary around them. A long-lived, broadly scoped key inside a perfect sandbox is a worse position than a five-minute, single-resource token inside a mediocre one, because the boundary only has to fail once and the credential’s power is what determines the cost when it does. Boundaries are probabilistic; credential scope is deterministic.

Check. Enumerate everything credential-shaped that a process inside the sandbox can read: environment variables, mounted secret files, the instance metadata endpoint (169.254.169.254), any inherited cloud SDK config, the agent’s own provider key. For each one, answer two questions — what can it reach, and when does it expire? Any answer of “everything in the account” or “it doesn’t” is a finding. Do this by actually running env and a metadata curl inside a live sandbox rather than reading the deployment manifest; the manifest routinely misses what the base image or the SDK injects.

Fix. Scope each credential to the single resource the task class needs, and issue it per run with an expiry on the order of the task’s runtime, not the sprint. Block the metadata endpoint at the same egress policy that handles everything else — it is a plain HTTP destination and default-deny already covers it if you did not carve an exception. Where a task genuinely needs broad access, split it into a step that runs outside the sandbox with the credential and a step that runs inside without it. Bounding per-run lifetime also interacts with agent execution under provider rate limits and concurrency caps: tokens that expire mid-run turn into retry storms unless the retry path re-issues rather than reuses.

Symptom: the policy is configured and traffic still bypasses it

Egress rules exist, they were tested, and traffic is reaching hosts they forbid.

Likely cause. An egress policy enforced inside the sandboxed process can be disabled by the code running there; enforcement has to sit at the boundary the workload cannot reach. If the control is an HTTP client interceptor, a monkey-patched fetch, an HTTP_PROXY environment variable, or an allowlist consulted by a library the agent’s code imports, then the agent’s code can also unset it, import a different library, or open a raw socket. Code that can be edited by the thing it constrains is documentation, not enforcement.

Check. Write a deliberately hostile task and run it in a copy of your real sandbox: unset every proxy variable, then open a raw TCP socket to an IP you control on port 443 and send a byte. If your listener sees it, your enforcement is in-process. Do the same for DNS with a direct UDP query to an off-allowlist resolver. Both tests take minutes and give an unambiguous answer, which is the point — this is the failure mode most likely to be believed fixed when it is not.

Fix. Move enforcement to a layer the workload has no privilege over: network namespace firewall rules applied by the supervisor, a sidecar or gateway that is the only route off the host, or a syscall filter. Seccomp BPF is the kernel-level version of the same idea — a filter installed before the untrusted code starts, which the filtered process cannot loosen afterwards, since seccomp is one-way. Whichever you pick, the test above is the acceptance criterion, and it belongs in CI rather than in a runbook. If your pipeline already handles CI evaluation of non-deterministic AI systems, this is the rare agent-adjacent check that is fully deterministic and cheap to assert on every change.

The order to do these in

If you are starting from filesystem-and-process isolation with no network policy, the sequence that removes the most risk per hour is: default-deny egress with a host allowlist and DNS closed in the same change, enforced outside the process; then credential scoping and short expiry; then, only if the threat model genuinely includes kernel escape, the isolation tier upgrade. The first two are configuration. The third is a project.