How would we know if a containment assumption stopped being true at runtime?

[ljefford2-cmyk]

A while back I invited people here to test v0.0.87 (pinning OpenShell v0.0.71) against a set of operating rules I’d written. Several of my mappings were wrong and got corrected, which was the useful outcome. This is a follow-on question in the same spirit.

Something I keep running into across AI generally, not specific to any project or lab: when an agent system fails, the failure almost always gets attributed to the model. Hallucination, misalignment, reward hacking, deception — the whole failure vocabulary names properties of the model. There is very little working language for “the harness was wrong,” “the authority grant was wrong,” “the observation path was wrong,” or “a boundary assumption was never written down.”

I think that is because the model is often the only component anyone has explicitly named, and a failure can only be attributed to a component that exists in your model of the system. So the repair menu ends up short: more training, better alignment, better prompts.

Which is why I am asking here. Sandbox projects are one of the few places where the other components do have names — isolation mechanisms, policy enforcement, egress control, lifecycle management. You are already doing the decomposition most of the field has not.

The Snyk write-up on bypassing the OpenClaw sandbox is what prompted the question, and not as criticism — bypasses are what eventually happens to isolation. The point is that if isolation can be bypassed, isolation alone cannot be the claim. Something has to survive the bypass, and the only candidate is evidence.

So, genuinely asking:

• Where is the containment claim stated, and what does it assume about things outside the sandbox?
• Which of those assumptions are discharged at design time, and which have to stay true while a run is in progress?
• For the runtime ones — is there an observation path for policy decisions and boundary crossings that the sandboxed agent cannot suppress, alter, or delay?
• If a capable agent partly defeated isolation during a run, what would the project be able to say about that run afterward? Is there a narrower claim it falls back to?

I have been writing this up more formally and would be glad to post it here if it is useful. I would rather understand how the project already handles this than assume there is a gap.

[farfromexact]

I would model containment as a set of explicit claims with an assumption register, rather than as one binary property of the sandbox.

For each run, separate:

• design-time guarantees, such as which policy component owns egress decisions;
• runtime assumptions, such as the host supervisor remaining alive and the observation channel remaining outside the agent's control;
• residual claims, such as what can still be reconstructed if isolation is bypassed.

The runtime observation path should be independent of the sandboxed process and should record at least policy decisions, boundary crossings, process lifecycle, network destinations, denied operations, and monitor health. An append-only or tamper-evident event stream with sequence numbers would make gaps visible. The agent should not be able to rewrite, delay, or selectively disable that stream.

Monitor failure must have an explicit meaning. If the supervisor or audit path disappears, the system should mark the run as containment-unknown and fail closed where possible, rather than continuing to report a successful containment result.

If an agent partly defeats isolation, the honest fallback claim is not �the run was safe.� It might be: �the control plane observed these requests and recorded no approved egress beyond the declared policy,� or, if the observation path was also compromised, �the run cannot be evaluated.� That distinction is important because evidence of observed events is not proof that no unobserved event occurred.

A useful design artifact would be a table mapping every containment assumption to its owner, detection signal, failure behavior, and post-run evidence. It would make �the harness was wrong� a first-class diagnosis instead of forcing every failure into a model-behavior category.

[ljefford2-cmyk]

After finishing the work that led me to ask this question, I realized I started in the middle instead of at the beginning.

My original post asked how we would know if a containment assumption stopped being true during a run. That is still an important question, but it assumes we have already defined what is actually being contained.

I think there is an earlier engineering question.

You cannot monitor something you have not defined.

A runtime monitor can only compare reality against assumptions that have already been identified. If the system boundary, authority boundary, observation path, or assumptions of use were never explicitly defined, then there is nothing objective for the monitor to determine has changed.

That is what I was really trying to get at.

One thing I appreciate about projects like NemoClaw is that they have already decomposed the problem into real engineering components instead of treating “the AI” as one indivisible thing. You have isolation mechanisms, policy enforcement, lifecycle management, egress controls, supervisors, and other pieces that can actually be reasoned about. That is much farther than much of the industry has gone.

What I am wondering is whether the next step is to define the evaluated item before asking whether containment succeeded.

For example:

• What exactly is the item for which NemoClaw is making a containment claim?
• Which components are inside that evaluated item?
• Which components remain outside it but are relied upon by assumption?
• Which assumptions are established at design time, and which must remain true throughout execution?
• If one of those runtime assumptions fails, does the engineering claim automatically narrow?

I am not asking these questions because I think NemoClaw has a flaw. Quite the opposite.

It seems to me that sandbox and containment projects are being asked to solve a problem before the field has agreed on how containment claims themselves should be defined and evaluated. The implementation work is moving faster than the engineering vocabulary.

The recent OpenAI and Anthropic incidents reinforced that thought for me. In both cases, the interesting engineering question was not simply that a boundary was crossed. It was whether the operational system being evaluated had ever been completely defined in the first place.

I am interested in understanding how the NemoClaw team thinks about that question because you are much closer to the implementation realities than I am. If I am missing something fundamental, I would much rather learn it here than continue building on a bad assumption.

[ljefford2-cmyk]

Working through this further, I think I finally understand the distinction I had been struggling to articulate.

There seem to be two separate engineering problems.

The first is defining the operating system.

The second is governing the operating system.

The discussion here has been about containment, isolation, policy enforcement, lifecycle management, observation, and runtime control. Those are all governance mechanisms. They govern an operating system that already has defined components, boundaries, assumptions, and responsibilities.

What I had not appreciated at first is that those mechanisms cannot define the system they are governing.

Before asking whether containment succeeded, or whether a runtime assumption failed, there is an earlier question:

What exactly is the operating system we are talking about?

What is the evaluated item?

Which components belong inside it?

Which components remain outside it but are relied upon by assumption?

Which assumptions are design-time properties, and which must remain true throughout execution?

What engineering claim are those assumptions intended to support?

Without those answers, a monitor has nothing objective to compare reality against. A controller cannot preserve assumptions that were never explicitly stated. A sandbox can faithfully enforce the boundaries it has been given, but it cannot determine whether those boundaries completely describe the operation they are meant to protect.

That realization also helped me understand why the earlier discussion about “responsible layers” was so useful.

A sandbox can legitimately own structural containment. An application can own authority. An evaluation harness can own the measurand. A governance layer can own claim licensing and oversight.

Those responsibilities can be shared or delegated.

What seems dangerous is not delegation. It is leaving one of those responsibilities undefined because everyone assumes it belongs somewhere else.

At least to me, that seems to be the larger engineering question AI systems are beginning to expose. We have become much better at building controls than we are at explicitly defining the systems those controls are intended to govern.

I’m curious whether that distinction resonates with the maintainers, or whether I’m still missing an important piece of how you think about the architecture.