We need a working, functional sandbox feature

[mindplay-dk]

I am not willing to risk my computer by running Pi without a sandbox extension.

Sadly, I've tried at least 10 different extensions - basically everything I can find, and nothing works. 🥲

Most of these are vibe coded, some are based on the example (which doesn't work either) and I've tried working with 3 different sandbox authors already to resolve the issues already - they can't do it, and I'm not smart enough to fix any of these or write one myself.

We need proper sandboxing - it is absolutely essential, and I won't use an agent that doesn't have this.

My use case is extremely basic, and this is how I prompt to test every sandbox:

run echo $PATH exactly once, even if it fails

Several extensions can't even do that.

Some of them can, but don't actually have the path.

run which npm exactly once, even if it fails

Most extensions (even if they have the PATH var) will find some outdated global install of APT npm.

Like most Node users, I use nvm, and there is maybe one or two sandbox extensions that can even find the binary.

The reason is most extensions don't even mount /home/rasmus.schultz/.nvm/versions/node/v/bin/ even if you configure them with a read permission.

run npm -v exactly once, even if it fails

There is no extension that gets this far - they all fail in various ways.

Of the two extensions that did at one point get this far, one no longer works (and I have no idea why) and the another stopped working because my environment vars are too long or something, I don't even know.

Most of these extensions look abandoned or unmaintained - a few use Anthropic's sandbox-runtime, but they don't work, and if they would have, the author isn't updating the package constraint, so you are stuck with an older version of the runtime. Several extensions maintain a fork (for reasons I don't understand) and these aren't getting updated either.

The bottom line is there is currently nothing that works, even for the most basic use case.

There are people out there trying their best to solve this, but let's be realistic, vibe coding isn't going to cut it for something like this - we need a carefully hand crafted solution created by someone who actually understands everything they're doing, which sadly isn't most of us.

I've really enjoyed Pi, while I had a working sandbox, but I haven't had that since May, and I am close to giving up.

I've considered various Docker-based tools as well, but this isn't strictly sandboxing - not in the sense where you can get an interactive confirmation prompt or check-in important settings with your project.

I really think we need a well-engineered, supported, maintained, first-party sandbox feature.

Please consider it. 🙏

[mroark1m]

FWIW, I ended up using nvidia openshell to wrap it and other agents too. It has better controls than most of the other sandboxing options I've seen and can even externalize API credentials from the sandbox (they can be injected at request time), so the agent does not have any credentials to lose. I also like the fact that it's not an extension or pi specific, because I can swap out the agent and get the same controls without having to reconfigure it according to whatever the new agent can do.

Not sure if you saw this one: https://github.com/earendil-works/gondolin it looks kind of similar to openshell but more minimal and its from the pi team. I didn't try it.

Although I'm using openshell successfully, I'll say it is still kind of an alpha version and it was not pain free. But they acknowledge that. I couldn't get microvm to work and had to fallback to docker for the compute, for instance.

[mindplay-dk]

Docker, Podman, Gondolin, Openshell - these are virtualization, which is not the same as sandboxing.

Granted, it's closely related - sandboxing is a form of virtualization, but what virtualization misses is interactive, blocking permission prompts and permissing persistence.

That's what I'm looking for. Deciding once for the project that an agent should or should not edit this or that file, or tun this or that tool, is different from being able to decide on a case-by-case basis, which is what I'm looking for.

Virtualization is probably fine for vibe coding, but I work with my agent, interactively, it doesn't just sit in the background and do stuff without my supervision or approval. 😊

[mroark1m]

I don't really agree on the terminology, docker/podman alone are quite different from gondolin and openshell, and the latter to me are firmly in the sandbox category, that's how they refer to themselves as well. But I get your point on interactive policy config vs up-front policy config, it is a different experience for managing the policies. Openshell can do interactive changes for the network policy but not on tools or file perms, because those are enforced by the container or vm itself, those have to be rebuilt to change those things. But you can do that and then resume the agent, its just a bit slower than doing it interactively.

There is still a difference between vibe coding (pure trust) and interactively working with the agent that has, IMHO not much to do with how you manage permissions, it's more about the style of interaction with the agent, and what you do with the agent output.

Personally I got frustrated by micro managing permissions and find it leads me to be lazy in the inspection of the permission requests over time, which I felt was way more likely to lead to a mistake than taking some time to harden the whole environment up front. You can make the same policy decisions in both cases.

[mindplay-dk]

There is still a difference between vibe coding (pure trust) and interactively working with the agent that has, IMHO not much to do with how you manage permissions, it's more about the style of interaction with the agent, and what you do with the agent output.

100%, this is what I'm trying to say 😊

Personally I got frustrated by micro managing permissions and find it leads me to be lazy in the inspection of the permission requests over time, which I felt was way more likely to lead to a mistake than taking some time to harden the whole environment up front. You can make the same policy decisions in both cases.

It's a balance. 🙂

Of course, I configure as many things as possible up front, so I don't have to answer a prompt every two seconds - with the extension I'm currently using (pi-landstrip) I get prompts for things like writing a log file to the .npm home folder, which I can safely save to my system level config, so I don't have to answer those again.

Other things are a on a case-by-case basis, for example, if my session is specifically to work in src I might block test from writes, because I don't want the agent to start changing any tests without confirming with me first.

pi-landstrip provides very useful prompts with options to allow once, allow for the session, allow for the project, or allow globally, so permissions build up very easily during the first sessions and persist over time.

So my daily experience is not that I'm micro managing - but that the agent stops and lets me make meaningful and important decisions when required, before burning oodles of tokens on doing the wrong thing. 🙂

[mindplay-dk]

(I also keep my ~/.pi/agents/sandbox.json checked into source control using ~ home folder aliases, so that permissions for things like writing to NPM log files carry over between the systems I work on. 🙂)

[Demonese]

I build custom Docker images (based on ubuntu) for each project, including only git, Node.js, pi, and the necessary development environment, then mount the project directory into the container for development.

[sn0rlaxlife]

This is my intention when running all agents minimal permissions in a container and a volume mount of just the project im working on git controlled

[bodograumann]

I am using the nono sandbox: https://github.com/nolabs-ai/nono
They have a built-in pi permissions profile, a pi extension and tons of features.
Of course you still need to decide what the agent can access by yourself.

[Pxtl]

Pi is built specifically for the YOLO approach, where you give it a computer (or sandbox, or VM, or whatever) to use and abuse as it sees fit. While the sandbox you run it upon can provide complex permissions, mr Zechner has said that interactively requesting permissions from within the agent is a non-goal.