Any plans for further hardening the sandbox with firewall rules?

[DataLabTechTV]

Other projects, like 89luca89/clampdown or mattolson/agent-sandbox have been adding iptables with sane default rules, using a sidecar container as a means of extra control.

Any plans to add something similar to ramalama sandbox?

[rhatdan]

We would be interested in your ideas on this.

I would want to make it as simple as possible, so a user does not need to be a Linux networking expert to set it up. Simplicity / Security should be the goal. We are looking for goldilocks level of security.

[DataLabTechTV]

Well, thanks! Allow me to elaborate then, as I was just working on this today.

As I was trying to find an approach that works for me, I realized that the easiest way would be to control network access at the podman level instead of the container level (a sidecar).

So far, I've tried creating an internal network in podman and passing that to RamaLama, and that completely dropped packages to the outside world, as expected.

Then I spent a little while trying to figure out if it's possible to apply firewall rules to the custom subnet I created for the podman network. So far no success on this front, but I'm positive this will beat the complex approaches everybody else is trying to push, with sidecars and all that. I was using firewalld for this, but I'm rusty (last time I wrote firewall rules was around 2000 using ipf and pf on FreeBSD and OpenBSD).

Assuming this is viable, I believe the best approach for RamaLama would be to simply provide an easy way to set this up out-of-the-box, and even offer a few rulesets for different use cases or security levels.

I'll take another look in a few days again, but I'll have to read a bit more about podman networking in the meantime.

[rhatdan]

@Luap99 @mheon Any chance you could help?

[Luap99]

I have no idea what the requirements are, AFAIK most of these tools use user space proxies to provide easier control and runtime decisions.

With firewall rules you either block or don't. Blocking outbound/inbound can while allowing communication between containers can be done as a internal network.

Podman 6 will add a new blackhole route option so you could block a specific subnet via a custom route in the container which is definitely simpler than firewall rules. podman-container-tools/podman#28230 (but anything that blocks via routes or firewall rules inside the container netns requires that you do not give it CAP_NET_ADMIN as they could just unset it then)

Everything else is pretty much write your own rules yourself. As root that is just adding the firewall rules on the host, as rootless it can be a bit more tricky, for custom networks it is podman unshare --rootless-netns or some oci hooks to also inject rules into the container namespace.

[DataLabTechTV]

First let me just be upfront and say that a lot of these security details escape me. I'm currently trying to get up to date on a lot of this stuff. Either way, here are my two cents.

Firewall rules inside the agent container won't work, I agree. The agent would just drop them as needed. I think this is why most projects trying to implement this use a sidecar instead. This way the agent won't be able to modify firewall rules. I assume the sidecar serves as a proxy to the agent, and all requests are served through there (so at least two containers).

I would, however, prefer to keep the architecture simpler. This is why I started looking into host-level firewall rules that would be applied to specific podman networks set with static subset ranges. I assumed the host would be able to filter these subnets, but I think I was wrong. Soon I'll try to learn a bit more about this.

The only real requirement here would be to secure the sandbox network wise. Let's say we're working on private codebase that we don't want to get leaked, and someone poisons a model to make it upload those files somewhere. I would prefer that the agent only communicates with specific hosts as defined by some kind of ruleset (it doesn't need to be simpler than a firewall; in fact I think it should use existing tooling like firewalld, which is already a nice simplification layer).

For now, let's assume there's nothing to do here yet. It needs a bit more research (mostly on my end). Blunt question: is it doable to use firewalld over a podman network, when running as rootless, or should I just drop it and try a different approach? And does it make sense to secure the agent with a firewall, or is there simpler alternative that I'm not seeing here?

Thanks!

[DataLabTechTV]

I think some of the confusion comes from this:

No Network Access – ramalama run is executed with --network=none, meaning the model has no outbound connectivity for which information can be leaked.

I would assume that, if I run the following command, I would not be able to connect to the outside world at all:

ramalama sandbox opencode -c 51200 --ngl 10 hf://unsloth/Qwen3.5-9B-GGUF:Q4_K_M

But when OpenCode opens and I prompt it to ping 1.1.1.1, it can ping.

On the other hand, it will only drop connections without a problem when I create an internal podman network that I pass as an argument afterward:

podman network create --internal --subnet 10.4.1.0/24 ramalama
ramalama sandbox opencode -c 51200 --ngl 10 hf://unsloth/Qwen3.5-9B-GGUF:Q4_K_M --network ramalama

Maybe the model itself is isolated properly, but isolating the agent is the critical part. This is why I was looking into podman networking and firewall rules for this.

[musaabhasan]

A practical "goldilocks" model could be profiles rather than raw firewall controls.

For example:

• offline: no network at all; good default for local model inference and untrusted prompts
• egress-allowlist: outbound only to configured hosts, ports, or registries
• loopback-only: allow local service composition but deny internet egress
• model-pull: temporary egress only for pulling model artifacts, then back to offline
• dev: broader access, but visibly marked as less restrictive

The UX matters because most users will not reason correctly about iptables/nftables. The command should expose intent-level options, while the implementation can choose network namespace, podman flags, nftables, or a sidecar proxy underneath.

I would also log the effective network policy at container start. When someone later asks whether a model/tool could have exfiltrated data, the answer should be recoverable from the run metadata, not from memory of which flags were used.

[rhatdan]

I like where you are going with this, Keeping it as simple as possible and then allowing users to build their own networks for the more complex ones.

Interested in opening a PR?

[olliewalsh]

Now that the OpenShell podman/docker compute drivers are available I think it would make more sense to delegate the sandboxing to it instead of trying to implement similar functionality - or perhaps https://github.com/always-further/nono if we want something very lightweight.

[rhatdan]

If ramalama is going not going to do anything with confinement of MCPs, then it would be nice if ramalama and openshell were further integrated in some way, so that RamaLama could either pull and launch openshell or at least document it.

[rhatdan]

sudo dnf -y install openshell
Updating and loading repositories:
fedora-44-x86_64/repodata/repomd.xml
Repositories loaded.
Failed to resolve the transaction:
No match for argument: openshell
You can try to add to command line:
  --skip-unavailable to skip unavailable packages

Not quite packaged in Fedora yet.

[olliewalsh]

Not quite packaged in Fedora yet.

Yes, for now can use maxamillion's copr:

$ sudo dnf copr enable maxamillion/nvidia-openshell
$ sudo dnf install openshell-gateway openshell

# start the rootless user systemd service unit
$ systemctl --user start openshell-gateway.service

# add the local openshell gateway to your client config
$ openshell gateway add https://localhost:17670 --local