Egress-side network policy -- where the FORWARD hook is installed, what it matches, when it fails open, and how the proxy host is pinned. Mostly LXC, but #755 covers Bubblewrap too and #766 is shared across backends. This consolidates 11 separately filed issues so the backlog carries one entry per area instead of one per finding. Every original report is reproduced below in full, unedited. The originals are closed and point here. | Folded from | Status | Title | | --- | --- | --- | | #755 | Open | [LXC/Bubblewrap] Firewall mode fails open when no veth interface is set (no FORWARD hook) | | #764 | Open | [LXC] Container runs unfiltered between start and firewall hook installation | | #766 | Open | [Network] Extract shared CIDR and address-family handling used by multiple backends | | #777 | Open | LXC network policy fails on RHEL 10: deprecated iptables `-m state` match unsupported | | #851 | Fixed in #849, closes on merge | [LXC] Firewall-enforced start accepts a container whose only interface is lxc.net.N for N != 0 | | #858 | Fixed in #849, closes on merge | LXC state-aware firewall mode does not verify that lxc.net.0 is a veth | | #869 | Open | [LXC] Scope the FORWARD return path by container address so a DROP-policy host still works | | #870 | Open | [LXC] deny-all-except-proxy does not cover the INPUT path to the bridge gateway | | #875 | Open | [LXC] Rewrite the /etc/hosts proxy pin from Rust with an open-once, no-follow handle | | #877 | Open | LXC firewall mode silently drops unresolvable blockedHosts entries and still reports the policy enforced | | #878 | Open | LXC veth discovery reads a failed probe as no interface, so firewall setup can succeed with no FORWARD hook | Rows marked *Fixed in #849* are already addressed by that pull request and are reproduced here only so the record stays complete. Read them as history once #849 merges, not as work remaining. --- ## [LXC/Bubblewrap] Firewall mode fails open when no veth interface is set (no FORWARD hook) *Originally #755.* ### Relevant area(s) Linux (LXC and Bubblewrap backends) ### Brief description of your issue In firewall enforcement mode (`network.enforcementMode: "firewall"` / `"both"`), `NetworkIptablesManager` creates the per-container `iptables`/`ip6tables` chain(s) with a default policy, but when **no veth interface is set** it skips the `FORWARD` hook and still returns success. The chain is therefore never in the packet path, so **no traffic is filtered** even though the caller believes the network policy was applied. - The **Bubblewrap** backend never calls `set_veth_interface` (`src/backends/bubblewrap/common/src/bwrap_runner.rs:205-206`), so **every** Bubblewrap firewall-mode run installs an orphan chain and enforces nothing — IPv4 and IPv6 alike. - The **LXC** backend discovers the veth (`src/backends/lxc/common/src/lxc_runner.rs:208-221`) but only sets it `if let Some(...)`, so it hits the same gap whenever `discover_veth_interface` fails. This is a **silent fail-open on a security control**. It is **pre-existing on `main`** (not introduced by any single PR); it was identified during review of #724, which corrects the LXC `FORWARD` direction (`-o` → `-i`) and adds LXC hook assertions but does not close this gap and does not cover Bubblewrap. Relevant code: `src/backends/lxc/common/src/network_iptables.rs:811-818` (the `else` branch logs `"No veth interface set for container. Cannot scope iptables rules. Skipping FORWARD hook."` and returns `Ok(())`), surfacing as `apply_firewall_rules` returning `Ok(true)` with `rules_applied = true` at `network_iptables.rs:670-673`. ### Steps to reproduce 1. Run a **Bubblewrap** container (or an **LXC** container where veth discovery fails) with: - `network.enforcementMode: "firewall"` - `network.defaultPolicy: "block"` - a non-empty `allowedHosts` / `blockedHosts` list. 2. Observe the log line: `Warning: No veth interface set for container. Cannot scope iptables rules. Skipping FORWARD hook.` 3. From inside the container, connect to a destination that the policy should block (e.g. any host not in `allowedHosts` under `defaultPolicy: "block"`). ### Expected behavior When network enforcement is requested but cannot be safely scoped to the container (no veth interface), setup should **fail closed** — return an error so the container does not run with an unenforced network policy. For Bubblewrap specifically, the backend should either implement process/cgroup-scoped enforcement (e.g. `OUTPUT` with cgroup/UID matching) or explicitly reject firewall mode rather than reporting success. ### Actual behavior `apply_firewall_rules` returns `Ok(true)` and sets `rules_applied = true`. The chain exists but is never hooked into `FORWARD`, so all container egress (IPv4 and IPv6) flows **unfiltered** while the run reports success. For Bubblewrap this occurs on every firewall-mode run because `set_veth_interface` is never called; for LXC it occurs whenever veth discovery fails. --- ## [LXC] Container runs unfiltered between start and firewall hook installation *Originally #764.* Split out of review feedback on #632. ## Problem The LXC runner starts the container before it installs any firewall hook, so there is a window in which the container has a live interface and no policy on it. In `src/backends/lxc/common/src/lxc_runner.rs` on the #632 branch: - `container.start()` at line 330 - optional `wait_for_network`, up to 10 seconds, at line 358 - veth discovery at lines 361-376 - `apply_firewall_rules` at line 408 Anything already inside the container -- init services, or processes retained across runs by `destroyOnExit: false` -- can send arbitrary egress for the whole span between line 330 and line 408. Under the deny-all-except-proxy posture that window is a hole in the guarantee the mode is named for. #632 moved proxy pinning to before `container.start()`, which removed host-side DNS work from the window, but it did not close the window itself. ## Why it is not fixed in #632 Closing it needs a second, earlier chain: a veth-scoped quarantine carrying only the DHCP carve-outs, installed as soon as the interface exists, then atomically replaced by the completed policy. That is a new enforcement stage with its own rollback, ownership, and teardown semantics rather than a reordering of the existing one, so it does not belong in a PR that is already addressing a different set of gaps. ## Suggested shape 1. Discover the veth as early as the interface exists rather than after the address wait. 2. Install a quarantine chain that drops everything except DHCPv4 and DHCPv6. 3. Let `wait_for_network` proceed under quarantine. 4. Replace the quarantine with the completed policy, and make the swap ordering such that no instant leaves the interface unhooked. ## Not verified None of this has been exercised against live iptables or a live LXC host. The window is established by reading the call ordering in `lxc_runner.rs`, not by observing a packet escape. --- ## [Network] Extract shared CIDR and address-family handling used by multiple backends *Originally #766.* Split out of review feedback on #724. ## Problem Address and CIDR handling for network policy is implemented per backend rather than once. The LXC path in `network_iptables.rs` parses CIDR prefixes, classifies IPv4 versus IPv6, and buckets resolved addresses by family; equivalent logic exists elsewhere for other backends. Duplicated parsing of security-relevant input is how two backends end up disagreeing about what a policy means, and a fix applied to one silently leaves the other wrong. ## Suggested shape Extract a single module owning: 1. CIDR parsing and validation, including prefix-length bounds per family. 2. IPv4 versus IPv6 classification, including IPv4-mapped IPv6 forms. 3. Bucketing a resolver result into per-family destination lists. Then have each backend depend on it rather than reimplementing it. ## Why it was not done in #724 #724 is a behavior fix for IPv6 destinations and CIDR ranges in firewall mode. Extracting a shared module touches every backend that does address handling and would turn a scoped fix into a cross-backend refactor, which is a worse change to review and a worse change to revert. ## Not verified This describes duplication observed by reading the backends, not a divergence observed at runtime. Whether the implementations currently disagree on any specific input has not been established -- that is worth checking as the first step of the extraction, since any disagreement found is a live bug rather than just duplication. --- ## LXC network policy fails on RHEL 10: deprecated iptables `-m state` match unsupported *Originally #777.* ### Relevant area(s) Linux ### Brief description of your issue The LXC backend builds its firewall rules with the deprecated `-m state` iptables match. On RHEL 10 (kernel 6.12, iptables 1.8.11 nf_tables backend) the kernel does not expose `state` revision 0, so applying any network policy fails outright. `-m state` (`xt_state`) has been superseded by `-m conntrack --ctstate` since ~2011 and was folded into `xt_conntrack`. Ubuntu and Debian still ship the compatibility shim, which is why this only surfaces on RHEL. The failure is fail-closed — `lxc_runner` destroys the container and returns a `backend_error` rather than running unprotected — so there is no silent policy bypass. However, it means **MXC cannot enforce LXC network policy at all on RHEL 10**; any run with a `network` section fails. Source: `src/backends/lxc/common/src/network_iptables.rs` (the `ESTABLISHED,RELATED` rule in `apply_firewall_rules`). The same deprecated syntax also appears in `src/backends/wslc/common/src/policy_mapping.rs`, which will hit this whenever its distro drops the shim. ### Steps to reproduce 1. On a RHEL 10 x64 host, install LXC (`lxc`, `lxc-templates`, `dnsmasq`, `iptables`) and build/obtain `lxc-exec`. 2. Run a config with a network policy using firewall enforcement, e.g. `tests/configs/lxc_network_test.json`: ``` ./lxc-exec tests/configs/lxc_network_test.json ``` 3. Observe the run fails before the script executes. ### Expected behavior The firewall chain is created and the network policy is enforced, matching the behavior on Ubuntu and Debian. The container runs and host filtering is applied. ### Actual behavior Rule application fails and the run aborts: ``` Network policy error: iptables -A MXC-CLI-LXC-Network-Test -m state --state ESTABLISHED,RELATED -j ACCEPT failed: Warning: Extension state revision 0 not supported, missing kernel module? iptables v1.8.11 (nf_tables): RULE_APPEND failed (No such file or directory): rule in chain MXC-CLI-LXC-Network-Test {"error":{"code":"backend_error","message":"Network policy error: ..."}} ``` Observed in CI on the `1es-mxc-e2e-rhel-10-x64` pool (image `rhel-10-x64`, version `10.2.2026080415`, kernel `6.12.0-211.40.1.el10_2.x86_64`). The other seven LXC tests pass on the same host, so LXC itself is functional — only the firewall rule syntax fails. Suggested fix: emit `-m conntrack --ctstate ESTABLISHED,RELATED` instead, which is the modern form supported across all currently supported distributions. --- ## [LXC] Firewall-enforced start accepts a container whose only interface is lxc.net.N for N != 0 *Originally #851. Fixed in #849; closes on merge.* Found by Copilot review on #849: https://github.com/microsoft/mxc/pull/849#discussion_r3780028309 ## Problem `LxcStateAwareBackend::start` refuses a container it cannot enforce a firewall policy on, but the refusal has a gap. It rejects `lxc.include`, rejects more than one interface, and rejects zero interfaces — then accepts any remaining case, which includes a container whose single interface is at a nonzero index: ```rust if net.indices.len() > 1 { /* refuse */ } if net.indices.is_empty() { /* refuse */ } // net.indices == [1] falls through and is accepted ``` `src/backends/lxc/common/src/state_aware.rs:311` Everything downstream assumes index `0`. The code pins `lxc.net.0.veth.pair` and hooks that deterministic veth, so a container configured with only `lxc.net.1` gets a chain installed against an interface it does not use. Start reports success and the caller believes a default-deny policy is in force while traffic routes around it. ## Why this matters This is a fail-open hole in the exact invariant the LXC state-aware work exists to establish: a container must never run believing it is filtered when no rule filters it. Every other branch in this function fails closed for that reason. ## Suggested fix Require the sole index to be `0` before installing the firewall, and refuse with the same `policy_validation` shape the neighboring branches use. Add a parser/spec test for the `lxc.net.1` case alongside the existing single-interface tests. ## Notes Deferred out of #849 to keep that PR reviewable — #633 was closed after accumulating 87 review comments, and this is a behavior change to fail-closed start validation that deserves its own review rather than a late amendment. --- ## LXC state-aware firewall mode does not verify that lxc.net.0 is a veth *Originally #858. Fixed in #849; closes on merge.* Found while reviewing #849. Distinct from #851, which covers the nonzero-index case. ## What happens `configured_net_interfaces()` returns only interface indices and an include flag -- it never reads the interface type: - `src/backends/lxc/common/src/lxc_bindings.rs:133-138` -- `pub struct NetInterfaceConfig { pub indices: Vec<u32>, pub has_include: bool }` - `src/backends/lxc/common/src/lxc_bindings.rs:154-185` -- `parse_net_interface_config` matches `lxc.net.<N>.` keys and `lxc.include` only, never a `.type` value. So the firewall path cannot check the type, and pins net.0 regardless: - `state_aware.rs:301` refuses `has_include` - `state_aware.rs:311` refuses more than one interface - `state_aware.rs:330` refuses zero interfaces - `state_aware.rs:346` `set_config_item("lxc.net.0.veth.pair", &veth)` An adopted container with a single `lxc.net.0.type = macvlan` or `phys` at index 0 passes every check, then gets a FORWARD hook pinned to a veth name that will never exist, while traffic uses the non-veth interface. MXC reports the firewall as enforced. ## Why it is not #851 #851 is `indices == [1]` -- a nonzero index. A single macvlan at index **0** gives `indices == [0]`, which never triggers #851. Same underlying assumption that net.0 is a veth, but a different trigger and a different fix. ## Suggested direction Capture the interface type in `NetInterfaceConfig`, and fail closed in `apply_network_policy` unless index 0 is a `veth`. Note: the code facts above are verified. That a macvlan/phys interface then routes around the FORWARD hook follows from `veth.pair` applying only to `type=veth`; that part is reasoned from the LXC config schema, not executed. --- ## [LXC] Scope the FORWARD return path by container address so a DROP-policy host still works *Originally #869.* Filed from review feedback on #798 (thread on `src/backends/lxc/common/src/network_iptables.rs:669`). ## What the reviewer found > These return rules do not match replies on the default bridged LXC topology, as the comment > itself notes: `FORWARD` sees `-o lxcbr0` before a bridge port is selected. Consequently, on > hosts whose FORWARD policy is DROP (notably hosts running Docker), allowed destinations and an > off-host proxy time out. The workflow masks this by changing FORWARD to ACCEPT. The > implementation needs a return rule scoped by the container address (or equivalent) so the proxy > exception works under a normal DROP host policy. ## This is accurate, and #798 documents it Both return-rule forms were measured inert on the default bridged topology: - The interface form matches `-o <veth>`, but a reply routed toward the bridge has an output device of `lxcbr0`, not the veth. - The physdev form matches `--physdev-out <veth>`, which requires that a bridge port has already been selected. On a reply routed in from outside, no port is selected while `FORWARD` runs. `build_forward_return_physdev_rule_args` says so in its doc comment, and `.github/workflows/lxc-e2e.yml` says so where it sets the host policy, naming the address-scoped return rule as the fix and calling it deferred. ## Impact On a host whose FORWARD policy is DROP — which includes any host running Docker — replies to the container are dropped by the host policy. The deny half of the posture still holds, but the one permitted exception does not: the proxy times out, so the container reaches nothing at all. The E2E suite does not catch this because the workflow sets FORWARD to ACCEPT first, and it does that for a real reason: under a DROP policy a container with no MXC hook at all is equally unreachable, so every deny case would pass vacuously and the suite would report success against a firewall that filters nothing. ## The fix Scope the return direction by the container's address rather than by its port — `-d <container-ip>` with `--state ESTABLISHED,RELATED` — so the rule matches while `FORWARD` still sees `-o lxcbr0`. That needs the container address plumbed through to the manager, which does not have it today, and a live bridged measurement to confirm the match. Adding a third rule form without that measurement would repeat the mistake the two current forms already made. ## Why it was not done in #798 It needs a new input to the manager and a real bridged host to verify against, and #798 is already 31 files. The limitation is documented where it bites rather than left for someone to rediscover. --- ## [LXC] deny-all-except-proxy does not cover the INPUT path to the bridge gateway *Originally #870.* Filed from review feedback on #798 (thread on `tests/scripts/run_lxc_network_proxy_test.sh:211`). ## What the reviewer found > This leaves a direct egress path open in the posture advertised as "deny all except the proxy." > Traffic to any host-local listener on the bridge gateway bypasses `FORWARD`, not only dnsmasq; > the reachable resolver can also be used as a DNS exfiltration channel. The proxy policy needs a > per-container `INPUT` hook (with only the configured proxy endpoint/port allowed) before this > test can treat gateway DNS as informational rather than an isolation failure. ## This is accurate The chain #798 installs is hooked into `FORWARD` only. A packet from the container to the bridge gateway's own address is delivered locally and traverses `INPUT`, so nothing in the chain ever sees it. `run_lxc_network_proxy_test.sh` reports `GATEWAY_DNS_REACHED` as a note rather than a failure for exactly this reason. The reviewer's point is broader than that note admits. The gap is not that gateway DNS is still answered. It is that **every** host-local listener bound on the bridge gateway is reachable from a container whose policy says it may reach the proxy and nothing else, and that a reachable resolver is an exfiltration channel in its own right — a name lookup carries attacker-chosen bytes out even when no other egress exists. ## The fix Hook `INPUT` per container the way `FORWARD` is already hooked, scoped with `--physdev-in` and `-i` on the container veth, allowing only the configured proxy endpoint and port and dropping the rest. Once that lands, `run_lxc_network_proxy_test.sh` should assert `GATEWAY_DNS_BLOCKED` instead of reporting the verdict, and the note explaining why it only reports should be deleted with it. ## Why it was not done in #798 It is a second enforcement surface with its own ownership, rollback, and teardown, and its own failure modes — an `INPUT` chain left behind is a host-wide artifact rather than a container-scoped one. #798 hooks one chain into one place and proves that hook works; adding a second hook belongs with its own tests. --- ## [LXC] Rewrite the /etc/hosts proxy pin from Rust with an open-once, no-follow handle *Originally #875.* ## What Rewrite the container's `/etc/hosts` proxy pin from Rust with an open-once, no-follow file handle instead of the current `grep` + `printf` shell command that `lxc_runner` sends through `attach_run`. ## Why The shell command is in `LxcScriptRunner::build_hosts_pin_command` and `build_hosts_unpin_command`, both of which share `LxcScriptRunner::hosts_read_prologue` (`src/backends/lxc/common/src/lxc_runner.rs`). It reads the file into a shell variable, then truncates the file and writes the variable back. Three known gaps follow from that shape, and the doc comment on `hosts_read_prologue` already records the last two as unclosed: 1. **The whole file is buffered in shell state.** A workload that ran earlier in a reused container can enlarge `/etc/hosts`, and the command then does unbounded command-substitution work and holds the entire result in memory before the rewrite. Raised in review on PR #798. 2. **A successful read still loses NUL bytes**, because a shell variable cannot hold one. A hosts file containing a NUL is rewritten truncated at that byte and still exits 0, so nothing observes the loss. 3. **A symlink swapped in between the `-h` test and the `>` redirect is still followed.** The `-h` test covers the unraced shape only. A dangling symlink is the worst case: it fails `-e` as well, and a redirect onto a dangling link creates the link's target, which on a writable host bind mount lands outside the container. All three come from the same constraint -- the command has to work on BusyBox, so it is restricted to `grep` and `printf`, and neither can open a file once and rewrite it in place. ## Proposed fix Do the rewrite from Rust against `/proc/<init_pid>/root/etc/hosts`, which is the container's own mount namespace as seen from the host, so no helper binary has to exist inside the container. Open with `O_NOFOLLOW`, stream the filter rather than buffering the file, and write through the same descriptor. Two things to get right, and neither is free: - `O_NOFOLLOW` only protects the final component. A hostile `/etc` inside the container still redirects the open, so this wants `openat2` with `RESOLVE_NO_SYMLINKS`, or a component-by-component `openat` walk, rather than a single `open` on the joined path. - The current command runs inside the container's namespaces via `attach_run`. Reaching the file through `/proc/<pid>/root` from the host instead changes which credentials the write happens under, so the permission model needs to be re-checked rather than assumed equivalent. ## Not urgent The pin is only written when a proxy is configured, `/etc/hosts` is normally a few hundred bytes, and gaps 2 and 3 both need a workload that is already manipulating its own `/etc/hosts`. This is correctness and robustness debt, not a live break. Raised by @MGudgin in review on #798: https://github.com/microsoft/mxc/pull/798#discussion_r3777621316 --- ## LXC firewall mode silently drops unresolvable blockedHosts entries and still reports the policy enforced `src/backends/lxc/common/src/network_iptables.rs` skips a `blockedHosts` entry that does not resolve, and the start still reports the policy enforced. --- ## LXC veth discovery reads a failed probe as no interface, so firewall setup can succeed with no FORWARD hook `src/backends/lxc/common/src/network_iptables.rs` reads a failed veth probe as no interface, so the FORWARD hook is never installed. ---