Skip to content

feat(vmm): add optional libvirt network filtering - #837

Open
kvinwang wants to merge 8 commits into
masterfrom
feat/libvirt-netfilter
Open

feat(vmm): add optional libvirt network filtering#837
kvinwang wants to merge 8 commits into
masterfrom
feat/libvirt-netfilter

Conversation

@kvinwang

Copy link
Copy Markdown
Collaborator

Summary

  • add an optional cvm.network_filter.mode = "libvirt" backend for bridge NICs; the default none mode preserves the existing unfiltered QEMU bridge-helper path
  • keep QEMU launch and every attestation-sensitive QEMU argument under dstack-vmm; libvirt is used only for standalone nwfilter bindings
  • add a small privileged dstack-vmm netd broker that creates deterministic TAPs, attaches them to the requested bridge, and creates/removes libvirt nwfilter bindings before/after QEMU
  • support one shared host netd across multiple VMM instances through a stable instance namespace, while allowing dedicated sockets for development
  • authenticate Unix-socket clients with SO_PEERCRED, bound protocol messages and fields, derive TAP names server-side, generate/escape binding XML internally, use fixed absolute host tools, and serialize operations across netd processes
  • roll back all prepared NICs if setup or Supervisor deployment fails; remove bindings and TAPs after VM stop/removal
  • let netd load only [netd], so it can use a small standalone root-owned configuration and does not initialize QEMU/VMM services
  • support running both VMM and netd directly without systemd through --socket, --allow-uid, and --netd-socket

This implementation starts from master and does not depend on or reuse the implementation in #835/#836. Those PRs remain available only as historical reference.

Configuration

[cvm]
instance_id = "worker-a" # optional; derived from run_path when empty

[cvm.network_filter]
mode = "libvirt" # or "none" (default)
filter = "clean-traffic"
parameters = { IP = "192.0.2.10" }

[netd]
socket = "/run/dstack/netd.sock"
allowed_uids = [991, 992]
libvirt_uri = "qemu:///system"

When mode is none, dstack does not connect to netd and continues to emit:

-netdev bridge,id=netN,br=<bridge>

When mode is libvirt, netd finishes the binding before dstack submits QEMU, which receives:

-netdev tap,id=netN,ifname=<derived>,script=no,downscript=no,vhost=off

Verification

  • cargo test -p dstack-vmm — 60 passed
  • cargo clippy -p dstack-vmm --all-targets -- -D warnings
  • cargo fmt --manifest-path vmm/Cargo.toml -- --check
  • git diff --check origin/master...HEAD
  • root integration test against the installed qemu:///system libvirt:
    • create temporary bridge
    • start the built dstack-vmm netd
    • prepare TAP with clean-traffic and static IP parameter
    • verify virsh nwfilter-binding-dumpxml
    • remove binding/TAP and verify both disappeared
  • direct development-mode test with a minimal [netd] config and a non-root client authorized by UID

Copilot AI review requested due to automatic review settings July 30, 2026 06:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants