Skip to content

Incus managed networking compatibility gaps observed on DStack 0.5.9 #1032

Description

@kingsleydon

Summary

We run Incus system containers inside a DStack CVM. On the released DStack
0.5.9 image (6.9.0-dstack), standard Incus managed-bridge paths encounter
several missing netfilter capabilities.

This is a released-image compatibility report, not a claim that ordinary
DStack Compose applications are broken. We reviewed current next at
312dc605ebaaf93372534ffb135ea011533d28e9, but have not boot-tested that
unreleased source and do not claim it reproduces these runtime failures.

The latest stable guest release is v0.5.11. Compared with v0.5.9, its
relevant kernel fragment only adds DirtyFrag hardening, while v0.5.10 added
the nft userspace package. That may change Incus firewall-driver selection,
but the release does not add the named kernel symbols or matching ebtables
module packages. We have not run the same probes on v0.5.11, so this issue
asks for its status rather than asserting it still fails.

Environment

  • DStack: 0.5.9
  • Kernel: 6.9.0-dstack
  • Incus: 6.0.6
  • Docker owns an active legacy IPv4 ruleset with FORWARD DROP

Reproduced failures

1. The default DHCP bridge path requires a missing CHECKSUM target

Incus attempted:

iptables -w -t mangle -I POSTROUTING -o incusbr0 -p udp --dport 68 -j CHECKSUM --checksum-fill

It failed with:

Warning: Extension CHECKSUM revision 0 not supported, missing kernel module?
iptables: No chain/target/match by that name.

The captured kernel configuration had no
CONFIG_NETFILTER_XT_TARGET_CHECKSUM. Incus' Xtables
NetworkSetup
adds this target when IPv4 DHCP access is enabled.

2. The Xtables cleanup path cannot list the legacy IPv6 NAT table

Bridge creation still failed with DHCP disabled, no IPv6 address, and no
address forwards:

Failed applying firewall address forwards: Failed to list IPv6 rules (table nat)

Incus calls
forwardSetupFirewall
even for an empty forward set, and Xtables
clearNetworkForwards
loops over IPv4 and IPv6.

That unconditional family-6 cleanup is Incus behavior. The DStack image gap it
exposes is narrower: legacy ip6tables cannot list the NAT table. The same
host successfully services that query through Debian's real ip6tables-nft
shim.

3. The legacy ebtables bridge-filter path has no usable kernel support

Enabling the Incus bridge-filter path failed with:

modprobe: FATAL: Module ebtables not found in directory /lib/modules/6.9.0-dstack
The kernel doesn't support the ebtables 'filter' table.

The captured configuration had CONFIG_BRIDGE_NF_EBTABLES=m, but the running
image had no usable matching module tree, and the ebtables filter-table options
were not enabled. Incus' Xtables
InstanceSetupBridgeFilter
executes ebtables rules on this path; installing userspace alone cannot provide
the missing kernel tables.

Secondary bridge-filter observation

An earlier two-container probe with Incus 6.0.0 also found the nftables
bridge family unavailable. Incus renders per-instance filters in that family
in
Nftables.InstanceSetupBridgeFilter.
We retained the canary conclusion but not its exact stderr, so this is included
as supporting context rather than presented as a fourth equally evidenced
failure. We can rerun it on a disposable CVM.

Current workaround and impact

Our current configuration:

  • disables Incus DHCP and IPv6 and injects static addresses;
  • uses legacy iptables for IPv4 and ip6tables-nft for the unavoidable IPv6
    cleanup;
  • does not request Incus MAC/IP filtering or ACLs; and
  • uses the native bridge-port isolation flag for basic east-west isolation.

This avoids wrappers, ignored errors, custom firewall rules, locally built
modules, and an Incus fork. It also means the normal Incus filtering features
are unavailable, and we make no anti-spoof or destination/port ACL claim.

Request

Could DStack provide or identify one released netfilter path that supports
Incus managed bridges, including:

  • the CHECKSUM target used by the DHCP path;
  • coherent IPv4/IPv6 NAT behavior alongside DStack's Docker firewall; and
  • one complete bridge-filtering path, preferably nftables bridge, or the full
    matching ebtables kernel support and module packages?

If a newer image already provides these capabilities, please identify the
first image version and whether each capability is built in or shipped as a
loadable module. We can rerun the exact bridge and two-container probes on a
disposable CVM. We can also provide the retained configuration values, command
output, and failed service state from the zero-capacity test node; a rerun can
capture and attach the complete generated configuration.

Responsibility boundary

Incus' empty-set IPv6 cleanup and its failure-path dnsmasq/INPUT-rule residue
are relevant context but are not being reported as DStack defects. This issue
is limited to capabilities missing from the released DStack guest image.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions