[feature request] nftables support

[senden9]

Docker seems to be optimized for iptables at the moment. Are there any plans to support nftables in future versions of Docker?

My workaround at the moment is to deactivate the iptables integration via --iptables=false and then set the right rules for nftables by hand.

[thaJeztah]

I'm not aware of plans in this direction

ping @aboch is this planned? Worth doing?

[aboch]

I remember @mrjana had thought of using nftables last year. He knows more about the plan.
From what I read online, it seems nftables made it into kernel 3.13. Given docker supports up to linux 3.10, it may not be possible to move to nftables yet.

[mrjana]

Yeah nftables are not in the kernel until 3.14 and we can't use it to generally replace iptables yet.

[Yamakaky]

Maybe add it as an option? That way, those who have the latest kernel can use it. Currently I have to disable my nftables firewall to get the network working, it's fine on my machine but it's not an option on a server.

[itagent]

Any new ideas or progress. We are in transition to nftables and really would appreciate

[ford-perfect]

+1 for optional nftables support
Just some dates:
Linux LTS 3.10 has it's projected EOL in October 2017.
Debian 7.0's kernel is not supported anyway but Debian 8.0's one has nftables.
RHEL-7.3 EOL is not until 2024-06 and it runs 3.10 so there is a conflict here;
but the proposition is for an optional nfttables support additionally to the existing iptables support.

[gdahlm]

I wanted that RHEL 7 does have nfttables as a tech preview, and it would greatly simplify ipv6 as well as allowing for a simpler implementations of throttling and very useful tools like connection tracking or load-balancing.

[Gunni]

I want to add that i've been using nftables on Centos 7 for over a year now i believe, on dozens of different servers both with and without nat, using ipv6 and more, and have had no issues other than understanding the parse errors when i mess up. And i'm using Ansible to manage and generate the nftables rules file and atomically reload the service to apply new rules, or do nothing if it fails to parse.

And since nftables applies the entire ruleset in one atomic operation, there is no moment when the system is in a partially configured state.

In my opinion i would NOT use nftables integration with docker unless i could control which file docker puts rules into and control the imports into my current ruleset myself and that docker would only issue reload commands to nftables (reload meaning nft -f , or systemctl which does it correctly).

I currently manage docker nat rules using ansible/manually.

[ojab]

Meanwhile iptables is officially deprecated.

[cpuguy83]

I don't see the reason to bother with nftables when the whole community seems to be (rightly) pushing for bpf.

[ojab]

nftables uses bpf internally. If you've implied bpfilter — it's not there yet.

[cpuguy83]

Sure it uses bpf internally, but it's not really any better than without bpf but rather about deduplication.
Even with bpf in the backend, nftables is still "slightly better" than iptables.

For that matter isn't iptables using nftables in the backed? (Don't quote me on that, I think I read that somewhere at some point, haven't looked into it).

[tianon]

According to https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_iptables_to_nftables (which I'd imagine is pretty authoritative), using nftables and iptables at the same time is highly discouraged:

Beware of using both the nft and the legacy tools at the same time. That means using both x_tables and nf_tables kernel subsystems at the same time, and could lead to unexpected results.

I'd been playing with firewalld for building a router system and got tired of the way firewalld does things, so I was evaluating nftables, but the fact that I'd then have to disable Docker's iptables behavior and handle Docker's routing rules myself is a bit of a hurdle.

I've looked at doing eBPF, but it doesn't seem like there's nearly as many good examples (even nftables is a bit low on examples, but I've managed to find a few people doing things similar enough to what I need that I'm comfortable), so I don't really think it's totally fair to tell folks "we should just go straight to BPF instead" yet.

Just to include what I've found for reference, here's a couple folks who've worked on getting what Docker needs implemented in nftables:

• https://gist.github.com/dearing/9388218f3c6ef6e48114
• https://github.com/oniGino/docker-nftables-scripts/blob/6f6378a82e8c20c8bfefc0901b19c9d755782127/docker-nft.conf

I think docker network create's ability to create arbitrary bridges is going to further complicate this, but for my own use case I'll be able to dictate a fixed number of Docker networks, so that won't be a huge deal (just bringing it up in case folks in the future find this and need to implement something similar).

On implementation details, is the current iptables/firewalld code tightly coupled with the rest of the networking system, or is it already abstracted out reasonably enough that eBPF or nftables could theoretically be implemented as an optional backend? Is there perhaps a way we could make that code pluggable, or at least pluggability friendly? Even just having Docker write out to a file the set of things it would've asked iptables to do would be an improvement; isn't it mostly port openings and masquerade settings?

(Not trying to be a bother, just trying to add some additional information about why folks might care about this and brainstorm ideas for how it could maybe move forward without being too invasive. ❤️)

Edit (2018-08-13): #35777 is also relevant (even with --iptables=false, Docker still currently touches iptables to create DOCKER-USER).

[cpuguy83]

is the current iptables/firewalld code tightly coupled with the rest of the networking system

It is horribly coupled right now. It's basically all the original iptables code from years ago moved out of docker/docker into docker/libnetwork and mostly not touched except to add more cruft to it to support custom chains (remember when docker didn't use it's own chain?) and firewalld, among other things.

[pentago]

Hi all, any new progress on this?

nftables are getting default with Debian 10 (Buster) due in couple of months and with it goes the wave of adoption in derivative distros such as Ubuntu I guess.

Having upgrade season and iptables deprecation closing in quickly something will need to happen.

[cpuguy83]

No progress as far as I am aware. Brian Goff

…

On Jan 2, 2019, at 03:59, Goran ***@***.***> wrote: Hi all, any new progress on this? nftables are getting default with Debian 10 (Buster) due in couple of months and with it goes the wave of adoption in derivative distros such as Ubuntu I guess. Having upgrade season and iptables deprecation closing in quickly something will need to happen. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[camAtGitHub]

Redhat 8 (currently in Beta) - Notes: The nftables framework replaces iptables in the role of the default network packet filtering facility.
That means CentOS 8 will follow suit more than likely also...

[mavenugo]

Thanks @camAtGitHub for the pointer.

The iptables, ip6tables, ebtables and arptables tools are replaced by nftables-based drop-in replacements with the same name. While external behavior is identical to their legacy counterparts, internally they use nftables with legacy netfilter kernel modules through a compatibility interface where required.

makes it the correct transition path. No change required for existing software making use of netfilter based iptables.

[tianon]

That's accurate except with the catch that some of the complex "iptables" expressions Docker invokes make the nftables shims choke, so either those tools need more attention specific to how Docker is trying to use them, or Docker needs explicit "nftables" support.

[cpuguy83]

I'd go for the latter, but I've also had PR's open for multiple years untouched on libnetwork.

[elboulangero]

Debian user reported that indeed, he can't use docker on a machine where a nftables-based firewall is enabled: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921600

[schka17]

It looks like it would work on Debian buster with the following conditions:

• use an ip and ipv6 table instead of inet
• name all chains exactly as in iptables: INPUT, OUTPUT & FORWARD

Source: https://ehlers.berlin/blog/nftables-and-docker/
Edit: I can confirm it :).

thank you for this tip, since my nftables I was troubleshooting looks like this now:

nft list ruleset | egrep "table|hook|chain"
table inet filter {
	chain global {
	chain INPUT {
		type filter hook input priority 0; policy drop;
	chain OUTPUT {
		type filter hook output priority 0; policy accept;
table ip nat {
	chain PREROUTING {
		type nat hook prerouting priority 0; policy accept;
	chain INPUT {
		type nat hook input priority 100; policy accept;
	chain POSTROUTING {
		type nat hook postrouting priority 100; policy accept;
	chain OUTPUT {
		type nat hook output priority -100; policy accept;
	chain DOCKER {
table ip filter {
	chain INPUT {
		type filter hook input priority 0; policy accept;
	chain FORWARD {
		type filter hook forward priority 0; policy accept;
	chain OUTPUT {
		type filter hook output priority 0; policy accept;
	chain DOCKER {
	chain DOCKER-ISOLATION-STAGE-1 {
	chain DOCKER-ISOLATION-STAGE-2 {
	chain DOCKER-USER {
table inet f2b-table-docker {
	chain f2b-chain {
		type filter hook forward priority -1; policy accept;

and it does not work as expected ..

Notice there is an input and output chain in the ip nat table....
Docker did that, because I didn't (checking now)

i can confirm it works creating an empty /etc/nftables.conf with this content:
`table ip nat {

    chain PREROUTING  {
            type nat hook prerouting priority -100; policy accept;
    }

    chain INPUT {
            type nat hook input priority 100; policy accept;
    }

    chain POSTROUTING {
            type nat hook postrouting priority 100; policy accept;
    }

    chain OUTPUT {
            type nat hook output priority -100; policy accept;
    }

}

table ip filter {
chain INPUT {
type filter hook input priority 0; policy accept;
}
chain FORWARD {
type filter hook forward priority 0; policy accept;
}
chain OUTPUT {
type filter hook output priority -100; policy accept;
}

}
`
then restart nftables and docker service. I didn't add nftable rules yet, but this is the next step

[mvorisek]

Is there any progress on this issue? Can Docker check if nftables is present and use it instead of iptables? The commands should be the same, only the grammar is different.

[tao12345666333]

I want to play with this package https://github.com/google/nftables

[camAtGitHub]

Hello again, Redhat 9 (just released) - Notes: The ipset and iptables-nft (iptables shim) packages have been deprecated.

When loading iptables modules you get a kernel message: Warning: <module_name> - this driver is not recommended for new deployments. It continues to be supported in this RHEL release, but it is likely to be removed in the next major release [1]

IPTables days (read: years) are numbered!

[marek22k]

Is there any news? It's been a while since the issue was created.

[polarathene]

There is work towards this in the networking support, starting with an IPVS feature that if successful may establish the way forward for introducing nftables support later IIRC.

[Gunni]

I'd like to re-emphasize that it be done in a way that supports Atomic rule replacement.

Being able to modify my nftables file and reload(1) my firewall without it ever going down is amazing, and I also don't want my docker rules to vanish if I reload the firewall either.

(1) AFAIK on systemd systems reload is implemented with a service that has
ExecReload=/sbin/nft 'flush ruleset; include "/etc/sysconfig/nftables.conf";'

[trainzkid]

(1) AFAIK on systemd systems reload is implemented with a service that has ExecReload=/sbin/nft 'flush ruleset; include "/etc/sysconfig/nftables.conf";'

While I agree with your point, I do want to make a note that fail2ban doesn't play nice with reloading like that, and as such, not all systemd systems provide a reload field for nftables. I know my Arch systems don't. That may be more of a conversation for fail2ban though, not nftables..

[trainzkid]

It looks like it would work on Debian buster with the following conditions:

• use an ip and ipv6 table instead of inet
• name all chains exactly as in iptables: INPUT, OUTPUT & FORWARD

Source: https://ehlers.berlin/blog/nftables-and-docker/
Edit: I can confirm it :).

thank you for this tip, since my nftables I was troubleshooting looks like this now:

nft list ruleset | egrep "table|hook|chain"
table inet filter {
	chain global {
	chain INPUT {
		type filter hook input priority 0; policy drop;
	chain OUTPUT {
		type filter hook output priority 0; policy accept;
table ip nat {
	chain PREROUTING {
		type nat hook prerouting priority 0; policy accept;
	chain INPUT {
		type nat hook input priority 100; policy accept;
	chain POSTROUTING {
		type nat hook postrouting priority 100; policy accept;
	chain OUTPUT {
		type nat hook output priority -100; policy accept;
	chain DOCKER {
table ip filter {
	chain INPUT {
		type filter hook input priority 0; policy accept;
	chain FORWARD {
		type filter hook forward priority 0; policy accept;
	chain OUTPUT {
		type filter hook output priority 0; policy accept;
	chain DOCKER {
	chain DOCKER-ISOLATION-STAGE-1 {
	chain DOCKER-ISOLATION-STAGE-2 {
	chain DOCKER-USER {
table inet f2b-table-docker {
	chain f2b-chain {
		type filter hook forward priority -1; policy accept;

and it does not work as expected ..

Notice there is an input and output chain in the ip nat table....
Docker did that, because I didn't (checking now)

i can confirm it works creating an empty /etc/nftables.conf with this content:
`table ip nat {

    chain PREROUTING  {
            type nat hook prerouting priority -100; policy accept;
    }

    chain INPUT {
            type nat hook input priority 100; policy accept;
    }

    chain POSTROUTING {
            type nat hook postrouting priority 100; policy accept;
    }

    chain OUTPUT {
            type nat hook output priority -100; policy accept;
    }

}

table ip filter {
chain INPUT {
type filter hook input priority 0; policy accept;
}
chain FORWARD {
type filter hook forward priority 0; policy accept;
}
chain OUTPUT {
type filter hook output priority -100; policy accept;
}

}
`
then restart nftables and docker service. I didn't add nftable rules yet, but this is the next step

Is policy accept required for all of those, or do the chains just need to be there for it to work? The last thing I want to do is leave the input chain as default accept.

[aviallon]

FYI, we had to remove almost every usage of Docker in our organisation because of this.
Now everything is migrated to podman.

[jakubgs]

You can use both nftables and iptables using different priority hooks, see this answer:
https://unix.stackexchange.com/questions/657545/nftables-whitelisting-docker

This does actually work. The answer mentions adding marks to packets handled by iptables, but it's not actually necessary.
The mark is there only if you want to handle packets filtered by iptables separately in nftables.

The key thing is to use higher hook priority than 0, which is the default for iptables:

type filter hook forward priority 10; policy drop;

Which means packets are first filtered by iptables, and then by nftables, allowing you to use both.

[Talkless]

You can use both nftables and iptables using different priority hooks

But this will not work if you want to use confiscators like firewalld, foomuuri, etc, running in nftables mode, as everything will probably be flushed by these tools?

[cpuguy83]

docker supports firewalld.
If firewalld is detected it will insert rules there rather than directly.

Is there some specific case with firewalld that is broken?
Having not used the firewalld integration for anything other than some test years ago, I'm definitely blind to this personally.

[Talkless]

docker supports firewalld.

Yes, but I believe it does only if firewalld runs in iptables mode, not on nftables.

[westurner]

FWIU podman 5.0 w/ containers/netavark has support for nftables and aardvark-dns:

From https://github.com/containers/podman/blob/7b95a8af80da7dcefe0bcbce3a24002a038c55a9/cni/README.md#L1-L4:

**Note**: The CNI backend is deprecated and will be removed
in the next major Podman version 5.0, in preference of Netavark,
see **[podman-network(1)](../docs/source/markdown/podman-network.1.md)** on how to change the backend.

There are a wide variety of different [CNI](https://github.com/containernetworking/cni) network configurations

netavark:

• src: https://github.com/containers/netavark
• docs: https://github.com/containers/netavark/tree/main/docs
  • https://github.com/containers/podman/blob/main/docs/tutorials/basic_networking.md
• https://bugzilla.redhat.com/show_bug.cgi?id=2023436
  • /? netavark nftables: https://issues.redhat.com/issues/?jql=text%20~%20%22netavark%20nftables%22
    • https://issues.redhat.com/browse/RUN-1577?jql=text%20~%20%22netavark%20nftables%22
    • Initial draft of nftables support containers/netavark#883
    • Add support for isolation to the nftables driver containers/netavark#888
    • https://issues.redhat.com/browse/RHEL-3088
• Feature request: allow forced netavark firewall driver containers/common#1338 (comment) ::

  I think the better solution would be config based network setup just like cni plugins

• https://github.com/containers/common/blame/main/docs/containers.conf.5.md#L458
  firewall_driver="" and pasta_options="" in containers.conf

fwiw also pasta is a replacement for slip4netns:

• https://github.com/rootless-containers/slirp4netns
• Add pasta networking mode containers/podman#16141