Native nftables dataplane

[caseydavenport]

Description

This is still a WIP, but putting it up in order to get initial feedback
and to get some CI runs started.

Todos

• UTs for new nftables code
• FVs for nftables
• Existing tests passing
• Add FelixConfiguration option to enable nftables
• Release note

Release Note

TBD

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

• docs-pr-required: This change requires a change to the documentation that has not been completed yet.
• docs-completed: This change has all necessary documentation completed.
• docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

• release-note-required: This PR has user-facing changes. Most PRs should have this label.
• release-note-not-required: This PR has no user-facing changes.

Other optional labels:

• cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
• needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

[caseydavenport]

One of the unfortunate consequences of using an interface type for the match is that leaving the Match type nil is no longer viable, as it results in nil pointer exceptions when calling bound functions where previously the function would have been called on a nil slice.

[fasaxc]

Yeah, I was trying to think of a way around that. Probably not worth it, but you could maybe do it with generics:

type Rule[M Match, A Action] {
  Match M
  Action A
}

Then the rule renderer could have [M Match, A Action] too and pass them through. Would create a different kind of boilerplate everywhere! Advantage would be that the compiler would enforce getting it right.

[tomastigera]

Would it make sense to rename generictables (sounds pretty generic, but it is not so much ;-) ) to say linuxtables or nettables and place both iptables and nftables below that? Now we have 3 top level tables packages, which are for linux only and one is just interfaces.

[tomastigera]

nit: What if we move this type now to generictables as say RuleChains and embed it in Table?

[caseydavenport]

Would it make sense to rename generictables (sounds pretty generic, but it is not so much ;-) ) to say linuxtables or nettables and

@tomastigera yep, I think those are both better!

[fasaxc]

Overall looks like great work! I don't think I spotted any major bugs, just a few nits and minor things along with a few places where it might be worth sharing some code between the dataplanes.

The Github UI is really struggling with the size of the PR so my preference would be for you to make the smaller markups in new commits (so I can review as a diff) and then to bank this before doing any further work to share code etc.

A few spec things to tie down:

• Config model; should we make a Dataplane enum and deprecate the old XXXEnabled flags?
• Interleaving rules, I know some customers like to add their own rules in between our "insert" and "append" layers, should we define a way to do that (e.g. can it be done with multiple chains at different priorities or something like that)

[fasaxc]

There's ConntrackState() above too? Could they be unified?

[caseydavenport]

In iptables mode, I believe these resolve down to the same match, but in nftables mode there are two separate matches - ct state and ct status for checking different types of conntrack information. ct state matches the state of the connection in conntrack (e.g., established, invalid, etc.) whereas ct status matches a number of other metadata bits related to the connection - like whether NAT is performed, or flow table offload is being used for this connection.

https://wiki.nftables.org/wiki-nftables/index.php/Matching_connection_tracking_stateful_metainformation#Matching_conntrack_metadata

[fasaxc]

If you did want to trim this down a bit, you could make a new Tables interface:

if nftables {
  tables = nftables.Tables(...)
}
...
ipSetsV4 = tables.IPSets()
...
mangleTableV4 = tables.Mangle()
filterTableV4 = tables.Filter()
...
<get rid of allTables field>
func apply(...) {
 ...
for _, t := range dp.tables.TablesToApply() {
  t.apply()
}

[fasaxc]

Yeah, I was trying to think of a way around that. Probably not worth it, but you could maybe do it with generics:

type Rule[M Match, A Action] {
  Match M
  Action A
}

Then the rule renderer could have [M Match, A Action] too and pass them through. Would create a different kind of boilerplate everywhere! Advantage would be that the compiler would enforce getting it right.

[danwinship]

(quick skim of just the nftables-specific bits...)

[danwinship]

hm, what does that mean?

[caseydavenport]

The in-memory metadata we keep for each set includes some things like the max size of the set. This function is attempting to synchronize our in-memory view of the set with what actually is programmed, but it's just copying the "desired" Size / Range metadata instead of reading it from the programmed set. I couldn't easily find a way to get this information from knft (can just get set names, and elements).

I think it's OK and actually not especially relevant, considering right now the nftables code doesn't actually use any of these pieces of metadata when programming the set.

[caseydavenport]

Config model; should we make a Dataplane enum and deprecate the old XXXEnabled flags?
Interleaving rules, I know some customers like to add their own rules in between our "insert" and "append" layers, should we define a way to do that (e.g. can it be done with multiple chains at different priorities or something like that)

I think I'm in favor of both of these, but they should probably each be separate PRs