Multi-VRFs documentation#2665
Conversation
✅ Deploy Preview for calico-docs-preview-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ Deploy Preview for tigera failed. Why did it fail? →Built without sensitive environment variables
|
ctauchen
left a comment
There was a problem hiding this comment.
@matthewdupre Thanks for this! A few comments, mostly small suggestions.
One general thing we could improve (perhaps before GA), would be to make the full procedure more specific. Much of the procedural content is pretty high-level, leaving users to fill in the details themselves.
I'm happy enough to get this in now for testing and the release. Might be worth chatting about it and revising before GA.
LGTM, with suggestions.
| ``` | ||
|
|
||
| :::tip | ||
| On Ubuntu <= 25.04, `apt install linux-generic`. |
There was a problem hiding this comment.
This < character is killing the build. Suggest you rewrite as follows:
On Ubuntu 25.04 or earlier,
On Ubuntu 25.10 or later,
This does leave me wondering about the .05-.09 releases, for what it's worth.
There was a problem hiding this comment.
Ubuntu use a year.month format with 2 releases a year (04 and 10). That said I(/Claude) will just remove these.
|
|
||
| ## Big picture | ||
|
|
||
| Attach pods to one or more Virtual Routing and Forwarding (VRF) domains |
There was a problem hiding this comment.
| Attach pods to one or more Virtual Routing and Forwarding (VRF) domains | |
| Attach pods to one or more virtual routing and forwarding (VRF) domains |
| - **Reach and be reached from overlapping external IPs.** The same external IP address can be reused | ||
| in multiple tenant networks outside the cluster. Pods on a VRF can reach the | ||
| copy of that IP in their own tenant network, and responses come back in the | ||
| correct VRF. Services backed by VRF attached pods can also be exported to |
There was a problem hiding this comment.
| correct VRF. Services backed by VRF attached pods can also be exported to | |
| correct VRF. Services backed by VRF attached pods can also be exported to |
worth removing all double spaces
|
|
||
| Typically the easiest way to use VRFs will be to attach pods to the flat network | ||
| (with the primary interface), and then whichever VRF networks are required using | ||
| Multus `NetworkAttachmentDefinition`s. `multus-service` can then be used to |
There was a problem hiding this comment.
Best to describe generically or use the code as a modifier. Helpful for people not already familiar with what you're referring to.
using Multus network attachment definitions.
using Multus NetworkAttachmentDefinition resources.
Same goes for multus-service and similar cases elsewhere.
| **Required** | ||
|
|
||
| - $[prodname] installed with the **nftables dataplane** (`linuxDataplane: Nftables` in the [Installation](../../reference/installation/api.mdx) resource). | ||
| The iptables and eBPF dataplanes are not supported in the tech preview, eBPF will be added before GA but iptables support is not planned. |
There was a problem hiding this comment.
Remove. The first bit is covered in your previous line, and we don't want to be announcing future work. We'll add that when it happens.
| ```bash | ||
| sudo modprobe vrf && lsmod | grep '^vrf ' | ||
| ``` | ||
|
|
||
| :::tip | ||
| On Ubuntu <= 25.04, `apt install linux-generic`. | ||
| On Ubuntu >= 25.10, the module is built in. | ||
| On RHEL, the module is built in. | ||
| ::: |
There was a problem hiding this comment.
I'm inclined to say we can expect people to figure out whether they have this module and to install if necessary. For economy's sake, I'd scrap these lines.
| ::: | ||
|
|
||
| - BGP peering configured between each node and the upstream router on each VRF (see [Create BGPPeers and BGPFilters](#create-bgppeers-and-bgpfilters)). | ||
| - [Multus](./multiple-networks.mdx) installed if you want to attach pods to a VRF using a secondary interface (the most common topology). |
There was a problem hiding this comment.
| - [Multus](./multiple-networks.mdx) installed if you want to attach pods to a VRF using a secondary interface (the most common topology). | |
| - If you want to attach pods to a VRF using a secondary interface (the most common topology), you need to have [Multus](./multiple-networks.mdx) installed on your cluster. |
Always best to lead with conditionals.
| **Recommended** | ||
|
|
||
| - Pin `nodeAddressAutodetection` in the [Installation](../../reference/installation/api.mdx) to a specific interface (for example `eth0`) or to `kubernetes: NodeInternalIP`. The default "first found" autodetection can chase a VRF-attached interface when additional interfaces are brought up, breaking the cluster. |
There was a problem hiding this comment.
This sounds important. Can we just make it a requirement?
| - **Dataplane**: only the [nftables dataplane](../../operations/nftables.mdx) is supported. iptables and eBPF are not supported. | ||
| - **kube-proxy**: must be in `nftables` mode. `ipvs` mode is **never** supported (this is a permanent restriction, not a tech-preview limitation). |
There was a problem hiding this comment.
covered already in the requirements section
| ## Requirements | ||
|
|
||
| - Linux kernel **5.6 or later** (for the `meta sdifname` nftables match used by VRF policy dispatch). | ||
| - The `vrf` kernel module must be loaded on every node. On Ubuntu this is part of `linux-modules-extra-$(uname -r)`. Confirm with `sudo modprobe vrf && lsmod | grep '^vrf '`. | ||
| - $[prodname] must be installed with `linuxDataplane: Nftables` and kube-proxy must also be in nftables mode. | ||
| - The cluster's [Installation](../installation/api.mdx) should pin `nodeAddressAutodetection` to a specific interface or to `kubernetes: NodeInternalIP`. The default "first found" autodetection can chase a VRF-attached interface and break the cluster when extra interfaces are added. |
There was a problem hiding this comment.
this is already covered above
Address ctauchen review comment on multi-vrf.mdx:15: > ```suggestion > Attach pods to one or more virtual routing and forwarding (VRF) domains > ``` Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on multi-vrf.mdx:31:
> ```suggestion
> correct VRF. Services backed by VRF attached pods can also be exported to
> ```
> worth removing all double spaces
Applied the suggested fix on line 31 and also collapsed the double-space
on line 59 ("`NetworkAttachmentDefinition`s. `multus-service`").
Resolves tigera#2665 (comment)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on multi-vrf.mdx:59: > Best to describe generically or use the code as a modifier. Helpful for > people not already familiar with what you're referring to. > > using Multus network attachment definitions. > using Multus `NetworkAttachmentDefinition` resources. > > Same goes for multus-service and similar cases elsewhere. `multus-service` is already code-formatted in this file, so the only edit needed is the NetworkAttachmentDefinition phrasing on line 59. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on multi-vrf.mdx:115: > Remove. The first bit is covered in your previous line, and we don't > want to be announcing future work. We'll add that when it happens. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on multi-vrf.mdx:116: > Isn't this covered in the first item? Can Calico be in nftables while > kube-proxy is something else? Calico's dataplane and kube-proxy's mode are technically independent knobs, but for multi-VRF both must be nftables. Merging the two bullets keeps both requirements visible without making them look like separate prerequisites. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ments Address ctauchen review comment on multi-vrf.mdx:117: > This is covered already by the general system requirements: > https://docs.tigera.io/calico-enterprise/latest/getting-started/install-on-clusters/requirements#node-requirements Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…uild) Address ctauchen review comment on multi-vrf.mdx:125: > This `<` character is killing the build. Suggest you rewrite as follows: > > On Ubuntu 25.04 or earlier, > > On Ubuntu 25.10 or later, > > This does leave me wondering about the .05-.09 releases, for what it's worth. Ubuntu does not ship .05-.09 point releases, so the wording covers all Ubuntu versions cleanly. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on multi-vrf.mdx:128: > I'm inclined to say we can expect people to figure out whether they > have this module and to install if necessary. For economy's sake, > I'd scrap these lines. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on multi-vrf.mdx:131: > ```suggestion > - If you want to attach pods to a VRF using a secondary interface (the most common topology), you need to have [Multus](./multiple-networks.mdx) installed on your cluster. > ``` > Always best to lead with conditionals. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on multi-vrf.mdx:135: > This sounds important. Can we just make it a requirement? Moved the `nodeAddressAutodetection` bullet from the **Recommended** section to **Required**, and dropped the now-empty **Recommended** heading. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on network.mdx:154: > covered already in the requirements section Removed the kube-proxy bullet from Limitations. The Requirements section already states that kube-proxy must be in nftables mode; the "ipvs is never supported" sub-point (a permanent restriction, not a tech-preview limitation) is folded into that Requirements bullet so it stays documented. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address ctauchen review comment on network.mdx:171: > this is already covered above The `nodeAddressAutodetection` recommendation is deployment guidance that belongs in the how-to (Configure multi-VRF networking), not in this API reference page. The how-to lists it as a Required item. Resolves tigera#2665 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…all instructions) Follow-up to the earlier "drop `vrf` kernel module bullet" commit. The full bullet (with the modprobe one-liner and the per-distro tip) was scrapped per ctauchen's suggestion at multi-vrf.mdx:128: > I'm inclined to say we can expect people to figure out whether they > have this module and to install if necessary. For economy's sake, > I'd scrap these lines. Per author follow-up: keep the bare statement that the module must be available — just drop the detection / install instructions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to the earlier "drop duplicated kube-proxy limitation" commit on network.mdx, which folded an `ipvs is never supported` parenthetical into the Requirements bullet. Per author follow-up: drop it. `ipvs` is only minimally supported elsewhere in the product, and requiring nftables mode already implies "not ipvs", so the parenthetical adds noise without information. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@ctauchen thanks! Markups made. I've left them as separate commits (I like reviewing Claude better that way), but we should definitely squash before merging! |
<!---PR title format: [GH#][DOCS#]: --->Product Version(s):
Issue:
Link to docs preview:
SME review:
DOCS review:
Additional information:
Merge checklist: