Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpf: Introduce per-cluster conntrack maps #22857

Conversation

YutaroHayakawa
Copy link
Member

This is a part of the Cluster Mesh with overlapping PodCIDR support. When the remote cluster has an overlapping PodCIDR, we must identify the remote endpoint using IP + ClusterID. CT is not an exception, so it's ideal for extending the CT map's key with a new ClusterID field. However, the problem is we cannot extend the CT map without breaking users' connections now. Thus, we created a new map-in-map containing the CT maps for each cluster.

This PR only contains basic map definitions and tests. No one uses it. We also don't have GC for per-cluster maps because GC depends on the NAT map. Once I implement per-cluster NAT (it has the same problem), I'll publish another PR for GC.

bpf: Introduce per-cluster conntrack maps

@YutaroHayakawa YutaroHayakawa added sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. area/clustermesh Relates to multi-cluster routing functionality in Cilium. release-note/misc This PR makes changes that have no direct user impact. labels Dec 23, 2022
@github-actions github-actions bot added the kind/community-contribution This was a contribution made by a community member. label Dec 23, 2022
@YutaroHayakawa YutaroHayakawa force-pushed the yutaro/clustermesh-overlapping-podcidr-oss-per-cluster-ct branch from 3e71591 to b3c2b39 Compare December 26, 2022 09:50
@YutaroHayakawa YutaroHayakawa marked this pull request as ready for review December 26, 2022 10:58
@YutaroHayakawa YutaroHayakawa requested review from a team as code owners December 26, 2022 10:58
// key with ClusterID, we chose to create CT map per-cluster. When
// we have a good way of extending global CT maps in the future, we
// should retire this entire file.
type PerClusterCtMap struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PerClusterCTMap? https://go.dev/talks/2014

)

// An interface to interact with all per-cluster CT maps
type PerClusterCtMapsIface interface {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the Go convention: PerClusterCTMapper

@@ -0,0 +1,199 @@
// SPDX-License-Identifier: Apache-2.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to rename this to per_cluster_ctmap_test.go. With the removal of the privileged_tests build tag, privileged tests no longer need to be split into separate files, and the filename is getting rather long.

c.Assert(err, IsNil)

// Per-cluster CT map depends on the map information
InitMapInfo(option.CTMapEntriesGlobalTCPDefault, option.CTMapEntriesGlobalAnyDefault, true, true, true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already called in ctmap_privileged_test.go:init().

@@ -15,6 +15,18 @@ const (
ClusterIDMax = 255
)

func ValidateClusterID(clusterID uint32) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this change already part of #22875 ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but both PRs depend on this change, so let me rebase the branch after the one is merged into the master.

@YutaroHayakawa YutaroHayakawa force-pushed the yutaro/clustermesh-overlapping-podcidr-oss-per-cluster-ct branch from b3c2b39 to eef31a9 Compare January 4, 2023 06:26
@YutaroHayakawa
Copy link
Member Author

Applied the fix suggested by Timo.

@YutaroHayakawa
Copy link
Member Author

/test

@YutaroHayakawa
Copy link
Member Author

YutaroHayakawa commented Jan 11, 2023

  • k8s-1.16-kernel-4.9 is failing because map-in-map is from 4.12
  • All other tests are failing with Skipping symbol substitution warning. I need to add the name of the new maps to ignore list.

To distinguish two same IP addresses belong to the different clusters,
we need to extend conntrack maps' keys to contain ClusterID. However,
currently it is impossible to change conntrack maps without breaking
user's connection. Thus, we'll take a different approach that creates
conntrack maps per cluster.

Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com>
@YutaroHayakawa YutaroHayakawa force-pushed the yutaro/clustermesh-overlapping-podcidr-oss-per-cluster-ct branch from eef31a9 to 0ebbc7c Compare January 11, 2023 09:51
@YutaroHayakawa
Copy link
Member Author

@YutaroHayakawa
Copy link
Member Author

/test

@YutaroHayakawa YutaroHayakawa removed the kind/community-contribution This was a contribution made by a community member. label Jan 11, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Jan 11, 2023
@aditighag aditighag merged commit 20de35e into cilium:master Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/clustermesh Relates to multi-cluster routing functionality in Cilium. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/misc This PR makes changes that have no direct user impact. sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants