Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

peering: ensure that merged central configs of peered upstreams for partitioned downstreams work #17179

Merged
merged 2 commits into from Apr 28, 2023

Conversation

rboyer
Copy link
Member

@rboyer rboyer commented Apr 28, 2023

Description

Partitioned downstreams with peered upstreams could not properly merge central config info (i.e. proxy-defaults and service-defaults things like mesh gateway modes) if the upstream had an empty DestinationPartition field in Enterprise.

Due to data flow, if this setup is done using Consul client agents the field is never empty and thus does not experience the bug.

When a service is registered directly to the catalog as is the case for consul-dataplane use this field may be empty and and the internal machinery of the merging function doesn't handle this well.

This PR ensure the internal machinery of that function is referentially self-consistent.

The easy way to observe the breakage is arrange for:

  • peering with at least the calling side being in a non-default partition
  • one server in each cluster
  • one mesh gateway in each cluster (in the non-default partition of choice)
  • the calling side should use consul-dataplane
  • the registration done to the catalog should leave the DestinationPartition field empty
  • create a partitioned proxy-defaults with meshgateway.mode="local"
  • have one mesh

After bringing everything up, the cluster to reach the upstream from the calling sidecar should point to the local cluster's mesh gateway address (local mode) and not the remote peer's mesh gateway (remote mode).

Needs manual backporting to at least 1.15 (1.14 is very different in this area and may not be applicable).

@rboyer rboyer self-assigned this Apr 28, 2023
psn.ServiceName.EnterpriseMeta.Normalize()

// Normalize the partition field specially.
if psn.Peer != "" {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A peered upstream MUST exist in the same partition as the downstream using it. When it is empty, inferring it as default is incorrect.

@@ -192,6 +206,12 @@ func MergeServiceConfig(defaults *structs.ServiceConfigResponse, service *struct
}

uid := us.DestinationID()

// Normalize the partition field specially.
if uid.Peer != "" {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the write earlier and the read here are now aligned on normalization, there will actually be map key hits now.

}
}

for _, partition := range partitions {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real method call was instrumented with a printf that printed the inputs/outputs to stdout.

One setup was done using client agents and the other using dataplanes and stdout capture was used to create these realistic input/outputs to showcase the shape things were in at this specific point.

LocalBindAddress: "0.0.0.0",
LocalBindPort: 5000,
MeshGateway: structs.MeshGatewayConfig{
Mode: "local", // This field vanishes if the merging does not work for dataplanes.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the only detectable divergence given these inputs.

@rboyer rboyer added the backport/1.15 Changes are backported to 1.15 label Apr 28, 2023
@rboyer rboyer changed the title peering: ensure that merged central configs for peered upstreams work peering: ensure that merged central configs of peered upstreams for partitioned downstreams work Apr 28, 2023
@rboyer rboyer marked this pull request as ready for review April 28, 2023 17:09
rboyer added a commit that referenced this pull request Apr 28, 2023
…ams for partitioned downstreams work

Backport of #17179 into release/1.15.x
@rboyer rboyer added pr/no-backport and removed backport/1.15 Changes are backported to 1.15 labels Apr 28, 2023
@rboyer rboyer merged commit 6b49869 into main Apr 28, 2023
125 checks passed
@rboyer rboyer deleted the fix-peering-merge-central-config branch April 28, 2023 17:36
rboyer added a commit that referenced this pull request Apr 28, 2023
…ams for partitioned downstreams work (#17181)

Backport of #17179 into release/1.15.x
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants