Skip to content

Commit

Permalink
envoy: Patch original_dst_cluster
Browse files Browse the repository at this point in the history
[upstream commit d34c0d3]

Patch original destination cluster to avoid multiple hosts for the same
address.

Connection pool containers use HostSharedPtr as map keys, rather than the
address of the host. This leads to multiple connections when there are
multiple Host instances for the same address. This is breaking use of the
original source address and port for upstream connections since only one
such connection can exist at any one time.

Original destination cluster implementation creates such duplicate Host
instances when two worker threads are racing to create a Host for the
same destination at the same time.

Fix this by keeping a separate 'updates_map' where each worker places a
newly created Host for the original destination. This map is used to look
for the Host is it can not be found from the shared read-only
'host_map'. Access to 'updates_map' is syncronized so that it can be
safely shared by the worker threads. The main threads consolidates the
updates from the 'updates_map' to a new instance of the shared, read-only
hosts map, so that the workers do not need to stall for possibly large
map updates.

Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
  • Loading branch information
jrajahalme committed Jun 4, 2024
1 parent 777afdf commit dcdcc72
Show file tree
Hide file tree
Showing 3 changed files with 468 additions and 3 deletions.
1 change: 1 addition & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ git_repository(
"@//patches:0002-upstream-Add-callback-for-upstream-authorization.patch",
"@//patches:0003-tcp_proxy-Add-filter-state-proxy_read_before_connect.patch",
"@//patches:0004-listener-add-socket-options.patch",
"@//patches:0005-original_dst_cluster-Avoid-multiple-hosts-for-the-sa.patch",
],
# // clang-format off: Envoy's format check: Only repository_locations.bzl may contains URL references
remote = "https://github.com/envoyproxy/envoy.git",
Expand Down
12 changes: 9 additions & 3 deletions cilium/socket_option.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
#include "envoy/config/core/v3/base.pb.h"
#include "envoy/network/listen_socket.h"

#include "source/common/common/hex.h"
#include "source/common/common/logger.h"
#include "source/common/common/utility.h"

Expand Down Expand Up @@ -189,9 +190,14 @@ class SocketMarkOption : public Network::Socket::Option,
absl::uint128 raw_address = ip->ipv6()->address();
addressIntoVector(key, raw_address);
}
// Add source port to the hash key
key.emplace_back(uint8_t(port >> 16));
key.emplace_back(uint8_t(port));
// Add source port to the hash key if defined
if (port != 0) {
ENVOY_LOG(trace, "hashKey port: {:x}", port);
key.emplace_back(uint8_t(port >> 8));
key.emplace_back(uint8_t(port));
}
ENVOY_LOG(trace, "hashKey after Cilium: {}, source: {}", Hex::encode(key),
original_source_address_->asString());
} else {
// Add the source identity to the hash key. This will separate upstream
// connection pools per security ID.
Expand Down
Loading

0 comments on commit dcdcc72

Please sign in to comment.