Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High memory utilization of istio-proxy proportional to number of services #7912

Closed
robertpanzer opened this issue Aug 14, 2018 · 9 comments
Closed

Comments

@robertpanzer
Copy link
Contributor

Describe the bug
The Istio Proxy has a very high memory consumption when many services (>100) are configured.
It also grows with the number of services.
And at some point pilot isn't able anymore to push all routes and clusters to the istio-proxy.

Expected behavior
Memory consumption should be independent on the number of services configured in the cluster and only depend on the number of services that are related to the local pod.

Steps to reproduce the bug
Create many ServiceEntries and VirtualServices and check the output of ps -A -o rss,cmd in the proxy.

Version
Istio 1.0.0
Kubernetes:
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:17:39Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:05:37Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Docker for Mac 18.06.0-ce

Is Istio Auth enabled or not?
No.
Installed with the helm chart:

helm install install/kubernetes/helm/istio --name istio --namespace istio-system

Environment
See above, running on Docker for Mac.

I created one deployment that just sleeps and runs with Istio, which also allows me to log into the container and send curl requests.

Then I created the services with this groovy script:

def s = ""
for (i=0; i < 1000; i++) {
    ip = "192.168.${(int)(i / 200)}.${i % 200}"
s += """---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: egresstestprovider${i}
spec:
  hosts:
  - egresstestproviderrp${i}.external
  ports:
  - number: 80
    name: http
    protocol: HTTP
  resolution: STATIC
  location: MESH_EXTERNAL
  endpoints:
  - address: ${ip}
    ports:
      http: 8086
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: egresstestservice${i}
spec:
  hosts:
  - egresstestproviderrp${i}.external
  http:
  - route:
    - destination:
        host: egresstestproviderrp${i}.external
"""

}
new File("externals.yaml").text = s

I fetched the proxy config and the Envoy config_dump and Envoy clusters before adding any additional services, after 100, after 500 and after 1000 services.
The rss looked like this after the 4 steps:

  RSS CMD
17168 /usr/local/bin/pilot-agent proxy sidecar --configPath /etc/istio/proxy --binaryPath /usr/local/bin/envoy --serviceCluster testcurl --drainDuration 45s --parentShutdownDuration 1m0s --discoveryAddress istio-pilot.istio-system:
38024 /usr/local/bin/envoy -c /etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --parent-shutdown-time-s 60 --service-cluster testcurl --service-node sidecar~10.1.7.4~testcurl-8578fbd87-x9gsl.default~default.svc.
 
  RSS CMD
17172 /usr/local/bin/pilot-agent proxy sidecar --configPath /etc/istio/proxy --
46736 /usr/local/bin/envoy -c /etc/istio/proxy/envoy-rev0.json --restart-epoch
 
  RSS CMD
17172 /usr/local/bin/pilot-agent proxy sidecar --configPath /etc/istio/proxy --binaryPath /usr/local/bin/envoy --serviceCluster testcurl --drainDuration 45s --parentShutdownDuration 1m0s --discoveryAddress istio-pilot.istio-system:
96632 /usr/local/bin/envoy -c /etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --parent-shutdown-time-s 60 --service-cluster testcurl --service-node sidecar~10.1.7.4~testcurl-8578fbd87-x9gsl.default~default.svc.
 
  RSS CMD
16280 /usr/local/bin/pilot-agent proxy sidecar --configPath /etc/istio/proxy --binaryPath /usr/local/bin/envoy --serviceCluster testcurl --drainDuration 45s --parentShutdownDuration 1m0s --discoveryAddress istio-pilot.istio-system:
133556 /usr/local/bin/envoy -c /etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --parent-shutdown-time-s 60 --service-cluster testcurl --service-node sidecar~10.1.7.4~testcurl-8578fbd87-x9gsl.default~default.svc

As you can see the memory utilization for the sidecar goes up to 133MB for a scenario that should be rather common in anything that is bigger than the Bookstore (considering that you have to declare services for every single external dependency)

You can find everything in the attached zip file:
logs.zip

I also have to note that I took the last snapshot when only 78% of the clusters were synched.
Even after waiting for more than 3 hours the proxy-status didn't show more.
And I checked the Envoy config directly that nothing more was pushed down.

@louiscryan
Copy link
Contributor

@costinm @PiotrSikora @mandarjog

Thanks for the detailed report. I suspect there is significant overlap with the issues collectively tracked in the Epic I've just attached this report to.

@louiscryan
Copy link
Contributor

See #7879

@PiotrSikora
Copy link
Contributor

Per @htuch, the per cluster overhead is ~118KB, which is close enough to the ~95MB overhead you're seeing for the 783 new clusters (not sure why it's not 1000 new clusters, though, perhaps you did config dump before proxy received all updates?), so unfortunately this is currently expected.

@robertpanzer
Copy link
Contributor Author

~95MB overhead you're seeing for the 783 new clusters (not sure why it's not 1000 new clusters, though

I created the dump about 10 minutes after I created the last 500 services.
At that time only about 78% of the clusters were pushed to the istio-proxy.
I kept the thing running for more than 3 hours and it was still hanging at 78%.
Pilot didn't show anything in the logs (except the usual messages http: multiple response.WriteHeader calls when calling istioctl ps)

@nmittler
Copy link
Contributor

@PiotrSikora I've assigned to you for now ... feel free to re-assign as appropriate.

PiotrSikora added a commit to PiotrSikora/proxy that referenced this issue Aug 22, 2018
This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68d stats: refactoring MetricImpl without strings (istio#4190)
36809d8 fuzz: coverage profile generation instructions. (istio#4193)
ba40cc9 upstream: rebuild cluster when health check config is changed (istio#4075)
05c0d52 build: use clang-6.0. (istio#4168)
01f403e thrift_proxy: introduce header transport (istio#4082)
564d256 tcp: allow connection pool callers to store protocol state (istio#4131)
3e1d643 configs: match latest API changes (istio#4185)
bc6a10c Fix wrong mock function name. (istio#4187)
e994c1c Bump yaml-cpp so it builds with Visual Studio 15.8 (istio#4182)
3d1325e Converting envoy configs to V2 (istio#2957)
8d0680f Add timestamp to HealthCheckEvent definition (istio#4119)
497efb9 server: handle non-EnvoyExceptions safely if thrown in constructor. (istio#4173)
6d6fafd config: strengthen validation for gRPC config sources. (istio#4171)
132302c fuzz: reduce log level when running under fuzz engine. (istio#4161)
7c04ac2 test: fix V6EmptyOptions in coverage with IPv6 enable (istio#4155)
1b2219b ci: remove deprecated bazel --batch option (istio#4166)
2db6a4c ci: update clang to version 6.0 in the Ubuntu build image. (istio#4157)
71152b7 ratelimit: Add ratelimit custom response headers (istio#4015)
3062874 ssl: make Ssl::Connection const everywhere (istio#4179)
706e262 Handle validation of non expiring tokens in jwt_authn filter (istio#4007)
f06e958 fuzz: tag trivial fuzzers with no_fuzz. (istio#4156)
27fb1d3 thrift_proxy: add service name matching to router implementation (istio#4130)
8c189a5 Make over provisioning factor configurable (istio#4003)
6c08fb4 Making test less flaky (istio#4149)
592775b fuzz: bare bones HCM fuzzer. (istio#4118)

For istio/istio#7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
PiotrSikora added a commit to PiotrSikora/proxy that referenced this issue Aug 22, 2018
This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68d stats: refactoring MetricImpl without strings (istio#4190)
36809d8 fuzz: coverage profile generation instructions. (istio#4193)
ba40cc9 upstream: rebuild cluster when health check config is changed (istio#4075)
05c0d52 build: use clang-6.0. (istio#4168)
01f403e thrift_proxy: introduce header transport (istio#4082)
564d256 tcp: allow connection pool callers to store protocol state (istio#4131)
3e1d643 configs: match latest API changes (istio#4185)
bc6a10c Fix wrong mock function name. (istio#4187)
e994c1c Bump yaml-cpp so it builds with Visual Studio 15.8 (istio#4182)
3d1325e Converting envoy configs to V2 (istio#2957)
8d0680f Add timestamp to HealthCheckEvent definition (istio#4119)
497efb9 server: handle non-EnvoyExceptions safely if thrown in constructor. (istio#4173)
6d6fafd config: strengthen validation for gRPC config sources. (istio#4171)
132302c fuzz: reduce log level when running under fuzz engine. (istio#4161)
7c04ac2 test: fix V6EmptyOptions in coverage with IPv6 enable (istio#4155)
1b2219b ci: remove deprecated bazel --batch option (istio#4166)
2db6a4c ci: update clang to version 6.0 in the Ubuntu build image. (istio#4157)
71152b7 ratelimit: Add ratelimit custom response headers (istio#4015)
3062874 ssl: make Ssl::Connection const everywhere (istio#4179)
706e262 Handle validation of non expiring tokens in jwt_authn filter (istio#4007)
f06e958 fuzz: tag trivial fuzzers with no_fuzz. (istio#4156)
27fb1d3 thrift_proxy: add service name matching to router implementation (istio#4130)
8c189a5 Make over provisioning factor configurable (istio#4003)
6c08fb4 Making test less flaky (istio#4149)
592775b fuzz: bare bones HCM fuzzer. (istio#4118)

For istio/istio#7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
PiotrSikora added a commit to PiotrSikora/proxy that referenced this issue Aug 22, 2018
This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68d stats: refactoring MetricImpl without strings (istio#4190)
36809d8 fuzz: coverage profile generation instructions. (istio#4193)
ba40cc9 upstream: rebuild cluster when health check config is changed (istio#4075)
05c0d52 build: use clang-6.0. (istio#4168)
01f403e thrift_proxy: introduce header transport (istio#4082)
564d256 tcp: allow connection pool callers to store protocol state (istio#4131)
3e1d643 configs: match latest API changes (istio#4185)
bc6a10c Fix wrong mock function name. (istio#4187)
e994c1c Bump yaml-cpp so it builds with Visual Studio 15.8 (istio#4182)
3d1325e Converting envoy configs to V2 (istio#2957)
8d0680f Add timestamp to HealthCheckEvent definition (istio#4119)
497efb9 server: handle non-EnvoyExceptions safely if thrown in constructor. (istio#4173)
6d6fafd config: strengthen validation for gRPC config sources. (istio#4171)
132302c fuzz: reduce log level when running under fuzz engine. (istio#4161)
7c04ac2 test: fix V6EmptyOptions in coverage with IPv6 enable (istio#4155)
1b2219b ci: remove deprecated bazel --batch option (istio#4166)
2db6a4c ci: update clang to version 6.0 in the Ubuntu build image. (istio#4157)
71152b7 ratelimit: Add ratelimit custom response headers (istio#4015)
3062874 ssl: make Ssl::Connection const everywhere (istio#4179)
706e262 Handle validation of non expiring tokens in jwt_authn filter (istio#4007)
f06e958 fuzz: tag trivial fuzzers with no_fuzz. (istio#4156)
27fb1d3 thrift_proxy: add service name matching to router implementation (istio#4130)
8c189a5 Make over provisioning factor configurable (istio#4003)
6c08fb4 Making test less flaky (istio#4149)
592775b fuzz: bare bones HCM fuzzer. (istio#4118)

For istio/istio#7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
istio-testing pushed a commit to istio/proxy that referenced this issue Aug 23, 2018
This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68d stats: refactoring MetricImpl without strings (#4190)
36809d8 fuzz: coverage profile generation instructions. (#4193)
ba40cc9 upstream: rebuild cluster when health check config is changed (#4075)
05c0d52 build: use clang-6.0. (#4168)
01f403e thrift_proxy: introduce header transport (#4082)
564d256 tcp: allow connection pool callers to store protocol state (#4131)
3e1d643 configs: match latest API changes (#4185)
bc6a10c Fix wrong mock function name. (#4187)
e994c1c Bump yaml-cpp so it builds with Visual Studio 15.8 (#4182)
3d1325e Converting envoy configs to V2 (#2957)
8d0680f Add timestamp to HealthCheckEvent definition (#4119)
497efb9 server: handle non-EnvoyExceptions safely if thrown in constructor. (#4173)
6d6fafd config: strengthen validation for gRPC config sources. (#4171)
132302c fuzz: reduce log level when running under fuzz engine. (#4161)
7c04ac2 test: fix V6EmptyOptions in coverage with IPv6 enable (#4155)
1b2219b ci: remove deprecated bazel --batch option (#4166)
2db6a4c ci: update clang to version 6.0 in the Ubuntu build image. (#4157)
71152b7 ratelimit: Add ratelimit custom response headers (#4015)
3062874 ssl: make Ssl::Connection const everywhere (#4179)
706e262 Handle validation of non expiring tokens in jwt_authn filter (#4007)
f06e958 fuzz: tag trivial fuzzers with no_fuzz. (#4156)
27fb1d3 thrift_proxy: add service name matching to router implementation (#4130)
8c189a5 Make over provisioning factor configurable (#4003)
6c08fb4 Making test less flaky (#4149)
592775b fuzz: bare bones HCM fuzzer. (#4118)

For istio/istio#7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
rshriram pushed a commit to istio/proxy that referenced this issue Aug 23, 2018
…). (#1939)

This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68d stats: refactoring MetricImpl without strings (#4190)
36809d8 fuzz: coverage profile generation instructions. (#4193)
ba40cc9 upstream: rebuild cluster when health check config is changed (#4075)
05c0d52 build: use clang-6.0. (#4168)
01f403e thrift_proxy: introduce header transport (#4082)
564d256 tcp: allow connection pool callers to store protocol state (#4131)
3e1d643 configs: match latest API changes (#4185)
bc6a10c Fix wrong mock function name. (#4187)
e994c1c Bump yaml-cpp so it builds with Visual Studio 15.8 (#4182)
3d1325e Converting envoy configs to V2 (#2957)
8d0680f Add timestamp to HealthCheckEvent definition (#4119)
497efb9 server: handle non-EnvoyExceptions safely if thrown in constructor. (#4173)
6d6fafd config: strengthen validation for gRPC config sources. (#4171)
132302c fuzz: reduce log level when running under fuzz engine. (#4161)
7c04ac2 test: fix V6EmptyOptions in coverage with IPv6 enable (#4155)
1b2219b ci: remove deprecated bazel --batch option (#4166)
2db6a4c ci: update clang to version 6.0 in the Ubuntu build image. (#4157)
71152b7 ratelimit: Add ratelimit custom response headers (#4015)
3062874 ssl: make Ssl::Connection const everywhere (#4179)
706e262 Handle validation of non expiring tokens in jwt_authn filter (#4007)
f06e958 fuzz: tag trivial fuzzers with no_fuzz. (#4156)
27fb1d3 thrift_proxy: add service name matching to router implementation (#4130)
8c189a5 Make over provisioning factor configurable (#4003)
6c08fb4 Making test less flaky (#4149)
592775b fuzz: bare bones HCM fuzzer. (#4118)

For istio/istio#7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
PiotrSikora added a commit to PiotrSikora/istio that referenced this issue Aug 23, 2018
This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/istio/proxy:

2656f34 Update Envoy SHA to latest with MetricImpl optimizations (release-1.0). (istio#1939)

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68dda stats: refactoring MetricImpl without strings (istio#4190)
36809d80a fuzz: coverage profile generation instructions. (istio#4193)
ba40cc933 upstream: rebuild cluster when health check config is changed (istio#4075)
05c0d52d3 build: use clang-6.0. (istio#4168)
01f403ec4 thrift_proxy: introduce header transport (istio#4082)
564d256fb tcp: allow connection pool callers to store protocol state (istio#4131)
3e1d643b9 configs: match latest API changes (istio#4185)
bc6a10c2f Fix wrong mock function name. (istio#4187)
e994c1c0b Bump yaml-cpp so it builds with Visual Studio 15.8 (istio#4182)
3d1325e89 Converting envoy configs to V2 (istio#2957)
8d0680feb Add timestamp to HealthCheckEvent definition (istio#4119)
497efb95b server: handle non-EnvoyExceptions safely if thrown in constructor. (istio#4173)
6d6fafdb3 config: strengthen validation for gRPC config sources. (istio#4171)
132302caf fuzz: reduce log level when running under fuzz engine. (istio#4161)
7c04ac255 test: fix V6EmptyOptions in coverage with IPv6 enable (istio#4155)
1b2219bd7 ci: remove deprecated bazel --batch option (istio#4166)
2db6a4ce1 ci: update clang to version 6.0 in the Ubuntu build image. (istio#4157)
71152b710 ratelimit: Add ratelimit custom response headers (istio#4015)
306287418 ssl: make Ssl::Connection const everywhere (istio#4179)
706e26238 Handle validation of non expiring tokens in jwt_authn filter (istio#4007)
f06e9588a fuzz: tag trivial fuzzers with no_fuzz. (istio#4156)
27fb1d353 thrift_proxy: add service name matching to router implementation (istio#4130)
8c189a552 Make over provisioning factor configurable (istio#4003)
6c08fb43c Making test less flaky (istio#4149)
592775b7b fuzz: bare bones HCM fuzzer. (istio#4118)

For istio#7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
PiotrSikora added a commit to PiotrSikora/istio that referenced this issue Aug 23, 2018
This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/istio/proxy:

c9d2230 Update Envoy SHA to latest with MetricImpl optimizations. (istio#1938)
a32a587 Add a check cache test for string map sub keys (istio#1931)

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68dda stats: refactoring MetricImpl without strings (istio#4190)
36809d80a fuzz: coverage profile generation instructions. (istio#4193)
ba40cc933 upstream: rebuild cluster when health check config is changed (istio#4075)
05c0d52d3 build: use clang-6.0. (istio#4168)
01f403ec4 thrift_proxy: introduce header transport (istio#4082)
564d256fb tcp: allow connection pool callers to store protocol state (istio#4131)
3e1d643b9 configs: match latest API changes (istio#4185)
bc6a10c2f Fix wrong mock function name. (istio#4187)
e994c1c0b Bump yaml-cpp so it builds with Visual Studio 15.8 (istio#4182)
3d1325e89 Converting envoy configs to V2 (istio#2957)
8d0680feb Add timestamp to HealthCheckEvent definition (istio#4119)
497efb95b server: handle non-EnvoyExceptions safely if thrown in constructor. (istio#4173)
6d6fafdb3 config: strengthen validation for gRPC config sources. (istio#4171)
132302caf fuzz: reduce log level when running under fuzz engine. (istio#4161)
7c04ac255 test: fix V6EmptyOptions in coverage with IPv6 enable (istio#4155)
1b2219bd7 ci: remove deprecated bazel --batch option (istio#4166)
2db6a4ce1 ci: update clang to version 6.0 in the Ubuntu build image. (istio#4157)
71152b710 ratelimit: Add ratelimit custom response headers (istio#4015)
306287418 ssl: make Ssl::Connection const everywhere (istio#4179)
706e26238 Handle validation of non expiring tokens in jwt_authn filter (istio#4007)
f06e9588a fuzz: tag trivial fuzzers with no_fuzz. (istio#4156)
27fb1d353 thrift_proxy: add service name matching to router implementation (istio#4130)
8c189a552 Make over provisioning factor configurable (istio#4003)
6c08fb43c Making test less flaky (istio#4149)
592775b7b fuzz: bare bones HCM fuzzer. (istio#4118)

For istio#7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
rshriram pushed a commit that referenced this issue Aug 23, 2018
…). (#8161)

This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/istio/proxy:

2656f34 Update Envoy SHA to latest with MetricImpl optimizations (release-1.0). (#1939)

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68dda stats: refactoring MetricImpl without strings (#4190)
36809d80a fuzz: coverage profile generation instructions. (#4193)
ba40cc933 upstream: rebuild cluster when health check config is changed (#4075)
05c0d52d3 build: use clang-6.0. (#4168)
01f403ec4 thrift_proxy: introduce header transport (#4082)
564d256fb tcp: allow connection pool callers to store protocol state (#4131)
3e1d643b9 configs: match latest API changes (#4185)
bc6a10c2f Fix wrong mock function name. (#4187)
e994c1c0b Bump yaml-cpp so it builds with Visual Studio 15.8 (#4182)
3d1325e89 Converting envoy configs to V2 (#2957)
8d0680feb Add timestamp to HealthCheckEvent definition (#4119)
497efb95b server: handle non-EnvoyExceptions safely if thrown in constructor. (#4173)
6d6fafdb3 config: strengthen validation for gRPC config sources. (#4171)
132302caf fuzz: reduce log level when running under fuzz engine. (#4161)
7c04ac255 test: fix V6EmptyOptions in coverage with IPv6 enable (#4155)
1b2219bd7 ci: remove deprecated bazel --batch option (#4166)
2db6a4ce1 ci: update clang to version 6.0 in the Ubuntu build image. (#4157)
71152b710 ratelimit: Add ratelimit custom response headers (#4015)
306287418 ssl: make Ssl::Connection const everywhere (#4179)
706e26238 Handle validation of non expiring tokens in jwt_authn filter (#4007)
f06e9588a fuzz: tag trivial fuzzers with no_fuzz. (#4156)
27fb1d353 thrift_proxy: add service name matching to router implementation (#4130)
8c189a552 Make over provisioning factor configurable (#4003)
6c08fb43c Making test less flaky (#4149)
592775b7b fuzz: bare bones HCM fuzzer. (#4118)

For #7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
istio-testing pushed a commit that referenced this issue Aug 24, 2018
This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/istio/proxy:

c9d2230 Update Envoy SHA to latest with MetricImpl optimizations. (#1938)
a32a587 Add a check cache test for string map sub keys (#1931)

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68dda stats: refactoring MetricImpl without strings (#4190)
36809d80a fuzz: coverage profile generation instructions. (#4193)
ba40cc933 upstream: rebuild cluster when health check config is changed (#4075)
05c0d52d3 build: use clang-6.0. (#4168)
01f403ec4 thrift_proxy: introduce header transport (#4082)
564d256fb tcp: allow connection pool callers to store protocol state (#4131)
3e1d643b9 configs: match latest API changes (#4185)
bc6a10c2f Fix wrong mock function name. (#4187)
e994c1c0b Bump yaml-cpp so it builds with Visual Studio 15.8 (#4182)
3d1325e89 Converting envoy configs to V2 (#2957)
8d0680feb Add timestamp to HealthCheckEvent definition (#4119)
497efb95b server: handle non-EnvoyExceptions safely if thrown in constructor. (#4173)
6d6fafdb3 config: strengthen validation for gRPC config sources. (#4171)
132302caf fuzz: reduce log level when running under fuzz engine. (#4161)
7c04ac255 test: fix V6EmptyOptions in coverage with IPv6 enable (#4155)
1b2219bd7 ci: remove deprecated bazel --batch option (#4166)
2db6a4ce1 ci: update clang to version 6.0 in the Ubuntu build image. (#4157)
71152b710 ratelimit: Add ratelimit custom response headers (#4015)
306287418 ssl: make Ssl::Connection const everywhere (#4179)
706e26238 Handle validation of non expiring tokens in jwt_authn filter (#4007)
f06e9588a fuzz: tag trivial fuzzers with no_fuzz. (#4156)
27fb1d353 thrift_proxy: add service name matching to router implementation (#4130)
8c189a552 Make over provisioning factor configurable (#4003)
6c08fb43c Making test less flaky (#4149)
592775b7b fuzz: bare bones HCM fuzzer. (#4118)

For #7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
@Lookyan
Copy link

Lookyan commented Sep 17, 2018

Do you plan to resolve this issue? Can we have on demand service discovery in current architecture? Or it's not possible because we need to listen all ip port pairs to receive inbound connections in envoy proxy?

I think it's a show stopper for medium-large clusters to use istio.

@smawson smawson added this to the 1.2 milestone Nov 1, 2018
@duderino duderino assigned duderino and unassigned PiotrSikora Nov 9, 2018
@munrodg
Copy link

munrodg commented Jan 4, 2019

@rshriram we believe your subsetting work should address this issue.

@rshriram
Copy link
Member

rshriram commented Jan 8, 2019

If you manage to run a snapshot of istio 1.1, I think you should find a lower memory usage as we have now trimmed stats [which saves a lot of memory].

@duderino
Copy link
Contributor

We think this is done except for a final measurement/validation which we will track in #10964

istio-testing pushed a commit to istio/proxy that referenced this issue Jan 31, 2019
* update envoy

Signed-off-by: Kuat Yessenov <kuat@google.com>

* update Envoy's SHA to latest

to include the SNI inspecting method, for using SNI in telemetry and
policies

Signed-off-by: Vadim Eisenberg <vadime@il.ibm.com>

* implement destination.principal

Signed-off-by: Kuat Yessenov <kuat@google.com>

* missing test

Signed-off-by: Kuat Yessenov <kuat@google.com>

* review

Signed-off-by: Kuat Yessenov <kuat@google.com>

* add AttributeName::kConnectionRequestedServerName

* fix format

* add GetRequestedServerName() to TCP CheckData

* add building attribute ConnectionRequestedServerName

* test building attribute ConnectionRequestedServerName

* add GetRequestedServerName() to tcp mixer filter

* fix compilation errors

* use explicit conversion from absl::string_view to std::string

* check that the requested server name is not emtpy in attributes builder

* fixed a compilation error

* add GetRequestedServerName to http mixer filters (check_data)

* add GetRequestedServerName to http MockCheckData

* specify the class of a method

* add setting connection.requested_server_name to the http attributes

* qualify Return by testing::

* use connection_ as a pointer

* add explicit conversion from absl::string_view to std::string

* add missing mock call

* Revert "Add connection requested server name attribute to TCP read filter"

* update the API SHA (istio/api#575)

* add the connection.requested_server_name attribute to tcp and http filters

* add missing parentheses

* add AttributeName::kConnectionRequestedServerName

* fix cardinality of a mock method

* fix the signature of TCP Filter::GetRequestedServerName()

* add declarations of GetRequestedServerName() to tcp Filter/http CheckData classes

* add peer and local distinction

Signed-off-by: Kuat Yessenov <kuat@google.com>

* Add connection open event.

* update comment.

* Replace boolean with enum.

* extract origin IP

Signed-off-by: Kuat Yessenov <kuat@google.com>

* Update Envoy SHA to latest. (#1852)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Revert "Update Envoy SHA to latest. (#1852)" (#1854)

This reverts commit f85f49c.

* update api sha (#1855)

* update api sha

* update api sha

* Update Envoy SHA to latest. (#1857)

* Update Envoy SHA to latest.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* review: install pkg-config on CircleCI.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* review: install pkg-config on build image.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Update Envoy SHA for WebSocket fix (#1859)

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* Revert "Update Envoy SHA for WebSocket fix (#1859)" (#1860)

This reverts commit f47be07.

* Update Envoy SHA to latest. (#1862)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Update istio.dep as valid json (#1863)

* Update software in the build image used by CircleCI. (#1874)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Improve build times on CircleCI. (#1875)

1. Stop trashing build's cache.
2. Split ASan and TSan tests into separate targets.
3. Fix output directory, so that caching works on macOS.

Before:
- build     : 49 mins
- macos     : 30 mins

After:
- build     : 27 mins
- linux_asan: 20 mins
- linux_tsan: 16 mins
- macos     : 30 mins

After (with warm cache):
- build     :  3 mins
- linux_asan:  4 mins
- linux_tsan:  2 mins
- macos     :  5 mins

Fixes #1815.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Install ninja in the build image used by CircleCI. (#1888)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* merge 1.0 branch to master (#1885)

* Reduce log level for jwt filter (#1866)

* Update_Dependencies (#1873)

* Correctly clean up headers used for payload from JWT authentication (#1879)

* Correctly clean up headers used for payload from JWT authentication

* Clang

* Update_Dependencies (#1883)

* destination.principal derivation fix (#1884)

* fix attribute extraction

Signed-off-by: Kuat Yessenov <kuat@google.com>

* seed mock

Signed-off-by: Kuat Yessenov <kuat@google.com>

* merge 1.0 to master

* Update API SHA (#1891)

* add needed dependencies for circle ci

* Remove fallback for attribute builder when authn filter is not available (#1887)

* Update Envoy SHA to latest with ORIGINAL_DST_LB fix. (#1894)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Using request dynamic metadata to pass data from JWT filter to authn filter. (#1893)

* Using dynamicMetadata to pass data between filters instead of headers

* Lint

* Populate authn result to dynamic data only.

* Integration test for authn

* Clean up and verify all tests

* Remove unused test configs

* Address reviews

* Lint

* Add the groups claim to the attribute request.auth.groups (#1896)

* Add the groups JWT claims to the attribute request.auth.groups

* Fix lint errors

* Simplify the code

* Fix lint error

* Simplify the code

* Add a test

* Fix the test error

* add context.proxy_error_code to report attributes (#1897)

* Add integration test with JWT+AuthN+Mixer filter chain (#1899)

* Add integration test with JWT+AuthN+Mixer filter chain

* Lint

* Rename helper function

* Lint

* Review

* send rbac shadow policies metrics to mixer (#1900)

* send rbac shadow policies metrics to mixer

* rename shadow -> permissive

* address comments

* address comments

* address comments

* Use dynamic metadata to for authentication filter output. (#1901)

* Use dynamic metadata to for authentication filter output.

* Clean up comments.

* Lint

* Reviews

* Lint

* Fix test

* Remove mis-type

* Lint

* Add two new attributes: request.url_path and request.queries (#1837)

* Add two new attributes:  request.url_path and request.queries

* Update api in repositories.bzl

* update ENVOY_SHA (#1904)

* update ENVOY_SHA

* change WORKSPACE file

* skip empty sni (#1909)

Signed-off-by: Kuat Yessenov <kuat@google.com>

* add rbac filter to istio http integration test. (#1907)

* add rbac filter to istio http integration test.

* rename issueer for rbac test.

* Fix macOS build on CircleCI. (#1916)

Apparently, automake is now installed automatically,
which broke the brew install step.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Install clang-format in the build image used by CircleCI. (#1917)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Provide source version information in the binary. (#1915)

Before:

    $ bazel-bin/src/envoy/envoy --version

    bazel-bin/src/envoy/envoy  version: 0/1.8.0-dev//DEBUG

After:

    $ bazel-bin/src/envoy/envoy --version

    bazel-bin/src/envoy/envoy  version: f315a32fc7c6f727fc9645cc1ca27d4160c1d0e0/1.8.0-dev/Clean/DEBUG

Fixes #1803.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* fixed broken links to dev guide and contribution guide (#1913)

* Update clang to 6.0 and use it for release binaries. (#1914)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Update Envoy SHA to latest with LcTrie optimizations. (#1918)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* add debug logs for collecting rbac attributes (#1922)

* populate source user without authn (#1926)

Signed-off-by: Kuat Yessenov <kuat@google.com>

* update API sha. (#1927)

* Add a check cache test for string map sub keys (#1931)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Update Envoy SHA to latest with MetricImpl optimizations. (#1938)

This is far from finished, but it reduces memory usage by ~10%.

Pulling the following changes from github.com/envoyproxy/envoy:

c1cc68d stats: refactoring MetricImpl without strings (#4190)
36809d8 fuzz: coverage profile generation instructions. (#4193)
ba40cc9 upstream: rebuild cluster when health check config is changed (#4075)
05c0d52 build: use clang-6.0. (#4168)
01f403e thrift_proxy: introduce header transport (#4082)
564d256 tcp: allow connection pool callers to store protocol state (#4131)
3e1d643 configs: match latest API changes (#4185)
bc6a10c Fix wrong mock function name. (#4187)
e994c1c Bump yaml-cpp so it builds with Visual Studio 15.8 (#4182)
3d1325e Converting envoy configs to V2 (#2957)
8d0680f Add timestamp to HealthCheckEvent definition (#4119)
497efb9 server: handle non-EnvoyExceptions safely if thrown in constructor. (#4173)
6d6fafd config: strengthen validation for gRPC config sources. (#4171)
132302c fuzz: reduce log level when running under fuzz engine. (#4161)
7c04ac2 test: fix V6EmptyOptions in coverage with IPv6 enable (#4155)
1b2219b ci: remove deprecated bazel --batch option (#4166)
2db6a4c ci: update clang to version 6.0 in the Ubuntu build image. (#4157)
71152b7 ratelimit: Add ratelimit custom response headers (#4015)
3062874 ssl: make Ssl::Connection const everywhere (#4179)
706e262 Handle validation of non expiring tokens in jwt_authn filter (#4007)
f06e958 fuzz: tag trivial fuzzers with no_fuzz. (#4156)
27fb1d3 thrift_proxy: add service name matching to router implementation (#4130)
8c189a5 Make over provisioning factor configurable (#4003)
6c08fb4 Making test less flaky (#4149)
592775b fuzz: bare bones HCM fuzzer. (#4118)

For istio/istio#7912.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Generate source.namespace in proxy. (#1912)

* Generate source.namespace in proxy.

The next step is to change Mixer to not to generate source.namespace
if it's already in the Check call.

* add comment.

* address comments.

* address comment

* fix format

* simplify the code.

* make check.

* small update

* fix test.

* Authn uses protobuf.Struct to store claims and add list support for RBAC (#1925)

* Authn uses protobuf.Struct to store claims and add list support for RBAC

- Change authn to use protobuf.Struct to store claims
- Add list support for RBAC

* Change based on the review comments

* format it (#1923)

Signed-off-by: Kuat Yessenov <kuat@google.com>

* update envoy SHA (#1941)

* mixer: clear route cache on header update (#1946)

* mixer: clear route cache on header update

Signed-off-by: Kuat Yessenov <kuat@google.com>

* check size

Signed-off-by: Kuat Yessenov <kuat@google.com>

* support per-path JWT validation. (#1928)

* support per-path JWT validation.

* address comment

* address comments

* Fix the peerIsOptional and originIsOptional for authn filter. (#1959)

* Mixer Client uses Node metadata to populate Mixer attributes (#1961)

* update control to include local_info

* WIP

* WIP2

* WIP 3

* WIP5

* WIP6

* WIP6

* WIP8

* WIP8

* WIP8

* WIP10

* WIP11

* WIP12

* Unit tests

* reduce unit test size

* WIP11

* WIP15

* hide unique_ptr and add forward logic to client context

* unique_ptr galore

* fix formatting

* intermediate

* add attribute injection in integration_tests

* format fix

* add correct deps for utils_lib

* add call to attr forwarding

* review comments

* Handle SerializeForwardedAttributes

* more review comments

* fix linter error

* move fn in anonymous ns

* Update Envoy SHA to latest with TCP proxy fixes. (#1964)

Pulling the following changes from github.com/envoyproxy/envoy:

f936fc6 ssl: serialize accesses to SSL socket factory contexts (#4345)
e34dcd6 Fix crash in tcp_proxy (#4323)
ae6a252 router: fix matching when all domains have wildcards (#4326)
aa06142 test: Stop fake_upstream methods from accidentally succeeding (#4232)
5d73187 rbac: update the authenticated.user to a StringMatcher. (#4250)
c6bfc7d time: Event::TimeSystem abstraction to make it feasible to inject time with simulated timers (#4257)
752483e Fixing the fix (#4333)
83487f6 tls: update BoringSSL to ab36a84b (3497). (#4338)
7bc210e test: fixing interactions between waitFor and ignore_spurious_events (#4309)
69474b3 admin: order stats in clusters json admin (#4306)
2d155f9 ppc64le build (#4183)
07efc6d fix static initialization fiasco problem (#4314)
0b7e3b5 test: Remove declared but undefined class methods (#4297)
1485a13 lua: make sure resetting dynamic metadata wrapper when request info is marked dead
d243cd6 test: set to zero when start_time exceeds limit (#4328)
0a1e92a test: fix heap use-after-free in ~IntegrationTestServer. (#4319)
cddc732 CONTRIBUTING: Document 'kick-ci' trick. (#4335)
f13ef24 docs: remove reference to deprecated value field (#4322)
e947a27 router: minor doc fixes in stream idle timeout (#4329)
0c2e998 tcp-proxy: fixing a TCP proxy bug where we attempted to readDisable a closed connection (#4296)
00ffe44 utility: fix strftime overflow handling. (#4321)
af1183c Re-enable TcpProxySslIntegrationTest and make the tests pass again. (#4318)
3553461 fuzz: fix H2 codec fuzzer post #4262. (#4311)
42f6048 Proto string issue fix (#4320)
9c492a0 Support Envoy to fetch secrets using SDS service. (#4256)
a857219 ratelimit: revert `revert rate limit failure mode config` and add tests (#4303)
1d34172 dns: fix exception unsafe behavior in c-ares callbacks. (#4307)
1212423 alts: add gRPC TSI socket (#4153)
f0363ae fuzz: detect client-side resets in H2 codec fuzzer. (#4300)
01aa3f8 test: hopefully deflaking echo integration test (#4304)
1fc0f4b ratelimit: link legacy proto when message is being used (#4308)
aa4481e fix rare List::remove(&target) segfault (#4244)
89e0f23 headers: fixing fast fail of size-validation (#4269)
97eba59 build: bump googletest version. (#4293)
0057e22 fuzz: avoid false positives in HCM fuzzer. (#4262)
9d094e5 Revert ac0bd74 (#4295)
ddb28a4 Add validation context provider (#4264)
3b47cba added histogram latency information to Hystrix dashboard stream (#3986)
cf87d50 docs: update SNI FAQ. (#4285)
f952033 config: fix update empty stat for eds (#4276)
329e591 router: Add ability of custom headers to rely on per-request data (#4219)
68d20b4  thrift: refactor build files and imports (#4271)
5fa8192 access_log: log requested_server_name in tcp proxy (#4144)
fa45bb4 fuzz: libc++ clocks don't like nanos. (#4282)
53f8944 stats: add symbol table for future stat name encoding (#3927)
c987b42 test infra: Remove timeSource() from the ClusterManager api (#4247)
cd171d9 websocket: tunneling websockets (and upgrades in general) over H2 (#4188)
b9dc5d9 router: disallow :path/host rewriting in request_headers_to_add. (#4220)
0c91011 network: skip socket options and source address for UDS client connections (#4252)
da1857d build: fixing a downstream compile error by noting explicit fallthrough (#4265)
9857cfe fuzz: cleanup per-test environment after each fuzz case. (#4253)
52beb06 test: Wrap proto string in std::string before comparison (#4238)
f5e219e extensions/thrift_proxy: Add header matching to thrift router (#4239)
c9ce5d2 fuzz: track read_disable_count bidirectionally in codec_impl_fuzz_test. (#4260)
35103b3 fuzz: use nanoseconds for SystemTime in RequestInfo. (#4255)
ba6ba98 fuzz: make runtime root hermetic in server_fuzz_test. (#4258)
b0a9014 time: Add 'format' test to ensure no one directly instantiates Prod*Time from source. (#4248)
8567460 access_log: support beginning of epoch in START_TIME. (#4254)
28d5f41 proto: unify envoy_proto_library/api_proto_library. (#4233)
f7d3cb6 http: fix allocation bug introduced in #4211. (#4245)

Fixes istio/istio#8310 (once pulled into istio/istio).

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* remove not used path patcher functions (#1966)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Mixer Client: Add support for TCP local attributes (#1967)

* Add support for TCP local attributes

* linter fixes

* review comments

* Fix a bug in origin authenticator that wrongly treats empty origin methods as pass (#1962)

* Fix a bug in origin authenticator that wrongly treats empty origin methods as pass.

* update

* update

* update

* update

* Remove delta update in Report batch (#1970)

* Remove delta update in Report batch

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* fix format

* reduce max_batch_number to 100

* Optimize report sending by re-use protobuf (#1973)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* To share mixer client across listeners (#1972)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Fix a bug of not setting global_word_count (#1974)

* Fix a bug of not setting global_word_count

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* not to save global_dict_size

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Update Envoy SHA. (#1975)

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Optimize mixer client by using protobuf arena allocator. (#1977)

* use arena 1st draft

* use arena for check proto

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* use arena for report proto

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* update api sha

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Update Proxy SHA (#1978)

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Remove obsolete v1 code (#1980)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Update README.md (#1979)

* Not to add check attributes if check is disabled (#1981)

* Not to add check attributes if check is disabled

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Remove ExtractRequestAttributes

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Update Envoy SHA to a637506 (#1982)

* Update Envoy SHA to a637506

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Fix tests based on envoyproxy/envoy#4512

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Fix formatting errors

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Fix CircleCI config

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Revert "To share mixer client across listeners (#1972)" (#1986)

This reverts commit b33ceb2.

* fix memory leak at report batching (#1988)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* reuse report protobuf instead of using arena allocation (#1989)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Update software in the build image used by CircleCI. (#1994)

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Fix attribute constant (#1996)

This commit fixes a misspelling in the attribute constants.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update bazel for circleCI (#1997)

* Update HUB to gcr.io/istio

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Update bazel image for circleCI

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Revert hub change.

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Update dependencies and fix compile errors. (#1993)

* Update dependencies and fix compile errors.

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* update bazel version to 0.17

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* update bazel version to 0.18

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* revert bazel version to 0.15

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Update bazel version for asan and tsan build in circieCI

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* move tool/bazel.rc to .bazelrc (#1998)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* update envoy SHA to point to json access log format functionality (#1999)

* bump Envoy SHA to latest (#2010)

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* Improve performance by removing MD5 for check cache keys (#2002)

* Improve performance by removing MD5 for check cache keys

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* not to allocate memory from stack

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Make debug string readable

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* alts: remove ALTS (#2003)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* Use std::hash for check cache. (#2009)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Send envoy dynamic metadata as part of reports

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* format

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* typos

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* http dynamic metadata

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* format

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* more formatting

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* Use string map instead of opaque struct for dynamic metadata (#2014)

* use string map instead of opaque struct for dynamic metadata

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* format

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* double format

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* skip empty maps

Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>

* Add a new TCP cluster rewrite filter (#2017) (#2018)

This commit adds a new TCP cluster rewrite filter which allows users to
rewrite TCP cluster names obtained via TLS SNI by matching via regex
configuration.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update TCP Cluster Rewrite filter name (#2019) (#2020)

This commit updates the TCP Cluster Rewrite filter name to
envoy.filters.network.tcp_cluster_rewrite.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Enable TCP Cluster Rewrite filter registration (#2021) (#2022)

This commit enables the static registration of the TCP Cluster Rewrite
filter by updating the build configuration.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update Envoy SHA to 4ef8562 (#2023) (#2024)

Envoy /server_info API was inconsistent intermittently causing errors on
a Proxy update on Istio. This update will bring in the API fix to Istio.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update Envoy SHA to 74de08a (#2026)

This commit updates the Envoy SHA to 74de08a
to bring in the new TCP RBAC filter to Istio.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Merge master into release-1.1 (#2031)

* Improve performance by removing MD5 for check cache keys (#2002)

* Improve performance by removing MD5 for check cache keys

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* not to allocate memory from stack

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Make debug string readable

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* alts: remove ALTS (#2003)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* Use std::hash for check cache. (#2009)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Remove tests to compare signature values (#2015)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* update sample envoy config to latest version (#2016)

* Add a new TCP cluster rewrite filter (#2017)

* Add a new TCP cluster rewrite filter

This commit adds a new TCP cluster rewrite filter which allows users to
rewrite TCP cluster names obtained via TLS SNI by matching via regex
configuration.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Make TCP cluster rewrite stackable on SNI filter

This commit updates the TCP Cluster Rewrite filter to be stackable on
the SNI Cluster filter.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update TCP Cluster Rewrite filter name (#2019)

This commit updates the TCP Cluster Rewrite filter name to
envoy.filters.network.tcp_cluster_rewrite.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Enable TCP Cluster Rewrite filter registration (#2021)

This commit enables the static registration of the TCP Cluster Rewrite
filter by updating the build configuration.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update Envoy SHA to 4ef8562 (#2023)

Envoy /server_info API was inconsistent intermittently causing errors on
a Proxy update on Istio. This update will bring in the API fix to Istio.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* add proxy postsubmit periodic (#2025)

* Merge master to release-1.1 (#2038)

* Improve performance by removing MD5 for check cache keys (#2002)

* Improve performance by removing MD5 for check cache keys

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* not to allocate memory from stack

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Make debug string readable

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* alts: remove ALTS (#2003)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* Use std::hash for check cache. (#2009)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Remove tests to compare signature values (#2015)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* update sample envoy config to latest version (#2016)

* Add a new TCP cluster rewrite filter (#2017)

* Add a new TCP cluster rewrite filter

This commit adds a new TCP cluster rewrite filter which allows users to
rewrite TCP cluster names obtained via TLS SNI by matching via regex
configuration.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Make TCP cluster rewrite stackable on SNI filter

This commit updates the TCP Cluster Rewrite filter to be stackable on
the SNI Cluster filter.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update TCP Cluster Rewrite filter name (#2019)

This commit updates the TCP Cluster Rewrite filter name to
envoy.filters.network.tcp_cluster_rewrite.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Enable TCP Cluster Rewrite filter registration (#2021)

This commit enables the static registration of the TCP Cluster Rewrite
filter by updating the build configuration.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update Envoy SHA to 4ef8562 (#2023)

Envoy /server_info API was inconsistent intermittently causing errors on
a Proxy update on Istio. This update will bring in the API fix to Istio.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* add proxy postsubmit periodic (#2025)

* Update Envoy SHA to c41fa71 (#2029)

* Update Envoy SHA

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Fix format.

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* bazel: Allow to distdir all dependencies (#2034)

To use --distdir option of Bazel (which allows to use previously
fetched tarballs instead of downloading dependencies during
build), all dependencies should use http instead of git and need
to have sha256 sums specified.

Signed-off-by: Michal Rostecki <mrostecki@suse.de>

* bazel: Remove BoringSSL repository (#2035)

Pull request #2002 removed signature calculation which was using
BoringSSL as a dependency. BoringSSL is not needed anymore.

Signed-off-by: Michal Rostecki <mrostecki@suse.de>

* Update Envoy SHA to fcc68f1 (#2037)

* Update Envoy SHA to fcc68f1

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Update SHA256

Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>

* Enable caching of dynamic metadata in mixer filter (#2040)

* Enable caching of dynamic metadata in mixer filter

This enables caching of dynamic metadata in the onData call in the mixer
filter and returns the same during the GetDynamicFilterState call.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update cloning to reflect Mongo format updates

This updates the deep cloning logic in the mixer filter to reflect the
latest changes in the Mongo proxy filter's dynamic metadata format.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update api sha to 1a7788d (#2044)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Bump enoy version to pickup json access log fix (#2047)

* Bump enoy version to pickup json access log fix

Signed-off-by: Christopher M. Luciano <cmluciano@us.ibm.com>

* Add sha256sum and note about how to retrieve the sum

Signed-off-by: Christopher M. Luciano <cmluciano@us.ibm.com>

* update envoy sha (#2048)

* update envoy sha

* update envoy SHA to 48b161e (#2041)

* update envoy SHA to 48b161e

* use Http::AsyncClient::RequestOptions() in AsyncClient::send()

* PerConnectionCluster::Key -> PerConnectionCluster::key()

* use Http::AsyncClient::RequestOptions() in AsyncClient::send() in the test

* fix format

* Forward Downstream SNI filter (#2045)

* initial implementation of forward_downstream_api filter

* fix the license message in BUILD (sh comments)

* add missing dependencies

* add definition of config_lib to BUILD

* remove public visibility from forward_downstream_sni_lib

* remove envoy_cc_binary dependency

* StreamInfo::UpstreamServerName -> Network::UpstreamServerName

* fix namespace (Extensions -> Tcp)

* remove config_test

* Add sni verifier filter (#2050)

* Copy the network_level_sni_reader filter

* Add SniVerifier filter based on NetworkLevelSniReader

* Add extra details to log

* refactoring, tests, make buf_ non-static, handle data in chunks

* add credit to TLS inspector of Envoy

* add ERR_clear_error()

* Handle inflight request canceling properly (#2053)

Signed-off-by: Wayne Zhang <qiwzhang@google.com>

* Fix build issue with bazel 0.20 (#2069)

* Clarify log messages (#2066) (#2067)

* Authenticate an exchanged token (#2070)

* Authenticate an exchanged token

* Change issuer name and jwt-authn output with key being original issuer

* Revised the code based on the discussion

* Address review comments and add a test

* Address new review comments

* Add integration tests and address review comments

* Fix a flaky test and address new review comments

* Small grammar fixes

* Revise the function of finding the token header

* Use case-insensitive compare for the header name

* Change the name of a variable

* Revise log statements

* support extract token from customer http header with prefix (#2073)

* support passing jwt token with prefix

* format

* address comments

* address comment  (#2075)

* address comment

* search from start

* update envoy sha (#2074)

* update envoy sha

* library bug fixes

* bug fix

* lint fixes

* lint fixes

* lint fixes

* test fixes

* rename rbac permissive related attributes key due to envoy recent change (#2076)

* rename rbac permissive related attributes key due to envoy recent change

* rename

* Removes dangling pointer in capture list of lambda function. (#2080)

* Replace this with members in capture list.

* update

* fix format

* Use shared_ptr to protect members which are passed to capture list of lambda function. (#2083)

* fix bug

* introduce ControlData

* fix format

* update

* fix format

* Revise

* Pass ControlDataSharedPtr into Control and hold ref

* pass JwtAuthenticationConstSharedPtr into JwtAuthStore to hold ref

* Revise

* Update Envoy SHA to latest with protobuf.Any configs (release-1.1). (#2082)

* Update Envoy SHA to latest with protobuf.Any configs (release-1.1).

For istio/istio#10993.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* review: don't open /dev/stdout in tests.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* Update Istio API to latest (1.1 branch) (#2084)

Signed-off-by: Yangmin Zhu <ymzhu@google.com>

* use route directive regardless of rpc status (#2087)

* use route directive regardless of rpc status

Signed-off-by: Kuat Yessenov <kuat@google.com>

* log response code

Signed-off-by: Kuat Yessenov <kuat@google.com>

* Upgrade Envoy to b3be571 (#2091)

* Upgrade Envoy to b3be571

This updates Envoy SHA to b3be571.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Update tls_utility paths

This updates the tls_utility library and include paths.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Fix http mixer filter

This adds the encodeMetadata implementation to the http mixer filter to
fix a build issue.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Fix sha256sum overwrite error

This removes the ln command from the CircleCI configuration to fix a
file overwrite error on macOS jobs.

Signed-off-by: Venil Noronha <veniln@vmware.com>

* Fix build on macOS. (#2090) (#2094)

sha256sum shouldn't be necessary, since we use gsha256sum on macOS.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* add mixer error details into metadata (#2093)

* add mixer error details into metadata

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* format

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* review

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* comment

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* Forwarded attributes override statically configured Local Attributes (#2097)

* WIP

* add local and override tests

* revert attributes_builder

* white list forward attributes

* add tests with whitelist

* fix builder test for white listed attributes

* comment out failed test case
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants