Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add manifest property for Envoy proxy HTTP/2 #593

Merged
merged 1 commit into from
Sep 17, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions docs/envoy-proxy-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ This document describes how to enable the per-container [Envoy proxy](https://gi
1. [Envoy Proxy Configuration for Route Integrity](#envoy-proxy-configuration-for-route-integrity)
1. [Additional Per-Instance Memory Allocation](#additional-per-instance-memory-allocation)
1. [Choosing a value for the additional memory allocation](#choosing-value-for-additional-memory-allocation)
1. [HTTP/2 Support via ALPN](#http2)
1. [Enabling Mutual TLS Configuration](#enabling-mutual-tls-configuration)
1. [Disabling Unproxied Port Mappings](#disabling-unproxied-port-mappings)

Expand Down Expand Up @@ -80,7 +81,13 @@ Container memory usage metrics sent through the Loggregator system and exposed o

The Diego team has in [story #155945585](https://www.pivotaltracker.com/story/show/155945585) done some investigation of how the Envoy proxy uses memory in practice, and as of Envoy version `0e1d66377d9bf8b8304b65df56a4c88fc01e87e8` has determined that Envoy initially uses between 5 and 10 MB of memory, and then uses approximately 30KB of memory per concurrent connection. The memory usage also remains at that level even if the number of concurrent connections decreases. Consequently, if operators have an estimate of `N` for the maximum number of concurrent connections from the gorouters to a single app instance, this assessment suggests that the `containers.proxy.additional_memory_allocation_mb` property should be set to the value `10 + 0.03 * N` (rounded to the nearest integer). This additional allocation may of course need to be adjusted according to the specifics of the applications running in each environment.

### <a name="enabling-mutual-tls-configuration"/> Enabling Mutual TLS Configuration
## <a name="http2"/> HTTP/2 Support via ALPN

By default, the Envoy proxy will advertise support for both HTTP/1.1 and HTTP/2
using ALPN (Application-Layer Protocol Negotiation). To disable ALPN (and
HTTP/2), configure `containers.proxy.enable_http2` to be `false`.

## <a name="enabling-mutual-tls-configuration"/> Enabling Mutual TLS Configuration

A deployment operator can enable mutual TLS configuration between the Envoy proxy which runs in the application container and the Gorouter by performing the following steps:

Expand All @@ -90,7 +97,7 @@ A deployment operator can enable mutual TLS configuration between the Envoy prox
1. In the `rep` job, also set the value of `containers.proxy.trusted_ca_certificates` to the CA certificate created in the first step.
1. Optionally, you can configure the Envoy proxy to validate the subject alternative name on the certificate provided by the gorouter. To do so, the certificate template needs to contain the subject alternative name, and that same name can be set in `containers.proxy.verify_subject_alt_name` in the `rep` job.

### <a name="disabling-unproxied-port-mappings"/> Disabling Unproxied Port Mappings
## <a name="disabling-unproxied-port-mappings"/> Disabling Unproxied Port Mappings

A deployment operator can also disable the legacy port mappings that bypass the Envoy proxy by setting the `containers.proxy.enable_unproxied_port_mappings` property on the `rep` job to `false`. Setting this value requires the Envoy proxies to be enabled.

Expand Down
3 changes: 3 additions & 0 deletions jobs/rep/spec
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,9 @@ properties:
containers.proxy.additional_memory_allocation_mb:
description: "Additional memory allocated to each container for the envoy proxy. This value must not be negative"
default: 32
containers.proxy.enable_http2:
description: "Whether envoy proxy advertises HTTP/2 support via ALPN."
default: true

containers.proxy.require_and_verify_client_certificates:
default: false
Expand Down
1 change: 1 addition & 0 deletions jobs/rep/templates/rep.json.erb
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
advertise_preference_for_instance_address: p("diego.rep.advertise_preference_for_instance_address"),
enable_unproxied_port_mappings: p("containers.proxy.enable_unproxied_port_mappings"),
proxy_memory_allocation_mb: p("containers.proxy.additional_memory_allocation_mb"),
proxy_enable_http2: p("containers.proxy.enable_http2"),
container_proxy_path: "/var/vcap/packages/proxy",
container_proxy_config_path: "/var/vcap/data/rep/shared/garden/proxy_config",
evacuation_polling_interval: "#{p("diego.rep.evacuation_polling_interval_in_seconds")}s",
Expand Down
3 changes: 3 additions & 0 deletions jobs/rep_windows/spec
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,9 @@ properties:
containers.proxy.additional_memory_allocation_mb:
description: "EXPERIMENTAL: Additional memory allocated to each container for the envoy proxy. This must not be negative. Currently doesn't work on windows cells but left here for compatability with the linux Rep"
default: 32
containers.proxy.enable_http2:
description: "EXPERIMENTAL: Whether envoy proxy advertises HTTP/2 support via ALPN. Currently doesn't work on windows cells but left here for compatability with the linux Rep"
default: true

containers.proxy.require_and_verify_client_certificates:
default: false
Expand Down
1 change: 1 addition & 0 deletions jobs/rep_windows/templates/rep.json.erb
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
advertise_preference_for_instance_address: p("diego.rep.advertise_preference_for_instance_address"),
enable_unproxied_port_mappings: p("containers.proxy.enable_unproxied_port_mappings"),
proxy_memory_allocation_mb: p("containers.proxy.additional_memory_allocation_mb"),
proxy_enable_http2: p("containers.proxy.enable_http2"),
container_proxy_path: "/var/vcap/packages/envoy_windows",
container_proxy_config_path: "/var/vcap/data/rep/shared/garden/proxy_config",
evacuation_polling_interval: "#{p("diego.rep.evacuation_polling_interval_in_seconds")}s",
Expand Down