feat(mads) add support for HTTP long polling

[austince]

Summary

Adds support for HTTP long polling by making the xds rest server Fetch call block until there is an update, or the operation times out.

Currently, it just uses the configuration to define default timeouts for updates. I've got an open question in envoy
to see if there is a standard way to set this timeout in the REST-JSON spec (see envoyproxy/envoy#16859), but if they
don't get back to us soon we can just add a simple timeout query param or request header.

This will unblock the native Prometheus SD efforts here: prometheus/prometheus#8844

Full changelog

• Implement a blocking Fetch call using context timeouts
• Change config option monitoringAssignmentServer.fetchTimeout to monitoringAssignmentServer.defaultFetchTimeout

Issues resolved

n/a

Documentation

n/a – need to finish the Prometheus SD updates before adding docs.

Testing

• Unit tests
• E2E tests
• Manual testing on Universal
• Manual testing on Kubernetes

[austince]

Once we decide on a way to specify this client-side, all that is needed is to parse it here and pass it along.

[nickolaev]

We will need a line in Upgrade.md for this name change.

[austince]

Let me check, but I don't think this has been included in a release yet. Would we still need one if that is the case?

[nickolaev]

Nope, we won't need it if this is a brand new config.

[austince]

Confirmed this is not in 1.1.x:

kuma/pkg/config/mads/config.go

Lines 19 to 26 in be98608

	// Monitoring Assignment Discovery Service (MADS) server configuration.
	type MonitoringAssignmentServerConfig struct {
	// Port of a gRPC server that serves Monitoring Assignment Discovery Service (MADS).
	GrpcPort uint32 `yaml:"grpcPort" envconfig:"kuma_monitoring_assignment_server_grpc_port"`
	// Interval for re-generating monitoring assignments for clients connected to the Control Plane.
	AssignmentRefreshInterval time.Duration `yaml:"assignmentRefreshInterval" envconfig:"kuma_monitoring_assignment_server_assignment_refresh_interval"`
	}

[nickolaev]

I think we can merge this. One remark about Configuration renaming that should land in Upgrade.md

[austince]

Any thoughts on the timeout parameter @nickolaev / @jakubdyszkiewicz?

Since the envoy issue is still silent, I would think either:

1. A header (x-kuma-fetch-timeout) that holds a GRPC TimeoutValue and TimeoutUnit https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md#requests

2. A query parameter ?fetch-timeout that holds the same

[nickolaev]

Any thoughts on the timeout parameter @nickolaev / @jakubdyszkiewicz?

Since the envoy issue is still silent, I would think either:

1. A header (x-kuma-fetch-timeout) that holds a GRPC TimeoutValue and TimeoutUnit https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md#requests
2. A query parameter ?fetch-timeout that holds the same

Are you referring to #2121 (comment) ?

if yes, I would vote for (2).

[austince]

Are you referring to #2121 (comment) ?

if yes, I would vote for (2).

Yes, exactly. Ok, will merge this now and add this addition ASAP today.