Expose grpc keepalive params in config

[d80tb7]

Expose GRPC keepalive parameters via Armada configuration (definitions can be found here). The motivation here is to allow users to tweak these parameters based on their particular Armada deployment (e.g. to increase the ping frequency for quicker notification of transport failures at the expense of server load).

• ServerParameters.MaxConnectionIdle
• ServerParameters.MaxConnectionAge
• ServerParameters.MaxConnectionAgeGrace
• ServerParameters.Time
• ServerParameters.Timeout
• EnforcementPolicy.MinTime
• EnforcementPolicy.PermitWithoutStream

I've added default values to config/armada/config.yaml, config/lookout/config.yaml and config/binoculars/config.yaml[1] so that all values will be defaulted to the values that were set in previous releases.

[1] I've omitted ServerParameters.MaxConnectionAge and ServerParameters.MaxConnectionAgeGrace from the default values as these would need to be set to either int64.maxValue or 0s to maintain the current behaviour, which I think is a bit cryptic. Instead by leaving them unset they will default to 0, which the grpc lib will treat as unset and apply the default value of infinity.

[kannon92]

Looks good to me! Thanks for adding this.

Have you looked at how these options match to clients? I didn't see a direct 1to1 from go to python. It may be fun trying to match options with the different clients.

[d80tb7]

@kannon92

So I think the main one we care about here from the client's point of view is EnforcementPolicy.MinTime. Basically, if a client tries to heartbeat more frequently than this the server will start rejecting it. I think the relevant python client channel setting is grpc.keepalive_time_ms.