spanner: Programmatic configuration of the channel pool size

[fornwall]

Desired feature

A supported programmatic way to configure the size of the Spanner client gRPC channel pool.

Today the pool is hard-coded to 4 channels, overridable only through the SPANNER_NUM_CHANNELS environment variable.

Why programmic configurability over environment variable

An environment variable is process-global, untyped, invisible in code review, and awkward to set in tests and benchmarks. std::env::set_var is unsafe, which increases the barrier.

Reasons to configure the pool size

• Cold-start latency on short-lived clients. Channels connect lazily (tonic'sConnection::lazy), but next_channel_hint() round-robins per RPC, so sequential RPCs land on different, cold channels. So a short-lived CLI tool doing some queries opens 4 connections instead of 1 (measured against the mock server), paying 4 TCP+TLS handshakes, which can be noticeable for cli tools on developer laptops. num_channels=1 removes this entirely.
  • Overhead can perhaps be mitigated in the pool itself, like preferring a channel with no in-flight RPCs over always advancing the round-robin counter? But maybe not worth it?
• Resource consumption. Because RPCs round-robin across the pool, a client doing sustained work leads to 4 connections open continuously (and similarly, a bursty cli tool leads to 4 short-lived connections), each with server-side state. Small, lightly loaded processes doing sequential work may want 1.
• Setup cost, paid whether or not a channel is ever used. Unlike the connection itself, this is not lazy: build() costs ~2.5ms/channel of pure CPU here, spent rebuilding the root certificate store in make_endpoint's per-channel ClientTlsConfig::new().with_enabled_roots(). The same build over a plaintext endpoint, which skips that branch, is ~1000x cheaper (16µs at 4 channels), confirming where it goes.
  • Relatively low cost, but still measurable. If possible, sharing one TLS config across channels might be good regardless?

Prior art in other Spanner clients

• Java - SpannerOptions.Builder.setNumChannels(int): a plain setter on the options builder for the number of gRPC channels, defaulting to 4.
• Go - option.WithGRPCConnectionPool(int): a generic client option that "creates a pool of gRPC connections that requests will be balanced between", passed to spanner.NewClient; it replaced the now-deprecated ClientConfig.NumChannels field.
• C++ - google::cloud::GrpcNumChannelsOption: a generic gRPC option (default 4 for Spanner) set on the connection options, documented with the rationale that gRPC limits a channel to 100 simultaneous calls, so more channels raise achievable parallelism.

Related PR

#6072 feat(spanner): add with_num_channels setter to the client builder proposed adding the setting to the client builder - this comment motivated opening a feature request:

We are considering further options for setting up the gRPC channel pool, and are therefore not ready to commit to a specific public API for this at the moment. I think that it would be better if you would open a feature request with the specific options / feature that you need for your use case.