Keep connection to remote builder alive to improve remote build latency

[roberth]

Is your feature request related to a problem? Please describe.

Currently when doing remote builds, I see a lot of connecting to <hostname>, which can be painfully slow when the remote is in another network, or when the builds are serialized, such as when doing IFD. Deep changes to config files, such as a small configuration change in a NixOS generation may also be particularly affected by this latency, unless we use or advance knowledge about the dependency graph in that case (don't think we do, but also not sure).

Describe the solution you'd like

Have a connection pool for each of the builders.

Describe alternatives you've considered

Use an SSH master connection instead of a pool. I don't think we should force use of this feature though; it is more fragile and it complicates the user's configurations.

Additional context

Priorities

Add 👍 to issues you find important.