Allow lazy connection on client

[danielhochman]

Is your feature request related to a problem? Please describe.
The ability to lazy connect to the server was removed in v1.14.0 with the addition of the getCapabilities call on client creation.

Ideally the SDK would still offer the option to connect lazily. Fail fast in a critical application with multiple concerns (i.e. Temporal access plus at least one other concern) is a non-starter. It can block and bring an entire service down during a deploy, scale up, or other lifecycle event even if only a small portion of the service is not operational.

Describe the solution you'd like
Each call on the client can check whether or not the connection has been created, and use a mutex to lock and create it before allowing any calls to proceed.

Describe alternatives you've considered
One option would be to duplicate the entire interface in our application to wrap every call with the solution described above. From a maintenance perspective it's not great. Would prefer to have this option in the upstream client.

A client factory can help, but the factory has side-effects in that case. This is an anti-pattern, so additional warnings are then needed to prevent a developer from calling the factory somewhere it could block (e.g. in the instantiation of a module during dependency injection or similar). To better mitigate the problem an application can implement a factory that creates the config and then an additional GetConnection() call off of the returned struct. This signals to the developer that the call will connect to the server and should not be used during application start.

Additional context
See this thread for more discussion: https://temporalio.slack.com/archives/CTRCR8RBP/p1645050691089599

[cretz]

After some discussion internally and across languages, it looks like we may end up moving to this model.