fix(cluster): add https req timeout & show time left waiting for healthz

[metral]

Proposed changes

fix(cluster): add https req timeout & show time left waiting for healthz

Related issues (optional)

Fixes #423

[metral]

Tested using the following setup with the client machine not going through a bastion or in the same VPC as the cluster.

const vpc = new awsx.ec2.Vpc("myvpc", {
    tags: { "Name": "myvpc" },
});

const cluster2 = new eks.Cluster("mycluster", {
    vpcId: vpc.id,
    publicSubnetIds: vpc.publicSubnetIds,
    endpointPrivateAccess: true,
    endpointPublicAccess: false,
});

See video demo below that shows new request timeout and updated information (the video is sped up mid-way for demo purposes). When the update does time out and eventually error, as would be expected in this setup w/o a bastion in use, it does so shortly after the 5 min endpoint test window, versus the 15+min that it's currently taking.

https://drive.google.com/file/d/1nerx0eBXMCi1dWrI-gUimmO6nRg-WVRM/view

cc @clstokes ^

[lblackstone]

Looks good other than the UX comment

[metral]

@clstokes any feedback?

[clstokes]

Does `reqTimeoutMilliseconds = 1000;` mean it will timeout after 1 second? Is this _too_ low of a value? How does this behave when deploying from the U.S. to a region on the other side of the world (or vice versa)? Otherwise, looks good to me.

…

On Thu, Aug 20, 2020 at 12:08 Levi Blackstone ***@***.***> wrote: ***@***.**** approved this pull request. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#427 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACHXA7NBH6BXXOK5HKRVWDSBVYB7ANCNFSM4QEIOAOA> .

[metral]

Does reqTimeoutMilliseconds = 1000; mean it will timeout after 1 second?

yes, 1 second timeout for the HTTP request itself, and 5 seconds in between retries.

How does this behave when deploying from the
U.S. to a region on the other side of the world (or vice versa)?

Ran tests without endpointP*Access set for each region below from a client in us-east-2 (Ohio) going over the public Internet.

• Brazil - endpoint ready on first attempt for 2 runs
• Frankfurt - endpoint ready on first attempt for 2 runs
• Ireland - endpoint ready on first attempt for first run, and after 10 sec for second run
• Oregon - endpoint ready on after ~1min on two runs, and ready after 10 sec on third run
• Ohio - endpoint ready on first attempt for 2 runs

There doesn't seem to be any issue or delays with the endpoint check for local or int'l regions.

If the cluster's are inaccessible e.g. no bastion with private access on the cluster, they'd timeout and quit shortly after the 5 min. window.