Set --connect-timeout when running curl, to make --retry work, and hopefully fix #1231

[njsmith]

Link: #1231

[njsmith]

Huh apparently the azure failures are unrelated, because nuget is down right now? of course any attempt to debug flaky infrastructure must hit unrelated flaky infrastructure. I don't know why I didn't expect that.

[codecov]

Codecov Report

Merging #1232 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1232   +/-   ##
=======================================
  Coverage   99.57%   99.57%           
=======================================
  Files         106      106           
  Lines       12823    12823           
  Branches      984      984           
=======================================
  Hits        12768    12768           
  Misses         35       35           
  Partials       20       20

[pquentin]

(I edited the title so that we get a link to the issue.)

[njsmith]

Caught a failure:

+ curl -v --retry 5 -o codecov.sh https://codecov.io/bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 35.199.43.247:443...
* TCP_NODELAY set

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:06 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:07 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:08 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:10 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:11 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:12 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:13 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:14 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:16 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:17 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:18 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:19 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:20 --:--:--     0* connect to 35.199.43.247 port 443 failed: Timed out
* Failed to connect to codecov.io port 443: Timed out

  0     0    0     0    0     0      0      0 --:--:--  0:00:21 --:--:--     0
* Closing connection 0
curl: (7) Failed to connect to codecov.io port 443: Timed out

...so that definitely looks like curl is simply ignoring the --retry switch. Weird!

[njsmith]

@pquentin I guess the problem was that GH doesn't parse titles for links. I edited it into the body instead.

[njsmith]

I thought the issue might be an old version of curl, but that failure I pasted above was from a Azure windows machine, and further investigation reveals that the Azure Windows machines have curl 7.65.3, which is even newer than the 7.64.0 I have on my laptop. But my laptop's version of curl provides clear evidence that it's handling retries properly:

~$ curl --connect-timeout 2 --retry 5 -v http://240.0.0.1
* Expire in 0 ms for 6 (transfer 0x55614e3b35c0)
* Expire in 2000 ms for 2 (transfer 0x55614e3b35c0)
*   Trying 240.0.0.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55614e3b35c0)
* Connection timed out after 2001 milliseconds
* Closing connection 0
Warning: Transient problem: timeout Will retry in 1 seconds. 5 retries left.
* Expire in 0 ms for 6 (transfer 0x55614e3b35c0)
* Expire in 2000 ms for 2 (transfer 0x55614e3b35c0)
* Hostname 240.0.0.1 was found in DNS cache
*   Trying 240.0.0.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55614e3b35c0)
* Connection timed out after 2001 milliseconds
* Closing connection 1
Warning: Transient problem: timeout Will retry in 2 seconds. 4 retries left.

...and so on.

Hmm, I did cheat there and use --connect-timeout to speed up the test. Could that be important? [several minutes of staring at an unchanged terminal later] Ah-hah!

~$ curl --retry 5 -v http://240.0.0.1                 
* Expire in 0 ms for 6 (transfer 0x557c953145c0)
*   Trying 240.0.0.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x557c953145c0)
* connect to 240.0.0.1 port 80 failed: Connection timed out
* Failed to connect to 240.0.0.1 port 80: Connection timed out
* Closing connection 0
curl: (7) Failed to connect to 240.0.0.1 port 80: Connection timed out

So apparently curl knows that the connect timed out, but it doesn't count as the connect timing out for retry purposes unless you explicitly set --connect-timeout. Which would make this a bug in curl. Sweet.

curl/curl#4461

[njsmith]

OK, so hopefully this fixes the issue now?