[Release] "archery crossbow download-artifacts" raises read timeout error

[asfimport]

I couldn't all artifacts for 6.0.1 RC1...

Downloading release-6.0.1-rc0-0's artifacts.
Destination directory is /home/kou/work/cpp/arrow.kou/packages/release-6.0.1-rc0-0

[  state] Task / Branch                                               Artifacts
-------------------------------------------------------------------------------
[SUCCESS] debian-bookworm-amd64                                uploaded 70 / 70
 └ https://github.com/ursacomputing/crossbow/runs/4111265571?check_suite_focus=true
                        apache-arrow-apt-source_6.0.1-1.debian.tar.xz [     OK]
                                  apache-arrow-apt-source_6.0.1-1.dsc [     OK]
                              apache-arrow-apt-source_6.0.1-1_all.deb [     OK]
                            apache-arrow-apt-source_6.0.1.orig.tar.gz [     OK]
                                   apache-arrow_6.0.1-1.debian.tar.xz [     OK]
                                             apache-arrow_6.0.1-1.dsc [     OK]
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 438, in _error_catcher
    yield
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 519, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/lib/python3.9/http/client.py", line 462, in read
    n = self.readinto(b)
  File "/usr/lib/python3.9/http/client.py", line 506, in readinto
    n = self.fp.readinto(b)
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/models.py", line 753, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 576, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 541, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 443, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='github-releases.githubusercontent.com', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/archery", line 33, in <module>
    sys.exit(load_entry_point('archery', 'console_scripts', 'archery')())
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/kou/work/cpp/arrow.kou/dev/archery/archery/crossbow/cli.py", line 349, in download_artifacts
    report.show(
  File "/home/kou/work/cpp/arrow.kou/dev/archery/archery/crossbow/reports.py", line 132, in show
    asset_callback(task_name, task, asset)
  File "/home/kou/work/cpp/arrow.kou/dev/archery/archery/crossbow/cli.py", line 342, in asset_callback
    asset.download(path)
  File "/usr/local/lib/python3.9/dist-packages/github3/repos/release.py", line 405, in download
    return utils.stream_response_to_file(resp, path)
  File "/usr/local/lib/python3.9/dist-packages/github3/utils.py", line 84, in stream_response_to_file
    for chunk in response.iter_content(chunk_size=512):
  File "/usr/lib/python3/dist-packages/requests/models.py", line 760, in generate
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='github-releases.githubusercontent.com', port=443): Read timed out.

Can we retry on timeout automatically?

Reporter: Kouhei Sutou / @kou
Assignee: Kouhei Sutou / @kou

PRs and other links:

• GitHub Pull Request #12996

_{Note: This issue was originally created as ARROW-14651. Please see the migration documentation for further details.}

[asfimport]

Kouhei Sutou / @kou:
A naive patch:

diff --git a/dev/archery/archery/crossbow/cli.py b/dev/archery/archery/crossbow/cli.py
index 42d6f5c73..b7ee47c6f 100644
--- a/dev/archery/archery/crossbow/cli.py
+++ b/dev/archery/archery/crossbow/cli.py
@@ -339,7 +339,16 @@ def download_artifacts(obj, job_name, target_dir, dry_run, fetch,
             path = target_dir / task_name / asset.name
             path.parent.mkdir(exist_ok=True)
             if not dry_run:
-                asset.download(path)
+                while True:
+                    try:
+                        asset.download(path)
+                    except Exception as error:
+                        print(error)
+                        import time
+                        time.sleep(60)
+                        pass
+                    else:
+                        break
 
     click.echo('Downloading {}\'s artifacts.'.format(job_name))
     click.echo('Destination directory is {}'.format(target_dir))

[asfimport]

Krisztian Szucs / @kszucs:
Issue resolved by pull request 12996
#12996