Skip to content

Downloads during build process can't handle reset connections #434

@jjacobson95

Description

@jjacobson95

In the case that a connection is broken, we may want to consider a retry attempt. This really applies to all downloads across all datasets, but this is an example for broad sanger proteomics

Error:

Traceback (most recent call last):
  File "/app/02a-broad_sanger_proteomics.py", line 80, in <module>
    main()
    ~~~~^^
  File "/app/02a-broad_sanger_proteomics.py", line 58, in main
    r = requests.get(sanger_protfile)
  File "/opt/venv/lib/python3.13/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/opt/venv/lib/python3.13/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/venv/lib/python3.13/site-packages/requests/sessions.py", line 746, in send
    r.content
  File "/opt/venv/lib/python3.13/site-packages/requests/models.py", line 902, in content
    self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
                    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/requests/models.py", line 822, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(10190848 bytes read, 102480742 more expected)', IncompleteRead(10190848 bytes read, 102480742 more expected))

Sub-issues

Metadata

Metadata

Assignees

Labels

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions