Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retries during chunked transfers fail due to loss of chunked flag #1714

Closed
lmvlmv opened this issue Oct 28, 2019 · 1 comment
Closed

Retries during chunked transfers fail due to loss of chunked flag #1714

lmvlmv opened this issue Oct 28, 2019 · 1 comment

Comments

@lmvlmv
Copy link
Contributor

lmvlmv commented Oct 28, 2019

Possibly related to #1352

Using urllib via requests but this doesn't seem to be the cause. Recently, in requests, a fix was implemented to use the innate chunking capability in urllib.

If a custom Retry class is implemented, a chunked transfer is taking place and a retry-able condition occurs during the transfer the chunked flag is dropped when urlopen() is re-called during the retry.

In this case a generator was passed as the data argument and results in the non-chunking version of the code attempting to read it like raw data: TypeError: 'generator' object has no attribute 'len'

@lmvlmv
Copy link
Contributor Author

lmvlmv commented Oct 28, 2019

Prototype fix here https://github.com/lmvlmv/urllib3/tree/chunkfix

Having trouble getting nox to pass tests in my environment before I submit as a PR...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant