SNOW-739316 Enforce content-length checking in vendored urllib3 #1702
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Notes
This is equivalent to cherry-picking
fd2759aa16b12b33298900c77d29b3813c6582deonto the current vendored version1.26.15. We are merging the cherry pick to unblock release.What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-739316: Snowflake Batch Results (get_result_batches()) #1425
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
urllib3 does not do content-length checking.
result_batch.py does a get request in line 298, which can therefore in case of connection issues return an incomplete response. This is passed directly to an ArrowIterator. The already existing retry mechanism does not catch this as no Exception is generated.
Please write a short description of how your code change solves the related issue.
4. Setting enforce_content_length checks the length of the response and raises an IncompleteRead error, which should already trigger the existing retry mechanism.
This is the same behaviour as in urllib 3 V2.0.
In other words: #1689 is the better fix to this problem. I'm sending this PR to document the issue and maybe to provide a quickfix version for the interested parties.
Also some info: I should add that this is not confirmed to fix the issue yet. It's just my best guess. I also thought about actually not only overriding the default, but also enforcing enforce_content_length to true in the function itself. But I think requests is the only user in this case and it's also vendored and it never sets it explicitly, so I just overrode the default.