pull: treat HTTP redirects without Location header as error #3169
Conversation
fabiosantoscode
changed the title
fabiosantoscode
changed the title
dvc/remote/http.py
Outdated
| if kwargs["allow_redirects"] and res.status_code in (301, 302): | ||
| # AWS s3 doesn't like to add a location header to its redirects | ||
| # from https://s3.amazonaws.com/<bucket name>/* type URLs. | ||
| # This should be treated as an error | ||
| raise requests.exceptions.RequestException |
There was a problem hiding this comment.
What if this is not s3 and location header is present?
There was a problem hiding this comment.
Then requests would've followed the redirect.
There was a problem hiding this comment.
Maybe better to be explicit and check that a location header is not present?
There was a problem hiding this comment.
Yes, that would be better. At least the comment will follow the code, now it looks like a bug in implementation. So adjusting comment would be good too.
There was a problem hiding this comment.
Makes sense
|
@fabiosantoscode For the record, your fix doesn't really address #2963 , because that issue is about Your PR is still useful, but I don't have any way of reproducing the issue that it is solving. But looks alright - so merging. Thanks! 🙂 |
3 participants
The reason for this problem is a 301 without a Location header. In our HTTP remote we can treat this as an error, since we don't know where to actually find the data.
Fixes #2963
❗ Have you followed the guidelines in the Contributing to DVC list?
📖 Check this box if this PR does not require documentation updates, or if it does and you have created a separate PR in dvc.org with such updates (or at least opened an issue about it in that repo). Please link below to your PR (or issue) in the dvc.org repo.
❌ Have you checked DeepSource, CodeClimate, and other sanity checks below? We consider their findings recommendatory and don't expect everything to be addressed. Please review them carefully and fix those that actually improve code or fix bugs.
Thank you for the contribution - we'll try to review it as soon as possible. 🙏