tree: http: try GET on failed HEAD

[pared]

• ❗ I have followed the Contributing to DVC checklist.

• 📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏
Fixes #4131

[pmrowla]

Should we maybe cache this (per-run) so that we remember what hosts reject HEAD but accept GET? It would keep us from making a lot of unnecessary failed requests when we have to download a large dataset .

[pared]

@pmrowla, I am not sure we want to do that. From our point of view, URL host might be completely different than the one actually forbidding the HEAD request.
Example from the original issue:
dvc get-url https://github.com/explosion/spacy-models/releases/download/ro_core_news_sm-2.3.1/ro_core_news_sm-2.3.1.tar.gz
From our point of view, it will be github.com that forbids HEAD. Inspecting the response shows that it is actually
github-production-release-asset-2e65be.s3.amazonaws.com that forbids that. And its fine, we can inspect the response and cache the github-production-release-asset-2e65be.s3.amazonaws.com as the reason, but the problem is that next time we call even the same link, we still need to make HEAD request (because github.com is not cached) just to know that it actually redirects to github-production-release-asset-2e65be.s3.amazonaws.com and that we should use GET instead of HEAD. We end up still doing 2 requests.