Resolve() fails on HEAD request returning 403 forbidden

[amalcgcg]

When retrieving a package using Source == URL, some packages will fail with "Error: download failed [file was truncated]." The root cause is a 403 forbidden returned by the initial HEAD request to get the file size. This in turn happens because some sites, including GitHub's service on AWS, don't allow HEAD requests.

See this same issue in a different software tool at cavaliergopher/grab#43

renv.lock contents to reproduce:

{
  "R": {
    "Version": "3.6.3",
    "Repositories": [
      {
        "Name": "CRAN",
        "URL": "https://cloud.r-project.org"
      }
    ]
  },
  "Packages": {
    "JGR": {
      "Package": "JGR",
      "Version": "1.9-1",
      "Source": "URL",
      "RemoteType": "url",
      "RemoteUrl": "https://github.com/markush81/JGR/releases/download/1.9-1/JGR_1.9-1.tar.gz"
    },
    "JavaGD": {
      "Package": "JavaGD",
      "Version": "0.6-1.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "95c317e22a41a5e64fd9fcbaa898e53a"
    },
    "rJava": {
      "Package": "rJava",
      "Version": "0.9-12",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "21b20702f0e8e171a81a5d9d75f2b2fd"
    },
    "renv": {
      "Package": "renv",
      "Version": "0.9.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "c1a367437d8a8a44bec4b9d4974cb20c"
    }
  }
}

[amalcgcg]

I attempted a fix in #387

The test suite passes on my local system, but for some reason the CI builds are failing. Could you please advise?

[kevinushey]

Thanks -- I'm going to take a closer look at this today.

[amalcgcg]

I verified that this is fixed by da7bffd. Thanks!

[kevinushey]

Great; I'm glad to hear it!

[kevinushey]

Interestingly, looks like GitHub now allows these sorts of HEAD requests:

$ curl -I https://github.com/kevinushey/skeleton/releases/download/v1.0.1/skeleton_1.0.1.tar.gz
HTTP/1.1 302 Found
Date: Mon, 11 Jan 2021 18:26:31 GMT
Content-Type: text/html; charset=utf-8
Server: GitHub.com
Status: 302 Found
Vary: X-PJAX, Accept-Encoding, Accept, X-Requested-With, Accept-Encoding
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/178079294/94a5a980-80a7-11ea-865f-41c436ecbf22?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210111%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210111T182631Z&X-Amz-Expires=300&X-Amz-Signature=3dd9d56f9acdba8fe54d0247c44fcacddb5fd28a8c90541b5524fa97001d0e14&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=178079294&response-content-disposition=attachment%3B%20filename%3Dskeleton_1.0.1.tar.gz&response-content-type=application%2Foctet-stream
Cache-Control: no-cache
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: no-referrer-when-downgrade
expect-ct: max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"
Content-Security-Policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; connect-src 'self' uploads.github.com www.githubstatus.com collector.githubapp.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events wss://alive.github.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com; frame-ancestors 'none'; frame-src render.githubusercontent.com; img-src 'self' data: github.githubassets.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; manifest-src 'self'; media-src 'none'; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/socket-worker-5029ae85.js gist.github.com/socket-worker-5029ae85.js
Set-Cookie: _gh_sess=T4N2x1HxC7AlIWUIXwLULXVsspZyuUifnotQ%2FfLhIb46mphDg1CS8lKvY8kAWOyC%2Bx2xMuXBcj2p0VW9O6M8eBBxH5NCOdjmNq%2FpgDLs9OcXxesG9h2s7PipvuEyiCe3QPy9D0VZ6ZQffEn0nYqp%2B7HhWV1tgIzyWbFv3XnZ5E6RRan0wdTVJu%2FBtZYdNgnVTpJiSXBnFX6pK7ixGnDxmYUL0%2FjdE0SCnW8%2BEuXd47T4XfZU59VAYtPcH3%2Fzxg4z1OEtANHRnpHHZN%2B%2BY%2Fqv5Q%3D%3D--32NwZaB%2F2huT7stl--k%2Bg7G7ll1kzk9kSgJgD8OQ%3D%3D; Path=/; HttpOnly; Secure; SameSite=Lax
Set-Cookie: _octo=GH1.1.284594889.1610389591; Path=/; Domain=github.com; Expires=Tue, 11 Jan 2022 18:26:31 GMT; Secure; SameSite=Lax
Set-Cookie: logged_in=no; Path=/; Domain=github.com; Expires=Tue, 11 Jan 2022 18:26:31 GMT; HttpOnly; Secure; SameSite=Lax
Content-Length: 647
X-GitHub-Request-Id: EFD7:3EAF:0D81:13D7:5FFC9857