Fetching with Authorization Header

[JCSanPedro]

From what I understand, we can fetch files from a private github repository with the following:

dhall <<< 'https://api.github.com/repos/:org/:repo/contents/:path using toMap { Authorization = "token ${env:GITHUB_TOKEN as Text }" }'

but I'm getting a 403 error. I'm not sure what I'm doing wrong here, or what else I can do to debug this further. I have also tried

dhall <<< 'https://api.github.com/repos/:org/:repo/contents/:path using [ { mapKey = "Authorization", mapValue = "token ${env:GITHUB_TOKEN as Text }" } ]'

Additionally, the following curl command does work

curl -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/:org/:repo/contents/:path

The dhall version is 1.25.0.

Any help would be appreciated, thanks.

[Gabriella439]

@JCSanPedro: Using a fake API token, could you provide the output of this command:

$ dhall <<< 'https://httpbin.org/headers using toMap { Authorization = "token ${env:FAKE_TOKEN as Text}" } as Text'

I haven't been able to reproduce the problem locally, but the output from that might help narrow down the problem.

[JCSanPedro]

Thanks for the response.

$ dhall <<< 'https://httpbin.org/headers using toMap { Authorization = "token ${env:FAKE_TOKEN as Text}" } as Text'                                                                                  [13:28:10]
''
{
  "headers": {
    "Accept-Encoding": "gzip",
    "Authorization": "token fakeToken",
    "Host": "httpbin.org"
  }
}
''

[Gabriella439]

@JCSanPedro: Does your GITHUB_TOKEN environment variable contain any whitespace or newlines?

Also, can you paste the error message that you are getting (with any sensitive information redacted)?

My intuition here is that there might be something wrong with the API URL that you are using, but without knowing the path I can't say for certain.

Other than that, the only thing I can think of is to forward your request through a local proxy that logs requests so that you can verify that the header and path look correct.

[JCSanPedro]

The problem was that the github api requires a valid user-agent header, otherwise it will return a 403.

Not an issue with dhall itself, so I'll close this now, but would it be possible if it returned more than just the status code? Currently it returns

dhall <<< 'https://api.github.com/repos/:org/:repo/contents/:path using
        [ { mapKey   = "Authorization"
          , mapValue = "token ${env:GITHUB_TOKEN as Text}"
          }
        , { mapKey   = "Accept"
          , mapValue = "application/vnd.github.v3.raw"
          }
        ]'

Error: Unexpected HTTP status code:

↳ 403

and the actual error message is

'Request forbidden by administrative rules. Please make sure your request has a User-Agent header (http://developer.github.com/v3/#user-agent-required). Check https://developer.github.com for other possible causes.\r\n'

I initially just assumed I was setting the authorization header incorrectly, until I followed your advice about using a proxy.

Cheers!

[Gabriella439]

I'll re-open this to improve the error message, but relocate this ticket to the dhall-haskell project since it's an issue specific to the Haskell implementation

[Gabriella439]

The fix is up here: #1253