API Returns 403 only for web UI

[stephen304]

I noticed that the web ui would stop working after a while, and permanently. The API still responds fine, but not when the UI makes the request. Chrome/firefox web tools lets you export a request to a cURL command, and I was able to export a working request (where I pasted in the api url into the address bar), and a broken request coming from the API. These results are replicated using cURL as well.

Chrome

Command from the address bar, working:

curl 'http://127.0.0.1:5001/api/v0/config?arg=Addresses.Gateway&stream-channels=true'
-H 'Accept-Encoding: gzip, deflate, sdch'
-H 'Accept-Language: en-US,en;q=0.8'
-H 'Upgrade-Insecure-Requests: 1'
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36'
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
-H 'Cache-Control: max-age=0'
-H 'Connection: keep-alive' --compressed

Response:

{
  "Key": "Addresses.Gateway",
  "Value": "/ip4/127.0.0.1/tcp/8080"
}%

Command from the web UI, broken:

curl 'http://127.0.0.1:5001/api/v0/config?arg=Addresses.Gateway&stream-channels=true'
-H 'Accept-Encoding: gzip, deflate, sdch'
-H 'Accept-Language: en-US,en;q=0.8'
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36'
-H 'content-type: undefined'
-H 'Accept: */*'
-H 'Referer: http://127.0.0.1:5001/ipfs/QmaaqrHyAQm7gALkRW8DcfGX3u8q9rWKnxEMmf7m9z515w/'
-H 'Connection: keep-alive' --compressed

Response:

403 - Forbidden%

The diff of these 2 requests are as follows (working on the left, web ui on the right):

I found that removing the referrer and adding the Cache-Control: max-age=0 fixed the request:

curl 'http://127.0.0.1:5001/api/v0/config?arg=Addresses.Gateway&stream-channels=true'
-H 'Accept-Encoding: gzip, deflate, sdch'
-H 'Accept-Language: en-US,en;q=0.8'
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36'
-H 'content-type: undefined'
-H 'Accept: */*'
-H 'Cache-Control: max-age=0'
-H 'Connection: keep-alive' --compressed

Result:

{                                                                 
  "Key": "Addresses.Gateway",
  "Value": "/ip4/127.0.0.1/tcp/8080"
}%  

Firefox

The requests coming from firefox are a bit different.

Working, from a request coming from the URL bar:

curl 'http://127.0.0.1:5001/api/v0/config?arg=Addresses.Gateway&stream-channels=true'
-H 'Host: 127.0.0.1:5001'
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:41.0) Gecko/20100101 Firefox/41.0'
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
-H 'Accept-Language: en-US,en;q=0.5' --compressed
-H 'Cookie: __utma=96992031.1150785634.1435613418.1435613418.1435687582.2; __utmz=96992031.1435613418.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'
-H 'Connection: keep-alive'
-H 'Pragma: no-cache'
-H 'Cache-Control: no-cache'

Broken, coming from the web ui:

curl 'http://127.0.0.1:5001/api/v0/config?arg=Addresses.Gateway&stream-channels=true'
-H 'Host: 127.0.0.1:5001'
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:41.0) Gecko/20100101 Firefox/41.0'
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
-H 'Accept-Language: en-US,en;q=0.5' --compressed
-H 'Content-Type: undefined'
-H 'Referer: http://127.0.0.1:5001/ipfs/QmaaqrHyAQm7gALkRW8DcfGX3u8q9rWKnxEMmf7m9z515w/'
-H 'Cookie: __utma=96992031.1150785634.1435613418.1435613418.1435687582.2; __utmz=96992031.1435613418.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'
-H 'Connection: keep-alive'
-H 'Pragma: no-cache'
-H 'Cache-Control: no-cache'

Diff:

P.S. Here's what the web ui looks when the api requests fail. You can see the network inspector open with the xhr filter, showing all the failed requests:

[jbenet]

Hmmmm this is tricky. can others familiar with the web concerns here CR this? i don't have the bandwidth atm. (maybe @diasdavid @cryptix @mappum )

[daviddias]

If I'm understanding it right, it is a CORS issue, localhost:5001 !== 127.0.0.1:5001, we whitelist localhost in the daemon so that the webui can make requests to the local node, but that is the only case, for every other domain/IP addr, it must be passed as a config option (API_ORIGIN env var).

[stephen304]

Hmmm... I tried both 127.0.0.1 and localhost to load the web ui from and neither work at the moment. Sometimes I can coax it to work for a few minutes by aggressively restarting everything.

Note: I was able to successfully use the web ui after running ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '["*"]'

I hope it will stay fixed.

[jbenet]

this may be related to ipfs/kubo#1883

[jbenet]

Sometimes I can coax it to work for a few minutes by aggressively restarting everything.

me too. ok what is going on!?

[dignifiedquire]

It might be a header issue

[jbenet]

ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '["http://localhost:5001"]'

worked for me which implies the default is not being set by the code as it should be: https://github.com/ipfs/go-ipfs/blob/27eece3e81f4e946db4294702c9ac39bd86ccce6/core/corehttp/commands.go#L70

[jbenet]

Has anyone found a fix for this yet? (beyond this workaround).

Need to trace how the logic goes around here: https://github.com/ipfs/go-ipfs/blob/27eece3e81f4e946db4294702c9ac39bd86ccce6/core/corehttp/commands.go#L69

[sebastiang]

after an upgrade to consul v0.6, the web UI no longer works for me either.

[dignifiedquire]

This should be fixed in 0.3.11 :) Please reopen if there are still problems

[hleb-albau]

Using FF, currently have some problem. If I edit request by removing Referer header, all works fine!

[bohendo]

Thank you @hleb-albau 🙏 My IPFS node is behind an nginx proxy so my 403 issues went away by adding a proxy_set_header Referer ""; directive.