Headers user-agent,... forbidden for GitHub API

[fiftin]

I try to use GitHub API from JSDOM. But I always receive error "Headers user-agent, authentication forbidden" when I set required headers. In browers all works fine.

I have explored a source code of XMLHttpReqest (xmlhttprequest.js and xhr-utils.js). I see preflight request to server but result of it is not used for validation of final resquest's result.

For example:

Client side:

var xmlhttp = new XMLHttpRequest();
var url = 'http://localhost/';
xmlhttp.open('GET', url, true);
xmlhttp.setRequestHeader('User-Agent', 'test');
xmlhttp.send(null);

Server side:

var express = require('express')
  , cors = require('cors')
  , app = express();

app.use(cors());

app.get('/', function (req, res) {
  res.send('Hello World!');
});

app.listen(80, function () {
  console.log(`Example app listening on port 80!`);
});

Result: Error: Headers user-agent forbidden

As I understand header access-control-allow-headers must be forworded from preflight request's response (xhr-utils.js:281) to real request's response validation (xmlhttprequest.js:958).

[domenic]

As far as I can tell this does not work in browsers: e.g. http://jsbin.com/paxoyuzemu/edit?html,console,output says Refused to set unsafe header "User-Agent" in Chrome. That's per spec.

You can adjust the user agent jsdom uses as part of the configuration when initially creating a jsdom, from the outside. But you cannot change the user agent of a single XHR from inside jsdom script.

Closing, but happy to reopen if there's a separate issue here besides user agent.

[BehindTheMath]

@domenic I believe OP's underlying issue is that jsdom's XHR does not set the User-Agent header for preflight requests. Usually, this is not an issue. However, the Github API requires that this header be set, even for preflight requests, and will reject any request without it.

As far as I can tell this does not work in browsers: e.g. http://jsbin.com/paxoyuzemu/edit?html,console,output says Refused to set unsafe header "User-Agent" in Chrome. That's per spec.

It is true that Chrome does not allow scripts to set the User-Agent header. However, that does not mean that it's against the spec.

The part of the spec you linked to defines the CORS-safelisted request-headers, which are just headers that are automatically allowed by CORS, and do not require a preflight request. However, the preflight request itself can include other headers. Chrome itself sends its own User-Agent header with every preflight request.

You can see this in this JSbin example. The request will fail, since Github's CORS doesn't allow that custom header. But if you open DevTools and look at the preflight request, you will see that Chrome included a User-Agent header, as well as some others not on the safe-list.

In fact, the spec for CORS-preflight fetch, in step 5, uses HTTP-network-or-cache fetch to actually send the preflight request. Step 11 there adds the User-Agent header, as well as some others in steps 9-14.

In summary: the spec does not forbid setting the User-Agent header, and on the contrary, it requires it. Browsers have added an extra security measure by disallowing it to be set by scripts, and instead it's controlled by the browser.

You can adjust the user agent jsdom uses as part of the configuration when initially creating a jsdom, from the outside. But you cannot change the user agent of a single XHR from inside jsdom script.

This is a valid decision, which mirrors the restriction put in place by the other browsers.

However, the underlying issue here is that the User-Agent header is not set at all for the preflight request, even by jsdom itself. This could be easily solved by setting the User-Agent header to whatever the internal User Agent is set to.

I've submitted a simple PR (#2103) to fix this.