proxy issues (X-Forwarded-For header related)

[KeithWoods]

Hi

Firstly, thanks for all the hard work, much appreciated :)

I've setup sinopia on a linux box which can only access a single site (http://registry.npmjs.org) via it's proxy.

On this box when I have http_proxy set in my profile. I can curl http://registry.npmjs.org. However when I spin up sinopia I get 407s trying to issues requests to the upstream repo.

I tried setting http_proxy in various places, .bashrc (gets loaded from .bash_profile), .npmrc, within config.yaml nothing worked.

I dug into the src to find sinopia is using request to issue the request (see here).

I did some hacking to see what gets set here as the request is created, when wrote a little spike to see what's the issue with the usage of request. I found that request itself honors http_proxy set in my profile. In the spike below, example 1 (how sinopia makes the request) doesn't work (status = 407), but example 2 does (status = 200, same as example 1 but without X-Forwarded-For).

// this assumes that http_proxy is set somewhere, i.e. in .bashrc or .bash_profile depending upon how you're running it
var request = require('request');

console.log('################### example 1 (similar to sinopia)  '); 
request({
    url      : 'http://registry.npmjs.org',
    method   : 'GET',
    headers  : {
        "Accept": "application/json",
        "Accept-Encoding": "gzip",
        "User-Agent": "Sinopia/1.3.0",
        "X-Forwarded-For": "REMOVED: The Ip of my local workstation that makes the request to sinopia",
        "Via": "1.1 c135f09bf68c (Sinopia)",
        "host": "registry.npmjs.org"
    }
  }, function (error, response, body) {
    console.log("STATUS ######################################: " + response.statusCode);
    console.log(body)   
    console.log(error)  
});

console.log('################### example 2 (similar to sinopia but without x-forward-for) '); 
request({
    url      : 'http://registry.npmjs.org',
    method   : 'GET',
    headers  : {
       "Accept": "application/json",
       "Accept-Encoding": "gzip",
       "User-Agent": "Sinopia/1.3.0",
       // "X-Forwarded-For": "REMOVED: The Ip of my local workstation that makes the request to sinopia",
       "Via": "1.1 c135f09bf68c (Sinopia)",
       "host": "registry.npmjs.org"
    }
  }, function (error, response, body) {
    console.log("STATUS ######################################: " + response.statusCode);
    console.log(body)   
    console.log(error)  
});

At this point I can get the whole thing working by commenting out these lines, but don't fully understand why.

X-Forwarded-For is the IP address of my local work station, and from what I read on X-Forwarded-For it seems sensible. Perhaps this is just an edge case our organisation proxy doesn't like?

Any ideas?

[rlidwka]

No idea. You can however use curl to check, just to make sure it's header that's causing trouble:

$ curl -H 'X-Forwarded-For: 1.2.3.4' https://registry.npmjs.org/

PS: hmm... this reminds me of an issue #19 from 1.5 years ago we never managed to solve. I wonder, what proxy server (as in software version) does your organization use?

[KeithWoods]

It does appear to be the header.

This works:
curl http://registry.npmjs.org/

This fails
curl -H 'X-Forwarded-For: 1.2.3.4' http://registry.npmjs.org/

I read issue #19 before I cracked open the source :)

I'm not sure what proxy software is being used (nor it's config) but I'll dig about and see if I can find out.

Thanks for coming back.

[KeithWoods]

Our infrastructure guys haven't come back to me on this. It's not a sinopia issue so I'm closing it.

Thanks for the help.

[rlidwka]

I conditionally disabled X-Forwarded-For in 0d266be if it is behind a proxy. I hope it'll help other people to avoid similar issues in the future, and that header is not that important anyway.