proxy closing connection before sending all the data

[NachoSoto]

I recently upgraded my backend to node 0.10.4 and node-http-proxy 0.10.1 and some of my services stopped serving static files correctly. After a while poking around, I curled the file locally using http and it was serving it correctly, but then I noticed this when curling behind the proxy:

curl -kv https://a.xxx.com
* About to connect() to a.xxx.com port 443 (#0)
*   Trying IP...
* connected
* Connected to a.xxx.com (IP) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES256-SHA
* Server certificate:
*    subject: OU=Domain Control Validated; OU=PositiveSSL; CN=a.xxx.com
*    start date: 2012-10-30 00:00:00 GMT
*    expire date: 2013-10-30 23:59:59 GMT
*    subjectAltName does not match a.xxx.com
> GET file HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: a.xxx.com
> Accept: */*
>
< HTTP/1.1 200 OK
< server: gunicorn/0.17.2
< date: Thu, 18 Apr 2013 01:59:57 GMT
< connection: close
< last-modified: Wed, 17 Apr 2013 04:24:04 GMT
< content-length: 340416
< content-type: application/javascript
< content-language: en-us
< vary: Accept-Language, Cookie
<
.
trimmed data here (missing the last part of the file)
.
* transfer closed with 176576 bytes remaining to read
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
curl: (18) transfer closed with 176576 bytes remaining to read

This is how I'm using http-proxy:

var options = {
    hostnameOnly: true,
    router: {
        'a.xxx.com': '127.0.0.1:8000',
        'b.xxx.com': '127.0.0.1:9000',
    },
    https: {
        ca: ca,
        key: fs.readFileSync('../ssl/xxx.com.key', 'utf8'),
        cert: fs.readFileSync('../ssl/xxx.com.crt', 'utf8')
    }
};

var proxyServer = httpProxy.createServer(options);
proxyServer.listen(443, function() {
    console.log('Proxy running');
});

Any ideas why this stopped working all of a sudden? Let me know if you need any more details.
Thanks in advance.

[RGBboy]

I'm having the same issue. I investigated further and found that any content over 10901 bytes in length has it's connection closed.

Any ideas?

Node 10.6
http-proxy 0.10.3

[RGBboy]

Ok so I figured out a solution to my problem. I was using a HTTPS to HTTPS proxy with an Apache server. The Apache server had Keep Alive turned off. Turning it on seems to have solved the issue.

[tuomassalo]

I have the same issue. I have a backend Apache server (http) and a frontend https node-http-proxy. The problem happens on about 20 % of the requests. I'm trying to fetch a document of 243543 bytes, but sometimes the (frontend) connection is closed when 14431 bytes are remaining.

This doesn't happen if I change the frontend to http.

Turning on keep-alive in the backend seems to fix it, but only for certain clients. If you run curl in HTTP 1.0 mode (curl -0 ...), it will fail again.

Maybe there's a timing problem somewhere in the http-proxy code, a one that' sonly triggered in a certain keepalive config, and for some reason only when serving https.

How to fix? Any workarounds?

[tuomassalo]

I tried to write a test case on this. It was difficult, since the problem seems to only appear when there's some latency between the client and the proxy server. I didn't have time to find a tool to simulate the latency, so the test doesn't show the problem when executed on one machine.

However, the problem should appear when server.js is started on one host and curl-test.sh on another, maybe in another network.

How to test:

• Clone my fork: https://github.com/tuomassalo/node-http-proxy.git
• Go to test/keepalive/
• run node server.js
• On another machine, modify curl-test.sh and run it. It should print a bunch of OK lines, and then an occasional error with something like * transfer closed with 6568 bytes remaining to read. That's the bug.

UPDATE: I first thought that the problem only appeared with some specific keepalive settings. Now I'm able to reproduce it without curl's -0 and without sending Connection: close in the backend server. Odd, but I don't have more time to look at this now.

UPDATE 2: It seems that the bug only happens on specific versions of eg. curl. This seems like a very hard one. I was able to reproduce on Ubuntu 10.04 and 12.04, but not OSX 10.8.4.

But there seems to be more to it. I created a simple HTTPS server that only sends data (see https://gist.github.com/tuomassalo/6296041), and most versions of curl seem to fail with even that. No http-proxy in that example. I'm running that server at https://212.246.29.149:11443, so you can try eg. this:

while true; do curl -s -k -0 https://212.246.29.149:11443 2>&1 | wc -c; sleep .5; done

My curl prints mostly 243543, but sometimes 229251.

I'm quite confused about this whole thing. Is there a node.js problem? A http-proxy problem?

Oh, btw, I'm running node v0.10.15 on all servers.

[tuomassalo]

@NachoSoto this bug is probably fixed in v0.11.x. See nodejs/node-v0.x-archive#6107. Can you test by running http-proxy on v0.11.x?

[tuomassalo]

This turned out to be a node.js problem, and will be fixed in node.js v0.10.21. See nodejs/node-v0.x-archive#6107.