Incorrect HTTP keep-alive handling when a Proxy-Connection: header is forwarded as JOSM does with old/limited HTTP proxies

[openstreetmap-trac]

Reporter: Fabi2
[Submitted to the original trac issue database at 10.09am, Tuesday, 21st September 2010]

I found this as I tried to trach down an proxy bug in JOSM which is reported here: https://josm.openstreetmap.de/ticket/5404

Since the API always force close the connection after one HTTP-request, it should sent a "Connection: close" header with HTTP/1.1. But this is not the real problem.

The force close of the HTTP connection is not done on the API
close call (e.g. PUT http://api.openstreetmap.org/api/0.6/changeset/xxxxxx/close). When java (JOSM) sends it's non standard Proxy-Connection:-header over a broken proxy, which forwards this header, then the client behind the proxy will wait forever, because the API-Webserver will transform this broken header into a "Connection: keep-alive" and don't close the network connection after a close request.

Maybe this is only a missing close() on the network file descriptor/socket when an API close call is made, i will assume this because no date is returned by the API server.

This bug is really a fun brake with JOSM and it seems as they don't know how to fix this in JOSM, but seems also an API bug.

[openstreetmap-trac]

Author: TomH
[Added to the original trac issue at 6.18pm, Wednesday, 22nd September 2010]

The API web server is apache 2.2 so if you are claiming a bug it's implementation of the HTTP protocol then it's a bug in apache and you will need to report it at httpd.apache.org - there is certainly nothing I can do to help.

[openstreetmap-trac]

Author: Fabi2
[Added to the original trac issue at 3.07pm, Thursday, 23rd September 2010]

Replying to [comment:1 TomH]:

The API web server is apache 2.2 so if you are claiming a bug it's implementation of the HTTP protocol then it's a bug in apache and you will need to report it at httpd.apache.org - there is certainly nothing I can do to help.

This ia an API bug. If have it written in more detail in the last posts of https://josm.openstreetmap.de/ticket/5404 .

As the apache passes the request and and an open socket for the output to the API-CGI the API should the do proper HTTP-request handling, including a "Connection: close" header for HTTP/1.1 and making sure that the API for the close-request really closes the socket after the processing of the HTTP-request is finished.

Apache is right in handling the "Proxy-Connection: keep-alive"-header in the same way as "Connection: keep-alive" for this request, as "Proxy-Connection" is a non standard header which should be a "Connection:" header. See http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/web-proxy-connection-header.html mor explaination. If the outgoing proxy request would be HTTP/1.0 then maybe the apache would do an auto-close on the socket, if it is not handled by the API-CGI, as it should.

At least HTTP/1.0 and thumb older proxies which will only forward all incoming headers, would fail with JOSM (or othe java programs with call OSM-API-close) on the API-close-call, because the API-CGI seems not to close the HTTP-connection. It think you would not do persistent connection handling now, as the API-calls befaore are all non-persistent.

I hope this will help to understand it better now.

[openstreetmap-trac]

Author: TomH
[Added to the original trac issue at 3.16pm, Thursday, 23rd September 2010]

I have no idea what you are trying to say.

What does "As the apache passes the request and and an open socket for the output to the API-CGI" mean exactly?

For the record apache is not passing sockets to anything - the API is not a set of CGI scripts!

Apache is (depending on exactly which API request is being handled) either acting as a reverse proxy in which case it has a separate socket talking to an application server and is forwarding traffic between them; or is using mod_rails aka passenger to process the request.

In either case there is absolutely nothing I can do to control how apache does HTTP handling and what headers it does or does not send and what sockets it does or does not open or close.

If you really think there is a bug here in OUR CODE OR CONFIGURATION rather than in apache or passenger then please provide a clear description of the problem from first principles along with details of how to reproduce it.

What I want to see is something like a precise description of what JOSM sends to the proxy, what the proxy sends to us, what we send back to the proxy, what the proxy sends back to JOSM and how (if at all) you think each of these steps is wrong and what should be happening.

[openstreetmap-trac]

Author: Fabi2
[Added to the original trac issue at 5.50pm, Thursday, 23rd September 2010]

Replying to [comment:3 TomH]:

I have no idea what you are trying to say.

What does "As the apache passes the request and and an open socket for the output to the API-CGI" mean exactly?

For the record apache is not passing sockets to anything - the API is not a set of CGI scripts!

Now had a short look at how the ruby on rails framework work and at first find no real low-level-processing of the HTTP-requests, as e.g. mod_php, which also parses the Reqest but after this it is a better CGI-Interface, where with ruby you only say something like: "output this file template and fill in the variables with this functions".

Apache is (depending on exactly which API request is being handled) either acting as a reverse proxy in which case it has a separate socket talking to an application server and is forwarding traffic between them; or is using mod_rails aka passenger to process the request.

In either case there is absolutely nothing I can do to control how apache does HTTP handling and what headers it does or does not send and what sockets it does or does not open or close.

Yes, you are right as I see no connection handling things in rails.
But I wondered why whith JOSM the changeset create and upload request succeed, but on the close request the connection over for the same changeset get stuck, after the request was sent to the local proxy here. As I don't control this proxy I don't really know what happens on the outside, but I thought it may be a problem by the handling of "Proxy-Connection:" header. Java adds this header by default when using an HTTP-proxy and as the loacal winroute proxy don't understand and/or strips this header, it thought the api (including all it's components) maybe keep the connection open, so i don't get any response for the API-close-request on the inside of the proxy.

What I want to see is something like a precise description of what JOSM sends to the proxy, what the proxy sends to us, what we send back to the proxy, what the proxy sends back to JOSM and how (if at all) you think each of these steps is wrong and what should be happening.

That is why I linked the bug I reported to JOSM, as there a all the details I have observed.
I can't get on or behind the proxy, but I think it first has to be analyzed further, as the api works different from what I thought.

[openstreetmap-trac]

Author: Fabi2
[Added to the original trac issue at 9.38pm, Thursday, 23rd September 2010]

Now testet JOSM with privoxy modified headers (should be had this idea earlier...), but as the result is the same with the correct headers, the bug now seems to be more on the local proxy here, as on the OSM-API, maybe it is in the upstream software, if it is really there.