Supporting Input decompression

[mmd-osm]

TODO once #152 is done:

Users with poor internet connectivity frequently experience issues when uploading changes. Apart from the performance issue on rails (not discussed here), there’s also the issue that the upload itself may be slow. Iirc we don’t support gzip compressed http post requests yet. Could we enable this feature so that Josm or iD could optionally compress an Osm change file before sending it to the server?

This has been enabled on the Rails port, but is not yet avaiable on cgimap.

See discussion on openstreetmap/operations#193

[pnorman]

I suspect the gains for this are low. Is this something that's better done with Apache?

[mmd-osm]

I suspect the gains for this are low

I'd say this highly depends on your internet connection speed. On 100MBit/s upload, there's virtually no difference. For much slower uploads (not too uncommon in many parts of the world, or on mobile links), upload compression will typically reduce the XML data volume by factor 7-9. In addition, our upload message always transmits entire objects, which adds to the overall data volume.

For someone waiting around 20s just for the upload part of the processing, reducing that down to 3s would be a very welcomed improvement.

Is this something that's better done with Apache?

In theory yes. However, I tried with mod_deflate and SetInputFilter DEFLATE and got very mixed results. Decompression seems to work, but HTTP headers don't correctly reflect the decompressed payload, which completely screwed up cgimap. I've seen some other forum posts where people tried this via Apache and finally gave up and implemented it in Rails (=on the application layer).

You've probably noticed that I proposed to use Apache for this purpose in openstreetmap/operations#193, and @tomhughes implemented it in Rails instead. I guess there was a good reason for doing so.

[pnorman]

I'd say this highly depends on your internet connection speed. On 100MBit/s upload, there's virtually no difference. For much slower uploads (not too uncommon in many parts of the world, or on mobile links), upload compression will typically reduce the XML data volume by factor 7-9. In addition, our upload message always transmits entire objects, which adds to the overall data volume.

For someone waiting around 20s just for the upload part of the processing, reducing that down to 3s would be a very welcomed improvement.

What's the median upload size like?
My point isn't that I don't think there will be gains, it's that the payload for upload is small compared to everything else downloaded in a typical mapping session where there's the data download, imagery and/or gpx, geolocated photos, etc.

[mmd-osm]

Upload and download speeds can differ quite significantly, see https://en.wikipedia.org/wiki/Asymmetric_digital_subscriber_line#ADSL_standards as an example. I.e. in general, data volume and speed on the download side are probably not a good reference point to assess the expected upload speed. In addition, we're already using compression on the download side, while on the upload side, we transfer uncompressed data on a potentially much slower connection.