Delta/incremental updates / diffs for map data

[jpds]

It would be neat if OM could support static deltas for updated map data, similiar to another tool that I use: https://ostree.readthedocs.io/en/stable/manual/formats/#static-deltas

If someone could point me to where the map data is generated and packaged - I could try to implement this.

[biodranik]

bsdiff already provides 5-30% of the original size and is already implemented, need only to properly enable it.

Efficient diffing deltas implementation requires detailed knowledge of the current format. And the efficiency depends on the size of the mwm sections. For example, there are several feature indexes sections that probably won't compress efficiently.

[pastk]

Related to #835

[jwmh]

Tip:
Find a rural campground.
Add a couple missing details from its official source (e.g. Payments Accepted; Reservations; etc).
Check the file size of the update needed in OG immediately after.

[jwmh]

Any progress on this issue?
It occurred to me:
Q:
What about making separate, partial Deltas?

1. Tags delta
2. Nodes delta
3. Ways delta
   and having each of these as a separate “Update_Maps-Download” option
   ?
   Also just the tags — or tags+nodes — could maybe speed up server processing too
   ?
   (and save costs)

A lot of folks maybe just want a specific area or use-case—
— especially on limited — or sloooooww — data connection,

don’t necessarily want nor need the whole multi-county region.
—> particularly not all of the diff types of updates.

Hikers may want Ways — and maybe also Nodes — and probably Tags.

City folk — or ppl who mainly use this in an urban setting— may more use it (like myself!!) to check for Nodes+Tags more than anything.

And I don’t really care about the Nodes getting updated;
more so the Tags.

So
Trying to make a case here in favor of Partial-Deltas divided up by category :-)

Benefits of this approach:

• reduce time and money for everyone (DevOps, servers, network, and users)
• • save our OG’s ISP a lot of money.

[biodranik]

Our data format and approach to storing map data are not as simple as tags/ways/nodes. Good news is that using the same mechanism as Google Chrome uses for diffs/patches works well enough.

[newsch]

bsdiff already provides 5-30% of the original size and is already implemented, need only to properly enable it.

Is bsdiff enabled now?

[biodranik]

It is present in the client's code. Diffs can be built on the server (a bit of scripting is needed). The missing part is a simple and scalable server-side API to let clients know that there are diffs available for their maps version, with a proper redirection. And a bit of client code to restore/activate from maps.me times, to properly apply the diff.

For example, a web server can send a json autoindex of files in it.

This task will unblock automatic/more frequent map data updates.