Result achavi edits empty

[Commodoortje]

Discussed the problem in this (OpenStreetMap) forum
https://forum.openstreetmap.org/viewtopic.php?pid=651816#p651816

[mmd-osm]

Can you please provide a concise description of the issue, preferably in English?

[Commodoortje]

Increasingly I see no edits visible through achavi.
An example is this
http://overpass-api.de/achavi/?changeset=49731295

[Commodoortje]

one of the topic reactions:

I've been sitting around doing puzzles with said changeset (via openstreetmap.org).
The problem seems to arise because the points in time of the change are not correct. The changeset 21:32:51 opens and closes one second later (21:32:52). According to the database of OSM have changed the points relevant to 21:32:50, which is one second before the changeset was opened. That would be impossible in my opinion.
This achavi fog changes due to the changes between opening and closing of the changeset is wanted.
I suspect that something goes wrong with the API to upload the changes, but the experts should examine.
By the time manually take slightly wider, the problem is likely to get around.

[Commodoortje]

a second reaction:

It's quite true, still had to manually just tried to take 1 second longer on both sides and the bbox take a bit bigger, but not.
By allowing the filter from time as 21:32:49 in

https://overpass-api.de/api/interpreter?data=[adiff:%222017-06-21T21:32:49Z%22,%222017-06-21T21:32:52Z%22];(node(bbox)(changed);way(bbox)(changed););out%20meta%20geom(bbox);&bbox=6.9002169,52.223864,6.9006125,52.2252645

[mmd-osm]

Thanks a lot for the details.

According to the database of OSM have changed the points relevant to 21:32:50, which is one second before the changeset was opened. That would be impossible in my opinion

I think we've seen this issue before, iirc this was something @tomhughes fixed on the main db (clocks out of sync).

//cc: @tyrasd, @nrenner

[tyrasd]

Yes, here's the corresponding thread on osm-dev: https://lists.openstreetmap.org/pipermail/dev/2016-August/029399.html

[nrenner]

Thanks for investigating and reporting.

Hope this gets resolved by adjusting the clock again.

Mapbox has implemented a better solution for changeset diffs by collecting them from minutely augmented diffs and is also caching the results: Preparing accurate history and caching changesets. Not sure about the future of achavi but I guess it's either integrating that or recommending using Changeset Map instead:

http://osmlab.github.io/changeset-map/#49731295

[mmd-osm]

I still find achavi pretty useful as it doesn't depend on an external cache and serves data back to September 2012. The cache based option you mentioned starts some time in 2017 iirc and for sure lots of data will pile up in the upcoming months and years, reaching TB disk usage quite easily.

Some of the current shortcomings for attic are on Rolands agenda, we'll see how this all works out in the future.

[mmd-osm]

@Commodoortje - bottom line is, that's really an upstream issue which needs to be dealt with on the OSM main instance.

There's really nothing to be done on Overpass API side, that's why I'd propose to close this issue and do a follow up with OSM sysops.

[tomhughes]

Well you've already copied in the OSM sysops. You just haven't explained what the actual problem is - this thread is basically incomprehensible to me.

I did check that all the servers were correctly synced and fix one web server that didn't have ntp running for some reason.

[mmd-osm]

@tomhughes : sure, let's compare the following two links:

• http://www.openstreetmap.org/api/0.6/changeset/49731295/download:

both nodes have timestamp="2017-06-21T21:32:50Z".

• http://www.openstreetmap.org/api/0.6/changeset/49731295

created_at="2017-06-21T21:32:52Z" closed_at="2017-06-21T21:32:52Z"

Wouldn't we expect the changeset to have been created before the node has been created? In this example, the node was created 2 seconds earlier than the changeset.

Maybe this is already fixed in the meantime by adding ntp to that one web server? In that case, no further follow up is needed, of course.

[tomhughes]

Well roughly speaking yes, but as I have explained many times the code is not entirely consistent so some timestamps are assigned on the database server and some on the web server. So there can be minor discrepancies.

If you want to fix the code to avoid that then fine, but it's not a sysadmin issue.

[Commodoortje]

but it's not a sysadmin issue.

It's the first issue I mentioned on github.
my apologies

[nrenner]

@Commodoortje Nothing wrong with your issue, appreciating it.

I did check that all the servers were correctly synced and fix one web server that didn't have ntp running for some reason.

@tomhughes Thanks a lot! We weren't aware that it's already fixed until your comment, since it's not straightforward to check when only a random part of the changesets is affected, hence the misunderstandings.

To confirm, I ran @joto's osm-data-validation tool with changesets-latest.osm.bz2 (170703) and a small history extract baden-wuerttemberg.osh.pbf as a sample [1].

The latest affected changeset in this sample is 49810362 at 2017-06-25T10:00:21Z, so it's indeed fixed for now.

Looking at daily counts [2] of the sample errors I can roughly identify these periods with diverging timestamps since 2016:

• 2016-07-02 - 2016-08-02
• 2016-11-14 - 2016-11-21
• 2016-12-18 - 2017-01-06
• 2017-05-24 - 2017-06-25

[1] check_changeset_timestamps changesets-latest.osm.bz2 baden-wuerttemberg.osh.pbf changesets-errors.opl data-errors.opl
[2] cat changesets-errors.opl | cut -d' ' -f3 | cut -c2-11 | uniq -c

[drolbr]

I keep this as an enhancement. I think this boils down to the request to make Achavi reliably displaying complete changesets.