osmium merge doesn't get rid of duplicate IDs

[nilsnolde]

I'm merging the PBFs of 2 adjacent areas where I expect some overlaps of OSM objects, i.e. some objects being present in both files. According to the (almost recent) documentation:

Objects that appear in multiple input files will only be in the output once.

However, when I run a osmium extract on the resulting PBF, it complains about duplicate node IDs:

Node ID twice in input. Maybe you are using a history or change file?
This command expects the input file to be ordered: First nodes in order of ID,
then ways in order of ID, then relations in order of ID.

Here are some minimal files from Austria (it's custom OSM data produced with osmium's python bindings), test_at<x>_sorted.pbf are the ones being used to merge to test.osm.pbf:
test_duplicate_ids.zip

The osmium extract cmd was:
osmium extract --bbox 15.44404670,47.73762607,15.58269764,47.78722480 test.osm.pbf -o test_cut.osm.pbf

Is it a possible regression in current master (built yesterday) or I shouldn't expect this to work?

[joto]

You have multiple versions of the same object in your input files. For instance node 10400012005579 is twice in test_at1_sorted.pbf . (osmium fileinfo -e will warn you in that case.)

So your input files are not correct, then the output file wont either.

[nilsnolde]

Oh no.. yes, I now realize what's wrong in my processing chain.. I suspect every intersection is duplicated looking at my code..

Thanks so much for the (as always) super quick and helpful answer. And sorry for the unnecessary hassle. Is there any osmium magic how to identify duplicate IDs with the same version in a file (i.e. how could you tell me that one duplicate ID)?

[joto]

osmium cat test_at1_sorted.pbf -f opl | cut -d' ' -f1 | uniq -c | grep -v ' 1 '