Creation of duplicate nodes

[davidoskky]

I don't know exactly what goes on with it, but it appears to create duplicate nodes at times.
Look at these two changesets: in the second one, several months afterwards, the waste basket was added again in the same spot; and I did not do it. 129200972, 138303681
Same happened with this: 138293099, 138303681

I would never have realized if it was not for this osmose correction: 138497435

[davidoskky]

Slightly worrying: this is definitely a problem: this travel agency has been added at least 4 times...

https://www.openstreetmap.org/node/11016233705
137962381
138025719
138511312

[deevroman]

Perhaps this is a faulty one #540

I see in https://osmcha.org/changesets/138511312 objects created on 2023-03-23. upd: even there is 2022-11-21

Try clearing downloaded objects in the settings. Fix #540 It was a month ago

[Mannivu]

I've encountered this same issue yesterday, but in this changeset I didn't move any node, I just wanted to create a gate and the app created two. Nevermind, I was using Vespucci.

[deevroman]

created_by... Vespucci 19.0.3.0 ?

[Mannivu]

Ups, I was totally sure I used EveryDoor

[davidoskky]

@deevroman I have cleared the downloaded objects and the cache; but in the settings it still displayed an uploading queue with a lot of old edits I had made.
I manually deleted all of those from the queue and I have used EveryDoor to map a little bit afterwards.
I'll later have to check if any duplicate node is still being added to the map or if this problem was solved.

If this problem is solved, it may still be a good idea to let people know it may happen and to clear the upload queue after updating to #540.

[deevroman]

but in the settings it still displayed an uploading queue with a lot of old edits I had made.

This is interesting. So you sent edits, but they still weren't all sent? If so, it looks like a new problem

[davidoskky]

The edits had all been sent already; among the things left to upload were the waste basket and the travel agency I previously mentioned.
The app considered those as not uploaded, and I imagine it would have uploaded them again the next time I made an edit.

[mnalis]

Could you provide a log from menu / About Every Door 3.1 / System Log / send? (mention this issue ID) after this duplicate upload happens (would be good to intentionally upload duplicates twice in a row).

I guess it might prove informative

[davidoskky]

As far as I can see, no duplicate nodes have been created since I cleared the upload queue. I am thus unable to reproduce the problem anymore.
All items in the upload queue were things that had been uploaded previously.

[pkoby]

Can confirm. I mapped from a fresh slate, deleted downloaded data beforehand. I added some POIs earlier today, uploaded. Went out tonight, mapped new things, uploaded, then noticed duplicates of all the things I had mapped earlier.

Now I check the upload queue, and there are things in there from two weeks ago saying Pending. I checked it, and there are three of it now (since deleted).

I did use the app to delete the duplicates, and it seems to have successfully done that and removed them from the queue, but it looks like new additions stick around.

[davidoskky]

@pkoby Are you able to submit a log reproducing the error before clearing the upload queue? I am unfortunately unable to reproduce the problem.

[pkoby]

Oops, I already cleared the queue. I have been trying to duplicate the issue, or at least I'm keeping track of what things I'm doing in the app to try to troubleshoot if it happens again, but so far no luck(?).

So far, events that haven't triggered it are:

• Adding a POI, closing app, reopening, then upload
• Adding a POI, moving a bunch, then upload
• Adding a POI, leaving it for a bunch of time, then upload
• Adding a POI, clearing the downloaded data, redownload, then upload

[Zverik]

Thanks for investigating this, guys. I've added a "help wanted" label so people could notice this issue and maybe look for duplicates themselves.

[deevroman]

I think I have a lead.

Let's start with @pkoby's edits

It can be seen that the edits resulted from a mechanism for splitting large edits into parts.
A bit later there are more edits that arose in this way and led to duplicates, but let's move on to @davidoskky edits

It seems to be a matter of splitting edits here too, but there's more weirdness:

If it’s clear from the last empty edit: the connection is probably broken, then where the edits come from without comments is unclear

What can be done:

• understand in the code why empty edits occur
• add a warning that there are unsubmitted edits. This would be a useful message to the user
• It would be useful to add a tag to edit changesets created by splitting edits that would inform other mappers of this.

[davidoskky]

As a note, I remember that sometimes I would open the app and it would tell me there were changes to upload before I made any change.

[deevroman]

It looks like the problem is different: https://t.me/everydoor_ru/5383

OsmApiHelper: Failed to delete node/ 11067320110: Precondition failed: Node 11067320110 is still used by ways 164039158..

It seems that when deleting points contained in other lines, the edit packet remains unsent. However, I was not able to reproduce this.

The splitting of the edit packages seems to be due to the fact that one of the edits was deleting objects. By the way, in this case there are also empty edits

[mnalis]

• here is new reproducible example: Items stuck in "pending uploads" create duplicates on OSM (v4.0, November 2023) #676

perhaps user can be instructed to gather logs?

[deevroman]

perhaps user can be instructed to gather logs?

An osc file with the changes that EveryDoor is trying to submit would probably be helpful.

[literannn]

one more case:

https://www.openstreetmap.org/changeset/144179402
https://www.openstreetmap.org/changeset/144376523

(duplicated these rock and clock, heh)

[deevroman]

How to reproduce. (for the tests I rebuilt EveryDoor and used the sandboxed API):

1. Download a zone in EveryDoor that has a POI that is not tied to a way (in my case it was a tree)

2. Open iD and attach this tree to some line (I attached it to the border of an industrial zone). Send it to the server.

3. In EveryDoor add some objects.

4. Remove the tree

5. Add new objects.

6. Add some objects a few kilometres away from the tree (so that Every Door splits the edits)

7. Send the edits.

8. EveryDoor will show that there are no edits not uploaded (!)

9. Go to the edit queue and find the conflict.
   9.1 If you download .osc, it will show that there is one edit left.

10. Close EveryDoor and unload it from memory.

11. Open it again.

12. The edits are back!
    12.1 The .osc edits are back as well

Video from step 3:

2023-11-25.01.46.15.mp4

---

What's also interesting is that you don't see any warning about broken edits. But if you try to make more edits and preferably within a few kilometres of each other you get:

---

Moreover now the deletion of loaded data is broken (video of another attempt):

Screen_Recording_20231124_234444.mp4