Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reprocessing relations multiple times #423

Closed
pnorman opened this issue Jul 31, 2015 · 0 comments · Fixed by #1996
Closed

Reprocessing relations multiple times #423

pnorman opened this issue Jul 31, 2015 · 0 comments · Fixed by #1996

Comments

@pnorman
Copy link
Collaborator

pnorman commented Jul 31, 2015

#419 (comment) showed that with diff updates, relations can be processed twice, once in parsing and once in pending relations. The output is correct, but doing so twice is wasteful.

Two possible solutions come to mind

  • Whenever a relation is processed in parsing, delete it from the trackers. If you have a relation that is then modified later in the diff, it will be re-added to the tracker and processed in pending.
  • Don't have the output tables process relations that are already flagged as pending. This would never process the same relation twice, but would defer more work to the pending stage, which is believed to be slower.

Bit of a brain-dump from IRC:

my understanding was that in processing when a relation is touched it has each backend table process that relation. depending on tag transforms, some may do nothing with it. then in pending, it does the same. now, if you have a relation modified then later on a way in that relation modified, it will have to do it in pending so maybe its better to instead skip processing anything in processing that is already marked as pending

@joto joto mentioned this issue Sep 2, 2021
joto added a commit to joto/osm2pgsql that referenced this issue Jul 7, 2023
In append mode relations could be processed twice if they are in the
input and also the parent of a node or way in the input. With this
change we are removing all ids of changed relations from the list of
pending relations before processing them.

Fixes osm2pgsql-dev#423.
joto added a commit to joto/osm2pgsql that referenced this issue Jul 7, 2023
In append mode relations could be processed twice if they are in the
input and also the parent of a node or way in the input. With this
change we are removing all ids of changed relations from the list of
pending relations before processing them.

Fixes osm2pgsql-dev#423.
joto added a commit to joto/osm2pgsql that referenced this issue Jul 8, 2023
In append mode relations could be processed twice if they are in the
input and also the parent of a node or way in the input. With this
change we are removing all ids of changed relations from the list of
pending relations before processing them.

Fixes osm2pgsql-dev#423.
joto added a commit to joto/osm2pgsql that referenced this issue Jul 9, 2023
In append mode relations could be processed twice if they are in the
input and also the parent of a node or way in the input. With this
change we are removing all ids of changed relations from the list of
pending relations before processing them.

Fixes osm2pgsql-dev#423.
joto added a commit to joto/osm2pgsql that referenced this issue Jul 10, 2023
In append mode relations could be processed twice if they are in the
input and also the parent of a node or way in the input. With this
change we are removing all ids of changed relations from the list of
pending relations before processing them.

Fixes osm2pgsql-dev#423.
joto added a commit to joto/osm2pgsql that referenced this issue Jul 11, 2023
In append mode relations could be processed twice if they are in the
input and also the parent of a node or way in the input. With this
change we are removing all ids of changed relations from the list of
pending relations before processing them.

Fixes osm2pgsql-dev#423.
joto added a commit to joto/osm2pgsql that referenced this issue Jul 13, 2023
In append mode relations could be processed twice if they are in the
input and also the parent of a node or way in the input. With this
change we are removing all ids of changed relations from the list of
pending relations before processing them.

Fixes osm2pgsql-dev#423.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant