BUGFIX: Fix name-based way fragmentation in transportation_name #1295
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ZeLonewolf
changed the title
|
Results evaluating commit e53122d (merged with base 0cff344 as 2a7201a). See run details. PostgreSQL DB size in MB: 2850 ⇒ 2852 (0.1% change)
expand for details... |
TomPohys
reviewed
Nov 15, 2021
Co-authored-by: Tomas Pohanka <TomPohys@gmail.com>
TomPohys
reviewed
Nov 16, 2021
|
Thank you for this PR. It looks awesome. Just a little comment for clarification if |
Correct name:xx vs name_xx Co-authored-by: Tomas Pohanka <TomPohys@gmail.com>
ZeLonewolf
commented
Nov 24, 2021
|
The unit test CI is actually failing here. There is something wrong with the unit test execution that is causing the failure to not bubble up into CI. |
|
All issues are resolved! The new test cases added in this PR also detected an issue with sac_scale not getting updates in the transportation table sequence; this was corrected in e53122d. Ready for review. |
|
Awesome! Thank you very much! |
TomPohys
approved these changes
Nov 25, 2021
This was referenced Dec 1, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I discovered this bug while investigating issues with the updates process related to #1190 #1292, and #814.
The
transportation_namelayer produces slightly differenttagshstore values in theosm_transportation_name_linestringtable during the initial import versus when running an update. As currently written, the import code produces null-value keys in thetagscolumn, while the update code suppresses them. This PR removes that difference and makes the import code use same method that is currently used in the update code.With a test case I've written, the import code produces a tags hstore that looks like this:
"name"=>"OpenMapTiles Secondary 2", "name:de"=>NULL, "name:en"=>NULL, "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"...while the update code produces a tags hstore that looks like this:
"name"=>"OpenMapTiles Secondary 2", "name_int"=>"OpenMapTiles Secondary 2", "name:latin"=>"OpenMapTiles Secondary 2"Note the missing NULL values.
This bug causes a small amount of space wastage after an update is run, because the update matching code detects the
tagsvalue as different, resulting in a duplicate copy of the tags value if that row is updated. This causes duplicate objects and breaks GROUP BY clauses that expect to group same-tagged features together. I've tested this by inspection of a generated mbtiles, database spot checks, and the unit test code included in this PR.This PR incorporates the changes in #1294, which were necessary for testing this PR.