Compute separate concurrency_index on osm_route_member #1135
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Results evaluating commit 08817f6 (merged with base 3f70b87 as be646ef). See run details. PostgreSQL DB size in MB: 2785 ⇒ 2788 (0.1% change)
expand for details... |
|
Thanks, looks good! |
TomPohys
approved these changes
Jul 2, 2021
|
@TomPohys i cant build anymore after that PR. I get the error |
farfromrefug
added a commit
to Akylas/openmaptiles
that referenced
this pull request
Jul 17, 2021
…aptiles#1135)" This reverts commit 1b0b1fd. # Conflicts: # layers/transportation_name/update_transportation_name.sql
|
Can you provide steps to reproduce? I've built many times with this change in place. |
|
I building for france and get it on every run. Though i know think it might be because i use a newer version of imposm. I will check |
|
I use this fork of imposm3 https://github.com/farfromrefug/imposm3 but it should not make a difference. Could you test with "Lyon bbbike" to see if you see the error too? It is small and broke here. |
|
I'm currently building a France mbtiles (using just a stock The section of SQL that the error message is referring to is this: What that code does is to add a column to This syntax is supposed to be valid as of postgres 9.5. Since it works on the dockerized openmaptiles postgres but fails on yours, I suspect the different may be with the version of postgres in use? The postgres in the dockerized container in the distro is reported as: Can you check on which version of postgres you're running? |
|
Here's the quickstart.log from a load of France using the current OpenMapTiles master. |
|
Thanks a lot for looking deeply at this. I ll report tonight whrn in front of my computer |
|
@ZeLonewolf sorry for the late answer. Looking at my openmaptiles_postgres_1 container the version is 9.6.21-1.pgdg90+1> |
|
@farfromrefug there's no obvious reason why that shouldn't work. What happens when you log into the container with psql and run the SQL fragment in #1135 (comment) ? |
|
@ZeLonewolf i ll test. My build for france is still running. It takes likes 4 days on my machine :P May i ask what computer you ran it on to be so fast? |
|
Just my day-to-day laptop with SSD hard drive: |
TomPohys
pushed a commit
that referenced
this pull request
Jul 26, 2021
Fixes #1128 This PR adds 6 new columns to the `transportation_name` column, named `route_1` through `route_6`. These columns contain route information for a section of roadway. The value is stored in the form **network=ref**. For example, Interstate 95 in the United States would be `US:I=95`. Thus, each `route_N` value contains enough information to render a highway shield. Since a section of road can be a part of more than one route, the `route_2`, `route_3`, etc, will contain the 2nd, 3rd, etc., concurrent routes. The technical approach was to extend the change in #1135, which added ordered concurrency indexes to the `osm_route_member` table by joining up to the the first six entries to the `osm_transportation_name_network` table. In addition, that PR provided a ranking system for concurrent highways, ordering first by `network_type` (for example, `us-interstate`, `us-state`, etc), then alphabetically by network name, and then by ref in ascending order. This ordering of concurrent route memberships is now exposed in this PR in the sequential `route_N` values, meaning that rendered concurrent highway shields will be reasonably sorted. The renderings below were generated using this branch of OpenMapTiles, as well as a separate branch of openstreetmap-americana: https://github.com/ZeLonewolf/openstreetmap-americana/tree/openmaptilers-new-features-test The rendering approach is to use the [formatted expressions](https://maplibre.org/maplibre-gl-js-docs/style-spec/expressions/) feature in mapLibre to insert images into a string of text. Blank shields are added to the sprite sheet for all possible route networks. Next, a [styleimagemissing](https://maplibre.org/maplibre-gl-js-docs/api/map/) callback is registered. As each shield ID is requested, the callback retrieves the sprite shield blank associated with the route's network, draw the `ref` text on the shield, and insert the complete shield back into the map. Of note, this approach currently results in shields which are rotated about the road rather than being viewport aligned. This issue is currently documented as maplibre/maplibre-gl-js#188. A separate repository (https://github.com/ZeLonewolf/maplibre-shield-rotation-sample) has been created as a test case to fix this rotation issue. Adding route concurrency information to OpenMapTiles would be a major step forward in achieving comprehensive highway shield renderings in a vector map! **Renderings**:    
Hi ! I'm having the same problem loading the whole world. It worked on proposed fragment do raise the error, however if i add INSERT INTO osm_route_member (id, osm_id, concurrency_index)
SELECT
id, osm_id,
ROW_NUMBER() over (PARTITION BY member ORDER BY network_type, network, LENGTH(ref), ref) AS concurrency_index
FROM osm_route_member
ON CONFLICT (id, osm_id) DO UPDATE SET concurrency_index = EXCLUDED.concurrency_index; |
|
@Cactusbone great catch - that index mismatch is definitely a problem. I'll work on a fix if you don't beat me to it! |
|
So, investigating this further, in my database, the primary key is the The proposed fix above gives me the error: I am not sure why your table has a primary key on both fields, while my table has a primary key on id alone, but it appears to be the route of the problem. The |
|
@ZeLonewolf indeed i use my own fork of imposm https://github.com/farfromrefug/imposm3/commits/master |
so yeah, something's wrong here, I'll try to narrow it down |
|
This commit looks like a good culprit, however it's been around since imposm 0.8.0 I'm using the latest openmaptiles-tools from docker hub, I'm not sure which imposm is shipped with the 5.3 version OK, it seems the problem stems from there, the 5.3 and 5.3.2 ships with a custom fork from v2017-10-18, see here bottom line: to make this work, ensure you use openmaptiles-tools 5.3 during all steps, and not the latest image. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #1134
Adds a
concurrency_indexcolumn toosm_route_memberas described in #1134. For example, listing concurrencies on a segment of the Washington Bridge in Providence, Rhode Island, USA: