Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change area decisions at import time #4032

Merged
merged 32 commits into from
Feb 20, 2020
Merged

Change area decisions at import time #4032

merged 32 commits into from
Feb 20, 2020

Conversation

pnorman
Copy link
Collaborator

@pnorman pnorman commented Feb 13, 2020

Fixes #3611

jeisenbe and others added 30 commits May 20, 2019 12:41
Adding these two features to the local polygon values list in openstreetmap-carto.lua will allow these features to be imported as polygons so that they will render properly
Import railway=station as a polygon
Add tourism=yes as a linestring exception
Import "allotments" as a polygon key
…ings (#3973)

* Add power=cable, man_made=cutline and man_made=pipeline as a linestring only features in the lua file
- These features will now be imported only as linestrings, not as polygons, when tagged on a closed way.
Import closed ways taggedwith aerialway=station as polygons for proper rendering
Import aerialway=station as a polygon feature
Add the key `craft=` to the list of keys which are imported as polygons when mapped as closed ways
Add waterway=tidal_channel to the list of linestring exceptions in openstreetmap-carto.lua so that closed ways with this tag will only be imported as a linestring
Import the key `craft=` as a polygon
A tag like building=no shouldn't cause the object to be considered an area.
Import natural=earth_bank as a linestring
Import closed ways with the key `club=` as polygons
Add golf key to list of polygon keys in openstreetmap-carto.lua so that closed ways with this key will be imported as polygons, except for golf=hole, golf=path and golf=cartpath which are always mapped as lines
Don't form areas from tags like building=no
Add boundary= aboriginal_lands, national_park, and protected_area to polygon keys
pnorman and others added 2 commits February 11, 2020 21:37
Pull request #3727 introduced the new condition osm_id > 0 but did
not update the condition for the indexes inteded to be used by the SQL
queries of these layers.
@pnorman
Copy link
Collaborator Author

pnorman commented Feb 14, 2020

Originally I was wondering if this needed a benchmarking run, but looking at the diff I don't see anything with performance implications, either at import time or at run time.

@pnorman pnorman marked this pull request as ready for review February 14, 2020 04:43
@pnorman pnorman added the lua label Feb 14, 2020
Copy link
Collaborator

@jeisenbe jeisenbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested these changes. While my system does not have sufficient RAM to import the whole planet, I tried importing the 153 MB Oregon extract (153 MB pbf file "oregon.osm.pbf", downloaded December 16th), 5 times with the current master branch and 5 times with schema_changes.

With schema_changes, the import took 685s, 647s, 669s, 684s, and 671s:
Range 647 to 685
Mean 671 (and median 671)

On master, the import took 665s, 576s, 554s, 607s, and 644s:
Range 554 to 665
Mean 609, median 607

So the range of times overlaps, but on average the new schema is about 10% slower, which is reasonable and usually will not be noticed.

@pnorman
Copy link
Collaborator Author

pnorman commented Feb 16, 2020

I wasn't expecting that much of a performance difference, so I'm running a benchmark myself

@pnorman
Copy link
Collaborator Author

pnorman commented Feb 17, 2020

I don't find a noticeable performance difference on Geofabrik's Canada extract with --slim --drop --flat-nodes.

@jeisenbe
Copy link
Collaborator

That's great. Is there documentation about how to use these options (--slim --drop --flat-nodes ) when importing with docker?

@pnorman
Copy link
Collaborator Author

pnorman commented Feb 17, 2020

Is there documentation about how to use these options (--slim --drop --flat-nodes ) when importing with docker?

I don't use docker, so don't know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Different geometry decisions (point/line/polygon) on database re-import
4 participants