Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import routing data only #915

Closed
zedxxx opened this issue Sep 21, 2020 · 8 comments
Closed

Import routing data only #915

zedxxx opened this issue Sep 21, 2020 · 8 comments

Comments

@zedxxx
Copy link

zedxxx commented Sep 21, 2020

I want to use only routing functionality in my application. Using Import tool I can convert small *.pbf files fast enough, but with large files (whole Europe, for example) it take ages. So my question is: Is it possible to skip unnecessary steps (if any) in Import process (one of this 27-28 steps) to speed up data importing?

@Karry
Copy link
Collaborator

Karry commented Sep 22, 2020

Hi.

You can remove some data types from map.ost to make import faster. You may removes almost all types that are marked with keyword PATH (some other types are required by code import code itself, but I don't know exact list).

Another way howto speedup import is running just selected steps. On this diagram https://osmscout.karry.cz/europe/czech-republic-21-20200922-1128/stats.svg you may track which temporary files and steps are required for "RouteDataGenerator". But I think that you cannot skip something big...

@zedxxx
Copy link
Author

zedxxx commented Sep 22, 2020

Can you explain what is it map.ost and how can I remove something from it? I use *.osm.pbf from here http://download.geofabrik.de/europe.html

@Karry
Copy link
Collaborator

Karry commented Sep 22, 2020

During import, you have to provide --typefile <somepath>/map.ost argument to Import tool. That map.ost file is placed in stylesheets/map.ost in repository. It is text file with definition of types that are extracted from OSM data file and used in database. When you are interested just for router feature, you don't need to import areas, nodes and even ways that are not used later for generating router data files...

Syntax of that file is described here: http://libosmscout.sourceforge.net/documentation/typedef/

@Framstag
Copy link
Owner

If you want to leave out steps, you can use -s or -e to limit the number of steps run. Note though, that for a correct database most steps have a dependency on earlier steps. This is the reason, why it is safer to keep all steps but reduce the list of imported data,

Note also that it is better, to set individual types to "IGNORE" instead of removing it. Since some objects could map to multiple types, removing some may result in wrong mapping to other types. IGNORE removes matching objects without importing them, prohibiting later matches,

@zedxxx
Copy link
Author

zedxxx commented Sep 23, 2020

Ok, I got it. I will try to edit the map.ost by adding IGNORE and report the results.

@zedxxx
Copy link
Author

zedxxx commented Sep 25, 2020

My stylesheet optimized for routing only: route.ost

Import is ~3x faster with this file.

@zedxxx zedxxx closed this as completed Sep 25, 2020
@zedxxx
Copy link
Author

zedxxx commented Sep 25, 2020

Also, steps 18-21/22 can be safely skipped. If you need street names for your route, than don't skip step 22 (location.idx generation).

@Framstag
Copy link
Owner

I'm glad to hear things are working out for you. Note that there a number of optimizations options for import and at runtime. If you have further questions, just ask...!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants