-
Notifications
You must be signed in to change notification settings - Fork 673
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
de-singleton tile extract in graphreader #3281
Conversation
e656389
to
7bcc7b9
Compare
…er for graphreader
this is as good as done @kevinkreiser , just a tiny pointer issue in graphreader.cc, maybe you can suggest the right way?
|
…ve different tar file sizes with predicted traffic
…so we don't get a race condition with the test which actually creates the tar
I removed setting the |
@nilsnolde ill review this today, might be tonight by the time i have time but ill get to it. thanks for hanging in there! |
ok i made a couple requests for minor changes otherwise looks good! |
thanks @kevinkreiser for final review. I changed all the things you requested:
finally done 🥳 next up: integrate the ability to reload tiles to python bindings (what else?) |
fixes #3117
I started with this on the side, working on it here and there. to re-iterate: we want to be able to load other tile extracts while a consumer application is running (e.g.
valhalla_service
). in #3117 @kevinkreiser proposed to use a fixed-width index file storing the tile_id and byte offset in the final tar file which the graphreader can read first (if available) to really fast initialize the tile_extract_t (faster because it does not have to scan through the entire tar file thanks to the index file).tasks:
this (currently) does not work for existing extracts, so far it only builds an extract from-> not necessary IMOmjolnir.tile_dir
. is that fine with you? we could also add that ability and it sorta was what @kevinkreiser initially suggestedwhere to add this functionality: python bindings for sure IMO, what about-> subsequent PRsvalhalla_service
?@kevinkreiser I didn't do much yet on the c++ side, will comment a bit on the files, a few pointers would be cool:)
example commandline:
python scripts/valhalla_build_extract --config site/valhalla.json