Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Change creation of tiles.zip and maps.zip to support very large countries #47

Merged
merged 4 commits into from
Oct 22, 2021

Conversation

treee111
Copy link
Owner

@treee111 treee111 commented Oct 19, 2021

This PR…

extracted from #40:

  • Changes how the resulting tiles.zip and maps.zip file is made. Now files are copied from the 'tilestore' to a country-name and country-name-maps folder. This folder is then passed to the compressor instead of building an enormous command line with all file names specified. This would, for large countries like Russia, result in errors because the command line was to long.
  • Added advantage: Optionally you can choose via CLI to keep these folders so you can instantly copy the files to your device or check them in Cruiser.
  • Copies the - currently static numbered - tile-present/version files (created in [FEATURE] Performance improvement for .osm.pbf creation and splitting. Use 'v12' tag (keep) filters #46) to the tiles.zip archive

@treee111 treee111 force-pushed the small-enhancements-osm-maps-processing branch 5 times, most recently from 537f163 to 8c88065 Compare October 20, 2021 20:06
@treee111 treee111 changed the title [FEATURE] Small enhancements of osm maps processing [FEATURE] Change creation of tiles.zip and maps.zip to support very large countries Oct 20, 2021
@treee111 treee111 marked this pull request as ready for review October 20, 2021 20:30
Copy link
Contributor

@Ebe66 Ebe66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I am overlooking it, but I seem to be missing the creation of the .lzma.12 files?

# Create "tile present" file
f = open(outFile + '.lzma.12', 'wb')
f.close()

- 7za / zip commands with other arguments
- filling the "new" country-maps folder
- keep or delete the country/region map folders after compression
- available via CLI argument
- delete & unify comments
- make variables setting less redundant
- introduce types for except
@treee111 treee111 force-pushed the small-enhancements-osm-maps-processing branch from a47235c to b3e1a37 Compare October 22, 2021 13:13
@treee111
Copy link
Owner Author

treee111 commented Oct 22, 2021

Thanks for your feedback!
I've done the .lzma.12 file creation in #49.

After rebasing this branch with develop and resolving the merge conflicts, the .lzma.12 changes are also visible here:

with open(out_file + '.lzma.12', 'wb') as tile_present_file:
tile_present_file.close()
tile_count += 1

PS: What do you think of copying the .lzma.12 files to the archive like suggested by @mweirauch in #51?
I think of leaving the copy-part in the separate PR or including it here.

@Ebe66
Copy link
Contributor

Ebe66 commented Oct 22, 2021

:-) You are already including the .lzma.12 files in the archive! It in this PR at line 542 :-)

@treee111
Copy link
Owner Author

:-) You are already including the .lzma.12 files in the archive! It in this PR at line 542 :-)

You are absolutely right 🥇
I totally missed that :D

Merging this in favor of #51

@treee111 treee111 merged commit 2e20d51 into develop Oct 22, 2021
@treee111 treee111 deleted the small-enhancements-osm-maps-processing branch October 22, 2021 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants