etl/mapping graph tools build all layers at once #147
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nyurik
changed the title
nyurik
changed the title
nyurik
force-pushed
the
multi-etl
branch
4 times, most recently
from
8098528
to
43d0386
Compare
nyurik
added a commit
to nyurik/openmaptiles
that referenced
this pull request
Dec 20, 2019
Requries openmaptiles/openmaptiles-tools#147 quicker and cleaner diagram image generation. Remove etl-graph and mapping-graph targets - redundant Several images have been updated due to a more inclusive mapping scan
nyurik
added a commit
to nyurik/openmaptiles
that referenced
this pull request
Dec 20, 2019
Requires openmaptiles/openmaptiles-tools#147 quicker and cleaner diagram image generation. Remove etl-graph and mapping-graph targets - redundant Several images have been updated due to a more inclusive mapping scan
nyurik
added a commit
to nyurik/openmaptiles
that referenced
this pull request
Jan 19, 2020
Requires openmaptiles/openmaptiles-tools#147 quicker and cleaner diagram image generation. Remove etl-graph and mapping-graph targets - redundant Several images have been updated due to a more inclusive mapping scan
eva-j
approved these changes
Jan 20, 2020
MartinMikita
approved these changes
Jan 21, 2020
TomPohys
approved these changes
Jan 21, 2020
generate-etlgraph and generate-mapping-graph can now automatically determine if the file is a tileset or a layer yaml, and if tileset, will generate all needed image files for all layers. This makes usage of this utility much faster for the whole tileset. Also both utilities can do binary comparisons with the existing files, and raise an error if differ. Minor breaking change: .dot files are not created by default unless --keep and the .dot files no longer have extension. No .svg is generated by default. Extra: added unit tests for mapping graph.
nyurik
added a commit
to nyurik/openmaptiles
that referenced
this pull request
Jan 21, 2020
Requires openmaptiles/openmaptiles-tools#147 quicker and cleaner diagram image generation. Remove etl-graph and mapping-graph targets - redundant Several images have been updated due to a more inclusive mapping scan
nyurik
added a commit
to nyurik/openmaptiles
that referenced
this pull request
Jan 22, 2020
Requires openmaptiles/openmaptiles-tools#147 quicker and cleaner diagram image generation. Remove etl-graph and mapping-graph targets - redundant Several images have been updated due to a more inclusive mapping scan
nyurik
added a commit
to openmaptiles/openmaptiles
that referenced
this pull request
Jan 23, 2020
quicker and cleaner diagram image generation. Remove etl-graph and mapping-graph targets - redundant Also, the obsolete "fields" is still in Imposm's code and both names are accepted, but "fields" is not documented anywhere (PR submitted), and could be removed at any moment. Our docs were not supporting it until this PR, so renaming it at the same time. Several images have been updated due to a more inclusive mapping scan Requires openmaptiles/openmaptiles-tools#147 (merged)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
generate-etlgraphandgenerate-mapping-graphcan now automatically determine if the fileis a tileset or a layer yaml, and if tileset, will generate all needed image files for all layers.
This makes usage of this utility much faster for the whole tileset.
One unintended consequence is that it found that we have been using obsolete
fieldsinstead ofcolumnsin the imposm3 files everywhere except in https://github.com/openmaptiles/openmaptiles/blob/f3667170f68c4c3574a1c5bfd2d7830781ca8c4f/layers/transportation/mapping.yaml#L376 (transportation). Will submit a PR to change all fields.