Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

speedup building of geometries from osm entities #111

Merged
merged 2 commits into from
Mar 8, 2019

Conversation

tyrasd
Copy link
Member

@tyrasd tyrasd commented Mar 7, 2019

includes two improvements for building geometries:

  • speedup building of complex multipolygons
    Relations with many (small) inner rings used to take a long time to assemble, because each of the expensive within operation in a loop. By using a prepared geometry, this is substantially faster. Eespecially for particularly complex example like relation/571669 this makes a large difference: processing times went down from several minutes to a few seconds.
  • avoid clipping if not necessary
    This applies the idea from improved splitting of "large" snapshot and contribution objects #66 to the cellIterator functions: Some objects might have a large OSH bbox, but small (snapshot or contribution) geometries. Then it is not always necessary to calculate actual intersections, which are potentially slow computations.

relations with many (small) inner rings used to take a long time to assemble, because each of the "within" operation in a loop.
by using a prepared geometry, this is substantially faster
@tyrasd tyrasd requested a review from SlowMo24 March 7, 2019 17:56
@tyrasd tyrasd changed the title speedup building of complex multipolygons speedup building of geometries from osm entities Mar 8, 2019
@tyrasd tyrasd added the performance Performance optimizations, bottlenecks of the current pipeline, etc. label Mar 8, 2019
@tyrasd tyrasd added this to the OSHDB v0.5.0 milestone Mar 8, 2019
@tyrasd tyrasd requested a review from rtroilo March 8, 2019 12:20
@tyrasd tyrasd self-assigned this Mar 8, 2019
@tyrasd
Copy link
Member Author

tyrasd commented Mar 8, 2019

selection_214

applies the idea from #66 to the cellIterator functions:

some objects might have large OSH bboxes, but small geometries. Then it is not always necessary to calculate actual intersections (which are potentially slow).
@tyrasd tyrasd force-pushed the performance-speedup-complex-multipolygons branch from 850b2a0 to 84a1393 Compare March 8, 2019 14:22
@tyrasd tyrasd merged commit 7e723c0 into master Mar 8, 2019
@tyrasd tyrasd deleted the performance-speedup-complex-multipolygons branch March 8, 2019 17:11
@SlowMo24
Copy link
Contributor

SlowMo24 commented Mar 8, 2019

can confirm speed improvement of 3x faster on ignite (tested locally)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance optimizations, bottlenecks of the current pipeline, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants