New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removed inefficient LayerFilter check #2324

Merged
merged 2 commits into from Aug 9, 2017

Conversation

Projects
None yet
2 participants
@lossyrob
Member

lossyrob commented Aug 9, 2017

The non-overlapping check took upwards of 2 minutes in a case where I was querying a zoom 12 512x512 GlobalLayout for the state of Nevada. This is a nice thing to ensure, but not at that price. Make a note in the scaladocs for the implementor's responsibility to ensure non-overlapping KeyBounds.

@lossyrob

This comment has been minimized.

Member

lossyrob commented Aug 9, 2017

@echeipesh need advice here. That check really slows things down unacceptably in the workshop. But the failing test reminds me why it’s there (hint, it’s for the or that I doubt has been used in anger once). Is there some clever way we can get around having to do the costly check without allowing users to potentially do invalid queries (which I believe pulled back duplicate tiles)?

@echeipesh

This comment has been minimized.

Contributor

echeipesh commented Aug 9, 2017

Actually this should be fine now. I think this check existed before we started using MergeQueue on the RDDReaders: https://github.com/locationtech/geotrellis/blob/master/s3/src/main/scala/geotrellis/spark/io/s3/S3RDDReader.scala#L58

Any intersecting ranges should be taken care of there.

@lossyrob lossyrob merged commit ac4a6c2 into locationtech:master Aug 9, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
ip-validation
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment