Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite Dimension::overlap_ratio to remove overflow, underflow, and divide by zero #2304

Merged
merged 8 commits into from Jun 7, 2021

Conversation

eric-hughes-tiledb
Copy link
Contributor

Dimension::overlap_ratio was full of possible ways to overflow and give nonsense results. The tests added in this PR all (but one) failed in the old code, including some divide-by-zero triggered by overflow. The new code accounts for all the possibilities of overflow.


TYPE: BUG
DESC: Rewrite Dimension::overlap_ratio

Minor (non-fix) changes to overlap_ratio. Most importantly, add an assert() to verify the postcondition.
Add helpers-dimension.h with some utility functions.
Fix an iteration error in a caller of overlap_ratio. Fix typos.
… before.

Added some test tags.
Reverted a change in subarray.cc
…loating point, not the minimum-positive element.
test/src/unit-dimension.cc Outdated Show resolved Hide resolved
tiledb/sm/array_schema/dimension.cc Outdated Show resolved Hide resolved
@@ -469,10 +469,10 @@ class Dimension {
template <class T>
static bool overlap(const Range& r1, const Range& r2);

/** Return ratio of the overalp of the two input 1D ranges over `r2`. */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one of those too in domain.h while you're at it :)

KiterLuc and others added 3 commits June 4, 2021 14:27
Now that the overlap ratio is correct, we get proper size estimation
and the partitioner doesn't do an unnecessary split of the results
anymore.
clang-format
@joe-maley joe-maley merged commit dd77029 into dev Jun 7, 2021
@joe-maley joe-maley deleted the eh/fix-defect-overlap-ratio branch June 7, 2021 15:36
github-actions bot pushed a commit that referenced this pull request Jun 14, 2021
… divide by zero (#2304)

* Add tests for overflow defects in overlap_ratio.
Minor (non-fix) changes to overlap_ratio. Most importantly, add an assert() to verify the postcondition.
Add helpers-dimension.h with some utility functions.
Fix an iteration error in a caller of overlap_ratio. Fix typos.

* Rewrote Dimension::overlap_ratio to pass all the tests it was failing before.
Added some test tags.
Reverted a change in subarray.cc

* clang format

* More clang-format

* Use the limits function that returns the actual minimum element for floating point, not the minimum-positive element.

* Fix Hilbert test now that size estimate is correct

Now that the overlap ratio is correct, we get proper size estimation
and the partitioner doesn't do an unnecessary split of the results
anymore.

* Comment changes.
clang-format

Co-authored-by: Luc Rancourt <lucrancourt@gmail.com>
Shelnutt2 pushed a commit that referenced this pull request Jun 14, 2021
… divide by zero (#2304) (#2349)

* Add tests for overflow defects in overlap_ratio.
Minor (non-fix) changes to overlap_ratio. Most importantly, add an assert() to verify the postcondition.
Add helpers-dimension.h with some utility functions.
Fix an iteration error in a caller of overlap_ratio. Fix typos.

* Rewrote Dimension::overlap_ratio to pass all the tests it was failing before.
Added some test tags.
Reverted a change in subarray.cc

* clang format

* More clang-format

* Use the limits function that returns the actual minimum element for floating point, not the minimum-positive element.

* Fix Hilbert test now that size estimate is correct

Now that the overlap ratio is correct, we get proper size estimation
and the partitioner doesn't do an unnecessary split of the results
anymore.

* Comment changes.
clang-format

Co-authored-by: Luc Rancourt <lucrancourt@gmail.com>

Co-authored-by: eric-hughes-tiledb <82400964+eric-hughes-tiledb@users.noreply.github.com>
Co-authored-by: Luc Rancourt <lucrancourt@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants