Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Branch 23.06 merge 23.04 #1021

Merged
merged 8 commits into from
Mar 30, 2023
Merged

Branch 23.06 merge 23.04 #1021

merged 8 commits into from
Mar 30, 2023

Conversation

harrism
Copy link
Member

@harrism harrism commented Mar 29, 2023

Description

Fix failed automerge

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

thomcom and others added 8 commits March 23, 2023 22:09
@thomcom
Implement geom_equals and binary predicates that depend only on it. (…
04c941a 
…rapidsai#926)

This PR implements binary predicates that depend only on equality, which is implemented here using columnar comparison in python.

I'm playing with benchmarks of this feature now. On only Point geometries, we begin to outperform geopandas at 50k points, with 60x performance at 10m points.

Authors:
  - H. Thomson Comer (https://github.com/thomcom)

Approvers:
  - Michael Wang (https://github.com/isVoid)

URL: rapidsai#926
@isVoid
Add python API pairwise_point_polygon_distance (rapidsai#988)
6c27f2d 
This PR closes rapidsai#756 , add `pairwise_point_polygon_distance` for python.

Depend on rapidsai#984 rapidsai#976

Authors:
  - Michael Wang (https://github.com/isVoid)

Approvers:
  - H. Thomson Comer (https://github.com/thomcom)

URL: rapidsai#988
@ajschmidt8
Add dependency-file-generator as pre-commit hook (rapidsai#1008)
a147faf 
Similarly to these [cudf](rapidsai/cudf#12819) and [cuml](rapidsai/cuml#5246) PRs, this PR adds an entry to `.pre-commit-config.yaml` to run the [dependency-file-generator](https://github.com/rapidsai/dependency-file-generator).

It also adds an argument to the `rapidsai/shared-action-workflows/.github/workflows/checks.yaml` shared workflow to disable the `dependency-file-generator` from running in that shared workflow. This avoids having the `dependency-file-generator` run in two places since pre-commit is run in CI [here](https://github.com/rapidsai/cuspatial/blob/branch-23.04/ci/check_style.sh#L23).

Authors:
  - AJ Schmidt (https://github.com/ajschmidt8)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)

URL: rapidsai#1008
@isVoid
Add ZipCode Counting Notebook (rapidsai#919)
f065a89 
This PR adds a notebook that demonstrate the use of quadtree PiP with a custom `QuadTree` structure and joins dataframe. We can use this PR to discuss the possibility of adding the custom structure to the codebase.

This also updates the dependency list to include notebook environment in "all" targeted conda envs.

Authors:
  - Michael Wang (https://github.com/isVoid)

Approvers:
  - H. Thomson Comer (https://github.com/thomcom)
  - Mark Harris (https://github.com/harrism)
  - AJ Schmidt (https://github.com/ajschmidt8)

URL: rapidsai#919
@harrism
Header-only quadtree_point_in_polygon (rapidsai#979)
c3f6670 
Closes rapidsai#985 

Also contains cleanup of docs for other spatial join functions, correct ordering of stream and MR parameters, and adds missing C++17 property from tests cmake configuration.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Michael Wang (https://github.com/isVoid)
  - Paul Taylor (https://github.com/trxcllnt)

URL: rapidsai#979
@harrism
Reduce gtest times (rapidsai#1018)
ff22dd1 
Fixes rapidsai#1017. 

Reduces C++ gtest total time (on my PC) from 47.9 seconds to 20.08 seconds.

Several tests were running large datasets and combinations of size parameters that would be better to run as benchmarks rather than gtests. Reducing these by a factor of 10-100 saves a lot of development time and still exercises the code. In the case of `HausdorffTest/1.10000Spaces10Points (4850 ms)`, reducing it to 1000 spaces, 10 points reduced the time by nearly 100x, likely because it's $O(N^2)$. 

I modified any test that used close to 1s or more total time, since most column-API tests use under that, and most header-only tests use under 0.2s. 


| Test | Time Before (s) | Time After (s)  | Speedup |
|---|---|---|---|
| DERIVE_TRAJECTORIES_TEST_EXP | 14.49 | 0.27 | 53.7x |
| HAUSDORFF_TEST_EXP | 9.21 | 0.26 | 35.4x |
| UTILITY_TEST | 1.86 | 0.30 | 6.2x |
|  POINT_BOUNDING_BOXES_TEST_EXP | 1.35 | 0.15 | 9x |
| TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP | 0.80 | 0.13 | 6.2x |
| TOTAL | 47.9 | 20.08 | 2.4x |


Before:

```
(rapids) coder ➜ ~/cuspatial/cpp/build/release $ ninja test
[0/1] Running tests...
Test project /home/coder/cuspatial/cpp/build/release
      Start  1: SINUSOIDAL_PROJECTION_TEST
 1/45 Test  #1: SINUSOIDAL_PROJECTION_TEST .................   Passed    0.81 sec
      Start  2: HAVERSINE_TEST
 2/45 Test  #2: HAVERSINE_TEST .............................   Passed    0.77 sec
      Start  3: HAUSDORFF_TEST
 3/45 Test  #3: HAUSDORFF_TEST .............................   Passed    0.75 sec
      Start  4: JOIN_POINT_TO_LINESTRING_SMALL_TEST
 4/45 Test  rapidsai#4: JOIN_POINT_TO_LINESTRING_SMALL_TEST ........   Passed    0.73 sec
      Start  5: JOIN_POINT_IN_POLYGON_TEST
 5/45 Test  rapidsai#5: JOIN_POINT_IN_POLYGON_TEST .................   Passed    0.79 sec
      Start  6: POINT_IN_POLYGON_TEST
 6/45 Test  rapidsai#6: POINT_IN_POLYGON_TEST ......................   Passed    0.80 sec
      Start  7: PAIRWISE_POINT_IN_POLYGON_TEST
 7/45 Test  rapidsai#7: PAIRWISE_POINT_IN_POLYGON_TEST .............   Passed    0.76 sec
      Start  8: POINT_QUADTREE_TEST
 8/45 Test  rapidsai#8: POINT_QUADTREE_TEST ........................   Passed    0.76 sec
      Start  9: LINESTRING_BOUNDING_BOXES_TEST
 9/45 Test  rapidsai#9: LINESTRING_BOUNDING_BOXES_TEST .............   Passed    0.76 sec
      Start 10: POLYGON_BOUNDING_BOXES_TEST
10/45 Test rapidsai#10: POLYGON_BOUNDING_BOXES_TEST ................   Passed    0.80 sec
      Start 11: POINT_DISTANCE_TEST
11/45 Test rapidsai#11: POINT_DISTANCE_TEST ........................   Passed    0.79 sec
      Start 12: POINT_LINESTRING_DISTANCE_TEST
12/45 Test rapidsai#12: POINT_LINESTRING_DISTANCE_TEST .............   Passed    0.78 sec
      Start 13: LINESTRING_DISTANCE_TEST
13/45 Test rapidsai#13: LINESTRING_DISTANCE_TEST ...................   Passed    0.78 sec
      Start 14: POINT_POLYGON_DISTANCE_TEST
14/45 Test rapidsai#14: POINT_POLYGON_DISTANCE_TEST ................   Passed    0.76 sec
      Start 15: LINESTRING_INTERSECTION_TEST
15/45 Test rapidsai#15: LINESTRING_INTERSECTION_TEST ...............   Passed    0.83 sec
      Start 16: POINT_LINESTRING_NEAREST_POINT_TEST
16/45 Test rapidsai#16: POINT_LINESTRING_NEAREST_POINT_TEST ........   Passed    0.77 sec
      Start 17: QUADTREE_POLYGON_FILTERING_TEST
17/45 Test rapidsai#17: QUADTREE_POLYGON_FILTERING_TEST ............   Passed    0.79 sec
      Start 18: QUADTREE_LINESTRING_FILTERING_TEST
18/45 Test rapidsai#18: QUADTREE_LINESTRING_FILTERING_TEST .........   Passed    0.76 sec
      Start 19: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST
19/45 Test rapidsai#19: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST .......   Passed    0.79 sec
      Start 20: DERIVE_TRAJECTORIES_TEST
20/45 Test rapidsai#20: DERIVE_TRAJECTORIES_TEST ...................   Passed    0.76 sec
      Start 21: TRAJECTORY_BOUNDING_BOXES_TEST
21/45 Test rapidsai#21: TRAJECTORY_BOUNDING_BOXES_TEST .............   Passed    0.75 sec
      Start 22: SPATIAL_WINDOW_POINT_TEST
22/45 Test rapidsai#22: SPATIAL_WINDOW_POINT_TEST ..................   Passed    0.75 sec
      Start 23: UTILITY_TEST
23/45 Test rapidsai#23: UTILITY_TEST ...............................   Passed    1.86 sec
      Start 24: HAVERSINE_TEST_EXP
24/45 Test rapidsai#24: HAVERSINE_TEST_EXP .........................   Passed    0.14 sec
      Start 25: POINT_DISTANCE_TEST_EXP
25/45 Test rapidsai#25: POINT_DISTANCE_TEST_EXP ....................   Passed    0.11 sec
      Start 26: POINT_LINESTRING_DISTANCE_TEST_EXP
26/45 Test rapidsai#26: POINT_LINESTRING_DISTANCE_TEST_EXP .........   Passed    0.11 sec
      Start 27: POINT_POLYGON_DISTANCE_TEST_EXP
27/45 Test rapidsai#27: POINT_POLYGON_DISTANCE_TEST_EXP ............   Passed    0.13 sec
      Start 28: HAUSDORFF_TEST_EXP
28/45 Test rapidsai#28: HAUSDORFF_TEST_EXP .........................   Passed    9.21 sec
      Start 29: LINESTRING_DISTANCE_TEST_EXP
29/45 Test rapidsai#29: LINESTRING_DISTANCE_TEST_EXP ...............   Passed    0.17 sec
      Start 30: LINESTRING_INTERSECTION_TEST_EXP
30/45 Test rapidsai#30: LINESTRING_INTERSECTION_TEST_EXP ...........   Passed    0.19 sec
      Start 31: POINT_LINESTRING_NEAREST_POINT_TEST_EXP
31/45 Test rapidsai#31: POINT_LINESTRING_NEAREST_POINT_TEST_EXP ....   Passed    0.12 sec
      Start 32: SINUSOIDAL_PROJECTION_TEST_EXP
32/45 Test rapidsai#32: SINUSOIDAL_PROJECTION_TEST_EXP .............   Passed    0.12 sec
      Start 33: POINTS_IN_RANGE_TEST_EXP
33/45 Test rapidsai#33: POINTS_IN_RANGE_TEST_EXP ...................   Passed    0.11 sec
      Start 34: POINT_IN_POLYGON_TEST_EXP
34/45 Test rapidsai#34: POINT_IN_POLYGON_TEST_EXP ..................   Passed    0.12 sec
      Start 35: PAIRWISE_POINT_IN_POLYGON_TEST_EXP
35/45 Test rapidsai#35: PAIRWISE_POINT_IN_POLYGON_TEST_EXP .........   Passed    0.11 sec
      Start 36: DERIVE_TRAJECTORIES_TEST_EXP
36/45 Test rapidsai#36: DERIVE_TRAJECTORIES_TEST_EXP ...............   Passed   14.49 sec
      Start 37: POINT_BOUNDING_BOXES_TEST_EXP
37/45 Test rapidsai#37: POINT_BOUNDING_BOXES_TEST_EXP ..............   Passed    1.35 sec
      Start 38: POLYGON_BOUNDING_BOXES_TEST_EXP
38/45 Test rapidsai#38: POLYGON_BOUNDING_BOXES_TEST_EXP ............   Passed    0.11 sec
      Start 39: LINESTRING_BOUNDING_BOXES_TEST_EXP
39/45 Test rapidsai#39: LINESTRING_BOUNDING_BOXES_TEST_EXP .........   Passed    0.11 sec
      Start 40: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP
40/45 Test rapidsai#40: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP ...   Passed    0.80 sec
      Start 41: POINT_QUADTREE_TEST_EXP
41/45 Test rapidsai#41: POINT_QUADTREE_TEST_EXP ....................   Passed    0.12 sec
      Start 42: OPERATOR_TEST_EXP
42/45 Test rapidsai#42: OPERATOR_TEST_EXP ..........................   Passed    0.14 sec
      Start 43: FIND_TEST_EXP
43/45 Test rapidsai#43: FIND_TEST_EXP ..............................   Passed    0.13 sec
      Start 44: JOIN_POINT_IN_POLYGON_SMALL_TEST_EXP
44/45 Test rapidsai#44: JOIN_POINT_IN_POLYGON_SMALL_TEST_EXP .......   Passed    0.11 sec
      Start 45: JOIN_POINT_IN_POLYGON_LARGE_TEST_EXP
45/45 Test rapidsai#45: JOIN_POINT_IN_POLYGON_LARGE_TEST_EXP .......   Passed    0.13 sec

100% tests passed, 0 tests failed out of 45

Total Test time (real) =  47.07 sec
```

After:

```
(rapids) coder ➜ ~/cuspatial/cpp/build/release $ ninja test
[0/1] Running tests...
Test project /home/coder/cuspatial/cpp/build/release
      Start  1: SINUSOIDAL_PROJECTION_TEST
 1/45 Test  #1: SINUSOIDAL_PROJECTION_TEST .................   Passed    0.78 sec
      Start  2: HAVERSINE_TEST
 2/45 Test  #2: HAVERSINE_TEST .............................   Passed    0.75 sec
      Start  3: HAUSDORFF_TEST
 3/45 Test  #3: HAUSDORFF_TEST .............................   Passed    0.74 sec
      Start  4: JOIN_POINT_TO_LINESTRING_SMALL_TEST
 4/45 Test  rapidsai#4: JOIN_POINT_TO_LINESTRING_SMALL_TEST ........   Passed    0.77 sec
      Start  5: JOIN_POINT_IN_POLYGON_TEST
 5/45 Test  rapidsai#5: JOIN_POINT_IN_POLYGON_TEST .................   Passed    0.76 sec
      Start  6: POINT_IN_POLYGON_TEST
 6/45 Test  rapidsai#6: POINT_IN_POLYGON_TEST ......................   Passed    0.78 sec
      Start  7: PAIRWISE_POINT_IN_POLYGON_TEST
 7/45 Test  rapidsai#7: PAIRWISE_POINT_IN_POLYGON_TEST .............   Passed    0.74 sec
      Start  8: POINT_QUADTREE_TEST
 8/45 Test  rapidsai#8: POINT_QUADTREE_TEST ........................   Passed    0.75 sec
      Start  9: LINESTRING_BOUNDING_BOXES_TEST
 9/45 Test  rapidsai#9: LINESTRING_BOUNDING_BOXES_TEST .............   Passed    0.75 sec
      Start 10: POLYGON_BOUNDING_BOXES_TEST
10/45 Test rapidsai#10: POLYGON_BOUNDING_BOXES_TEST ................   Passed    0.73 sec
      Start 11: POINT_DISTANCE_TEST
11/45 Test rapidsai#11: POINT_DISTANCE_TEST ........................   Passed    0.73 sec
      Start 12: POINT_LINESTRING_DISTANCE_TEST
12/45 Test rapidsai#12: POINT_LINESTRING_DISTANCE_TEST .............   Passed    0.74 sec
      Start 13: LINESTRING_DISTANCE_TEST
13/45 Test rapidsai#13: LINESTRING_DISTANCE_TEST ...................   Passed    0.76 sec
      Start 14: POINT_POLYGON_DISTANCE_TEST
14/45 Test rapidsai#14: POINT_POLYGON_DISTANCE_TEST ................   Passed    0.76 sec
      Start 15: LINESTRING_INTERSECTION_TEST
15/45 Test rapidsai#15: LINESTRING_INTERSECTION_TEST ...............   Passed    0.78 sec
      Start 16: POINT_LINESTRING_NEAREST_POINT_TEST
16/45 Test rapidsai#16: POINT_LINESTRING_NEAREST_POINT_TEST ........   Passed    0.77 sec
      Start 17: QUADTREE_POLYGON_FILTERING_TEST
17/45 Test rapidsai#17: QUADTREE_POLYGON_FILTERING_TEST ............   Passed    0.75 sec
      Start 18: QUADTREE_LINESTRING_FILTERING_TEST
18/45 Test rapidsai#18: QUADTREE_LINESTRING_FILTERING_TEST .........   Passed    0.77 sec
      Start 19: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST
19/45 Test rapidsai#19: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST .......   Passed    0.74 sec
      Start 20: DERIVE_TRAJECTORIES_TEST
20/45 Test rapidsai#20: DERIVE_TRAJECTORIES_TEST ...................   Passed    0.75 sec
      Start 21: TRAJECTORY_BOUNDING_BOXES_TEST
21/45 Test rapidsai#21: TRAJECTORY_BOUNDING_BOXES_TEST .............   Passed    0.74 sec
      Start 22: SPATIAL_WINDOW_POINT_TEST
22/45 Test rapidsai#22: SPATIAL_WINDOW_POINT_TEST ..................   Passed    0.75 sec
      Start 23: UTILITY_TEST
23/45 Test rapidsai#23: UTILITY_TEST ...............................   Passed    0.30 sec
      Start 24: HAVERSINE_TEST_EXP
24/45 Test rapidsai#24: HAVERSINE_TEST_EXP .........................   Passed    0.12 sec
      Start 25: POINT_DISTANCE_TEST_EXP
25/45 Test rapidsai#25: POINT_DISTANCE_TEST_EXP ....................   Passed    0.12 sec
      Start 26: POINT_LINESTRING_DISTANCE_TEST_EXP
26/45 Test rapidsai#26: POINT_LINESTRING_DISTANCE_TEST_EXP .........   Passed    0.12 sec
      Start 27: POINT_POLYGON_DISTANCE_TEST_EXP
27/45 Test rapidsai#27: POINT_POLYGON_DISTANCE_TEST_EXP ............   Passed    0.13 sec
      Start 28: HAUSDORFF_TEST_EXP
28/45 Test rapidsai#28: HAUSDORFF_TEST_EXP .........................   Passed    0.26 sec
      Start 29: LINESTRING_DISTANCE_TEST_EXP
29/45 Test rapidsai#29: LINESTRING_DISTANCE_TEST_EXP ...............   Passed    0.14 sec
      Start 30: LINESTRING_INTERSECTION_TEST_EXP
30/45 Test rapidsai#30: LINESTRING_INTERSECTION_TEST_EXP ...........   Passed    0.19 sec
      Start 31: POINT_LINESTRING_NEAREST_POINT_TEST_EXP
31/45 Test rapidsai#31: POINT_LINESTRING_NEAREST_POINT_TEST_EXP ....   Passed    0.11 sec
      Start 32: SINUSOIDAL_PROJECTION_TEST_EXP
32/45 Test rapidsai#32: SINUSOIDAL_PROJECTION_TEST_EXP .............   Passed    0.11 sec
      Start 33: POINTS_IN_RANGE_TEST_EXP
33/45 Test rapidsai#33: POINTS_IN_RANGE_TEST_EXP ...................   Passed    0.13 sec
      Start 34: POINT_IN_POLYGON_TEST_EXP
34/45 Test rapidsai#34: POINT_IN_POLYGON_TEST_EXP ..................   Passed    0.11 sec
      Start 35: PAIRWISE_POINT_IN_POLYGON_TEST_EXP
35/45 Test rapidsai#35: PAIRWISE_POINT_IN_POLYGON_TEST_EXP .........   Passed    0.14 sec
      Start 36: DERIVE_TRAJECTORIES_TEST_EXP
36/45 Test rapidsai#36: DERIVE_TRAJECTORIES_TEST_EXP ...............   Passed    0.27 sec
      Start 37: POINT_BOUNDING_BOXES_TEST_EXP
37/45 Test rapidsai#37: POINT_BOUNDING_BOXES_TEST_EXP ..............   Passed    0.15 sec
      Start 38: POLYGON_BOUNDING_BOXES_TEST_EXP
38/45 Test rapidsai#38: POLYGON_BOUNDING_BOXES_TEST_EXP ............   Passed    0.13 sec
      Start 39: LINESTRING_BOUNDING_BOXES_TEST_EXP
39/45 Test rapidsai#39: LINESTRING_BOUNDING_BOXES_TEST_EXP .........   Passed    0.12 sec
      Start 40: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP
40/45 Test rapidsai#40: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP ...   Passed    0.13 sec
      Start 41: POINT_QUADTREE_TEST_EXP
41/45 Test rapidsai#41: POINT_QUADTREE_TEST_EXP ....................   Passed    0.14 sec
      Start 42: OPERATOR_TEST_EXP
42/45 Test rapidsai#42: OPERATOR_TEST_EXP ..........................   Passed    0.14 sec
      Start 43: FIND_TEST_EXP
43/45 Test rapidsai#43: FIND_TEST_EXP ..............................   Passed    0.15 sec
      Start 44: JOIN_POINT_IN_POLYGON_SMALL_TEST_EXP
44/45 Test rapidsai#44: JOIN_POINT_IN_POLYGON_SMALL_TEST_EXP .......   Passed    0.12 sec
      Start 45: JOIN_POINT_IN_POLYGON_LARGE_TEST_EXP
45/45 Test rapidsai#45: JOIN_POINT_IN_POLYGON_LARGE_TEST_EXP .......   Passed    0.13 sec

100% tests passed, 0 tests failed out of 45

Total Test time (real) =  20.08 sec
```

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Michael Wang (https://github.com/isVoid)
  - Paul Taylor (https://github.com/trxcllnt)

URL: rapidsai#1018
@harrism
Merge branch 'branch-23.04' into branch-23.06-merge-23.04
8704a78
@harrism
Re-add enabled_check_generated_files:false
6fbecbd
@harrism harrism requested review from a team as code owners March 29, 2023 22:13
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@github-actions github-actions bot added cmake Related to CMake code or build configuration conda Related to conda and conda configuration libcuspatial Relates to the cuSpatial C++ library Python Related to Python code labels Mar 29, 2023
@harrism harrism added non-breaking Non-breaking change improvement Improvement / enhancement to an existing function labels Mar 29, 2023
msadang
msadang approved these changes Mar 30, 2023
View reviewed changes
Copy link
Contributor

@msadang msadang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@msadang msadang merged commit a7466af into rapidsai:branch-23.06 Mar 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@isVoid isVoid isVoid approved these changes

@msadang msadang msadang approved these changes

@vyasr vyasr Awaiting requested review from vyasr vyasr is a code owner automatically assigned from rapidsai/cuspatial-cmake-codeowners

@thomcom thomcom Awaiting requested review from thomcom thomcom is a code owner automatically assigned from rapidsai/cuspatial-python-codeowners

@trxcllnt trxcllnt Awaiting requested review from trxcllnt trxcllnt is a code owner automatically assigned from rapidsai/cuspatial-cpp-codeowners

Assignees
No one assigned
Labels
cmake Related to CMake code or build configuration conda Related to conda and conda configuration improvement Improvement / enhancement to an existing function libcuspatial Relates to the cuSpatial C++ library non-breaking Non-breaking change Python Related to Python code
Projects
cuSpatial
Status: Todo
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@harrism @isVoid @msadang @thomcom @ajschmidt8