[RELEASE] cugraph v22.04#2184
Merged
Merged
Conversation
[gpuCI] Forward-merge branch-22.02 to branch-22.04 [skip gpuci]
Fix Merge Conflicts for `2024`
[gpuCI] Forward-merge branch-22.02 to branch-22.04 [skip gpuci]
[gpuCI] Forward-merge branch-22.02 to branch-22.04 [skip gpuci]
…2050) * Updates to enable `NumberMap` to generate unique src/dst column names instead of assuming the names `src` and `dst`. This is needed since cudf no longer allows for duplicate column names, and some user input may contain "src" and "dst" as pre-existing column names intended for other purposes. * This commit also cleans up various tech debt items (exception types, docstrings, etc.). Tested by ensuring the existing python unit tests for SG and MG (2 GPUs) passed, and also ran all notebooks as tests. New tests to emphasize support for specific column names that previously failed with the latest cudf were also added. Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - Brad Rees (https://github.com/BradReesWork) - Chuck Hastings (https://github.com/ChuckHastings) URL: #2050
new GraphStore for DGL intergration Code will need to be added to DGL Authors: - Brad Rees (https://github.com/BradReesWork) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: #2046
…ning_wrapper()` return value (#2048) `experimental_warning_wrapper()` returned a wrapper function that prints a `PendingDeprecationWarning` when an `experimental` API is used. It can be used for experimental functions and classes, however for classes, the wrapper it returns was a `function` type instead of a `type` type. This caused the following problem: ``` >>> from cugraph.experimental import PropertyGraph >>> pG = PropertyGraph() >>> isinstance(pG, PropertyGraph) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: isinstance() arg 2 must be a type or tuple of types ``` This fix identifies the object to be wrapped and either creates a standard function wrapper for `function` types, or a class wrapper for `type` types: ``` >>> from cugraph.experimental import PropertyGraph >>> pG = PropertyGraph() >>> isinstance(pG, PropertyGraph) True ``` NOTE: the `api_tools.py` module is copied in both cugraph and pylibcugraph. When cugraph starts using pylibcugraph, the copy in cugraph can be removed (see FIXME). Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - Joseph Nke (https://github.com/jnke2016) - Don Acosta (https://github.com/acostadon) - Brad Rees (https://github.com/BradReesWork) URL: #2048
#2054) Some general cleanup of C++ Louvain tests. * Updated to use the latest structure/override features * Added Rmat tests for the MG tests Authors: - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Seunghwa Kang (https://github.com/seunghwak) URL: #2054
…raph()` and `num_vertices` performance (#2056) Various updates needed for adding new features for DGL, improving `extract_subgraph()` and `num_vertices` performance: * made `PropertyGraph` column name constants public for use by clients * changed `num_vertices` property impl for better performance * added `get_vertices()` * allow a `None` type_name for `add_*_data()` * updated docstrings * changed the `edge_data` added to Graphs to be a DataFrame for efficiency * added tests and benchmarks * refactored and added test fixtures * updated `pytest.ini` for new benchmarks Tested by ensuring all new tests added passed, benchmarks ran, and existing `test_graph_store.py` tests still passed. Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - Brad Rees (https://github.com/BradReesWork) URL: #2056
This PR replaces ccache with sccache. Authors: - Jake Awe (https://github.com/AyodeAwe) - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: #2045
Closes #2005 when merged Authors: - https://github.com/betochimas Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: #2049
This PR adds a CMake `install` rule for test targets. This step is a prerequisite to being able to package tests in their own `conda` package, which will enable us to deprecate _Project Flash_. Authors: - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - Robert Maynard (https://github.com/robertmaynard) - Rick Ratzel (https://github.com/rlratzel) URL: #2057
R-mat usecase parameters weren't set for multi-GPU tests. Fixed in this PR. Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - Kumar Aatish (https://github.com/kaatish) URL: #2061
…ity (#2035) Authors: - Don Acosta (https://github.com/acostadon) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: #2035
Refactored common code that copies device arrays returned by `cugraph_*` C APIs to cupy arrays into a utility function. This eliminates a significant amount of repeated code, and will be especially helpful as new `pylibcugraph` APIs are added that copy the existing patterns. Tested by running existing `pylibcugraph` unit tests. Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - https://github.com/betochimas - Brad Rees (https://github.com/BradReesWork) - Chuck Hastings (https://github.com/ChuckHastings) URL: #2060
Improve multi-node multi-GPU scalability of graph creation (especially the code computing renumber_map) Fix an overflow bug when creating a graph with more than 2^31 vertices Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - Kumar Aatish (https://github.com/kaatish) URL: #2044
Authors: - Corey J. Nolet (https://github.com/cjnolet) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - Rick Ratzel (https://github.com/rlratzel) URL: #2067
Changed to be in-line with:rapidsai/cudf#10182 Authors: - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Chuck Hastings (https://github.com/ChuckHastings) - Brad Rees (https://github.com/BradReesWork) URL: #2053
This adds a new node2vec method to the C API. Implementation will follow in a subsequent PR. Authors: - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Brad Rees (https://github.com/BradReesWork) URL: #2059
From comment in #2065, removing SPMV calls which are obsolete. Authors: - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Seunghwa Kang (https://github.com/seunghwak) URL: #2066
We need to do several things: - Add libcugraphops to conda (both in the cugraph recipe as well as build environment) - Add libcugraphops to cmake (I added a new thirdparty include here) - Ensure that the right headers and objects are added in cmake <s>For now, this is a work in progress since libcugraphops doesn't seem available on anaconda yet.</s> Authors: - Matt Joux (https://github.com/MatthiasKohl) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Brad Rees (https://github.com/BradReesWork) - Robert Maynard (https://github.com/robertmaynard) - Rick Ratzel (https://github.com/rlratzel) - Jordan Jacobelli (https://github.com/Ethyling) URL: #2019
[skip-CI] Branch 22.04 merge 22.02
…eight value on extracted Graphs even when a weight property wasn't specified (#2071) Allows a Graph to be extracted with a default weight value even if an edge property for a weight is not specified. This PR also changes the arg names that are intended to take column name strings to make it more obvious that strings are expected and not Column/Series objects. Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - Brad Rees (https://github.com/BradReesWork) - Don Acosta (https://github.com/acostadon) - Joseph Nke (https://github.com/jnke2016) URL: #2071
Completes NVIDIA/raft#400 Authors: - Victor Lafargue (https://github.com/viclafargue) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) URL: #2076
Addresses unit test failures affecting CUDA 11.4 builds, skips ktruss doctest and avoids assertion error. Specifically addresses cugraph-gpu-matrix-branch-22.04 in ops) Authors: - https://github.com/betochimas Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: #2074
This is one of the series of planned PRs to address #2003 We want to consistently use source & destination (or src & dst as abbreviation) instead of mixing source/destination and row/column and better group function/class/parameter names based on the concept of vertex partition and edge partition. This PR renames `row/col_properties_t` to `edge_partition_src/dst_property_t` and `copy_to_adj_matrix_partition_row/col` to `update_edge_partition_src/dst_property` and `transform_reduce_by_adj_matrix_row|col_key_e` to `transform_reduce_by_src|dst_key_e` and fix resulting compile errors. Possibly a breaking change (even though I am not aware of any external projects to be affected). Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Andrei Schaffer (https://github.com/aschaffer) - Chuck Hastings (https://github.com/ChuckHastings) URL: #2100
…esent (#2125) Removed unused cython code referencing RAFT APIs that are no longer present. Tested locally by doing a build with the latest RAFT. Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - Jordan Jacobelli (https://github.com/Ethyling) - Brad Rees (https://github.com/BradReesWork) URL: #2125
Define the uniform neighborhood sampling C API. The implementation will be done in a separate PR. An outline of what the implementation will resemble is provided (but ifdef'ed out). Authors: - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Seunghwa Kang (https://github.com/seunghwak) - Andrei Schaffer (https://github.com/aschaffer) URL: #2112
Add utilities to enable multi gpu gathering of adjacency lists to be used for mnmg sampling. Authors: - Kumar Aatish (https://github.com/kaatish) Approvers: - Andrei Schaffer (https://github.com/aschaffer) - AJ Schmidt (https://github.com/ajschmidt8) - Chuck Hastings (https://github.com/ChuckHastings) - Seunghwa Kang (https://github.com/seunghwak) URL: #2117
Partially addresses #2003 We should not use major,minor (instead of src, dst) in the public non-detail API and should use major,minor in the detail space. The public non-detail namespace renumber_edgelist currently uses major, minor and this PR fixes this. This PR also replaces row/col to src/dst in `single_gpu_renumber_edgelist_given_number_map` as we're aiming to consistently use src/dst instead of row/col in our public C++ API. This is possibly breaking as this PR changes the public API (but I am not aware of any non-cugraph libraries directly calling renumber_edgelist). Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - Brad Rees (https://github.com/BradReesWork) - Joseph Nke (https://github.com/jnke2016) - Rick Ratzel (https://github.com/rlratzel) URL: #2116
Authors: - Paul Taylor (https://github.com/trxcllnt) Approvers: - Brad Rees (https://github.com/BradReesWork) - Rick Ratzel (https://github.com/rlratzel) - AJ Schmidt (https://github.com/ajschmidt8) URL: #2127
Pin changes to be in-line with: rapidsai/cudf#10481 Authors: - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: #2147
I recently committed a config file to be used by the [rapidsai/ops-bot](https://github.com/rapidsai/ops-bot/) and in hindsight, I should've had the new `external_contributors` functionality set to `false` until we're ready to roll it out everywhere. This PR fixes that. Authors: - AJ Schmidt (https://github.com/ajschmidt8) Approvers:
This is a partial fix for #1979. Specifically, given `N = out-deg(v)` and a random number `r ∈ [0,1]`, one must obtain the equivalent discrete `index ∈ {0,1,...,N-1}`. Previous implementation used an upper bound `ubound = N-1` and a linear interpolation. As the issue above mentioned that approach creates problems near the (lower) boundary. The fix uses a better bound, namely `ubound = N` and the discrete transformation: `index = floor(r >= 1.0 ? N-1 : r*N)`. Attached Mathematica plots show the graphs for, say, `N = 13` and `N=17`.   This fix is not high priority for release 22-04, and can be included in the 22-06 release. Also, not all of the concerns formulated in the issue above are addressed by this PR. For example a uniform random generator callable from device is not yet available, but there are plans to perhaps expose something like that in `raft`. Authors: - Andrei Schaffer (https://github.com/aschaffer) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) URL: #2153
Depends on NVIDIA/raft#583 and rapidsai/cugraph-ops#70 This PR uses `CPMFindPackage` to find (or build) `cugraph-ops`. This enables building from source against a local `cugraph-ops` build via `-D cugraph-ops_ROOT=/cugraph-ops/build`. Authors: - Paul Taylor (https://github.com/trxcllnt) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: #2148
Closes #2092 Provide the C API implementation for HITS. Adds a unit tester for MG, updates the unit tests for both SG and MG. Authors: - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Brad Rees (https://github.com/BradReesWork) - Seunghwa Kang (https://github.com/seunghwak) URL: #2150
Updated random_walk_benchmark notebook for API change in cudf; to_array -> to_numpy Authors: - Mike McCarty (https://github.com/mmccarty) Approvers: - Brad Rees (https://github.com/BradReesWork) URL: #2164
Authors: - Brad Rees (https://github.com/BradReesWork) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Rick Ratzel (https://github.com/rlratzel) URL: #2171
Provide the implementation for the C API for neighborhood sampling. Currently needs some debugging. merge after 2150 Authors: - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Seunghwa Kang (https://github.com/seunghwak) URL: #2156
This PR wraps the MG C++ implementation of HITS following the pylibcugraph API Awaiting MG support to the C API closes #2026 Authors: - Joseph Nke (https://github.com/jnke2016) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Don Acosta (https://github.com/acostadon) - Chuck Hastings (https://github.com/ChuckHastings) - Rick Ratzel (https://github.com/rlratzel) URL: #2088
…lgos (#2099) Code to inherit all of networkx graph then call cugraph pagerank with just an import change. This was originally going to do bfs first but pargerank was considered a more useful first algorithm. Authors: - Don Acosta (https://github.com/acostadon) Approvers: - Brad Rees (https://github.com/BradReesWork) URL: #2099
…sting coverage (#2120) This PR was initially created because of two issues: 1. The pylibcugraph and cugraphs APIs differed from C++ and C because a compress flag was reinterpreted into a padding flag 2. Because of a renumbering bug, edges and vertices are not being renumbered correctly, causing multiple vertices and edges to be potentially identified as the same 3. Because of a type mismatching bug, `start_vertices` would yield garbage values and capture only half of the paths when using int64 values for vertices (this was initially thought to be a padding bug) What this PR does to address each: 1. Renamed the `use_padding` flag to `compress_result` to be inline with libcugraph. If a user desires to enforce padding to keep track of paths via formula, one would set `compress_result=False` (use_padding=True) 2. `random_walks.cpp` now unrenumbers local vertex ids for use with pylibcugraph and cugraph 3. node2vec now works with the requirement of using `int32` only for graph vertices. Future plans include adding support to `int64` and other types. Testing has been adapted from the C level, and further testing has been added to ensure invalid inputs are caught and results are expected with the new changes Authors: - https://github.com/betochimas - Rick Ratzel (https://github.com/rlratzel) Approvers: - Seunghwa Kang (https://github.com/seunghwak) - Chuck Hastings (https://github.com/ChuckHastings) - Rick Ratzel (https://github.com/rlratzel) URL: #2120
CMake 3.23 has a bug that breaks our conda-build based builds in CI, this avoids that issue. Authors: - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: #2176
Closes #2108 when merged. Requires both #2088 and #2156 to be merged before, the former because this uses MGGraph, and the later because of the C implementation of neighborhood sampling. Authors: - https://github.com/betochimas - Joseph Nke (https://github.com/jnke2016) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Don Acosta (https://github.com/acostadon) - Rick Ratzel (https://github.com/rlratzel) - Joseph Nke (https://github.com/jnke2016) - Chuck Hastings (https://github.com/ChuckHastings) - Jordan Jacobelli (https://github.com/Ethyling) URL: #2118
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
BradReesWork
added
! - Release
non-breaking
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
❄️ Code freeze for
branch-22.04and v22.04 releaseWhat does this mean?
Only critical/hotfix level issues should be merged into
branch-22.04until release (merging of this PR).What is the purpose of this PR?
branch-22.04intomainfor the release