[FEA] All points membership vector for HDBSCAN #4800
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
tarang-jain
commented
Jul 5, 2022
•
tarang-jain
commented
- All points distance membership vector
- All points outlier membership vector
- All points probability in some cluster
- All points membership vector
- Tests
tarang-jain
changed the title
tarang-jain
force-pushed
the
fea-all-points-membership-vector-hdbscan
branch
from
b823a4b
to
dbb7d48
Compare
tarang-jain
changed the title
cjnolet
requested changes
Aug 11, 2022
cpp/include/cuml/cluster/hdbscan.hpp
Outdated
| */ | ||
| void hdbscan(const raft::handle_t& handle, | ||
| const float* X, | ||
| size_t m, | ||
| size_t n, | ||
| raft::distance::DistanceType metric, | ||
| HDBSCAN::Common::HDBSCANParams& params, | ||
| HDBSCAN::Common::hdbscan_output<int, float>& out); | ||
| HDBSCAN::Common::hdbscan_output<int, float>& out, | ||
| bool prediction_data, |
There was a problem hiding this comment.
Why not add prediction_data to params? Ideally we'd use std::optional for prediction_data_
cpp/src/hdbscan/hdbscan.cu
Outdated
| exemplar_label_offsets.resize(n_selected_clusters_ + 1, handle.get_stream()); | ||
| deaths.resize(n_clusters, handle.get_stream()); | ||
| selected_clusters.resize(n_selected_clusters, handle.get_stream()); | ||
| raft::copy(exemplar_idx.begin(), exemplar_idx_, n_exemplars_, handle.get_stream()); |
There was a problem hiding this comment.
This is expensive in both memory usage and runtime. Rather than accepting raw pointers in cache, I would suggest creating an allocate() function that will resize the device_uvector members and then using the getters for the underlying pointers inside build_prediction_data to have the computations populate the values directly. That would remove the additional memory usage.
cpp/src/hdbscan/runner.h
Outdated
| @@ -255,6 +258,15 @@ void _fit_hdbscan(const raft::handle_t& handle, | |||
| * starting at 0 even in the presence of noise (-1) | |||
| */ | |||
|
|
|||
| if (prediction_data) { | |||
There was a problem hiding this comment.
if(params.prediction_data) or you could also do if(prediction_data_.has_value()) if you use std::optional.
cpp/test/sg/hdbscan_test.cu
Outdated
|
|
||
| params = ::testing::TestWithParam<SoftClusteringInputs<T, IdxT>>::GetParam(); | ||
|
|
||
| Logger::get().setLevel(CUML_LEVEL_DEBUG); |
|
rerun tests |
…points-membership-vector-hdbscan
…:tarang-jain/cuml into fea-all-points-membership-vector-hdbscan
…points-membership-vector-hdbscan
Codecov Report
@@ Coverage Diff @@
## branch-22.10 #4800 +/- ##
================================================
+ Coverage 78.02% 78.04% +0.01%
================================================
Files 180 180
Lines 11385 11422 +37
================================================
+ Hits 8883 8914 +31
- Misses 2502 2508 +6
Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
cjnolet
approved these changes
Aug 26, 2022
|
@gpucibot merge |
This was referenced Aug 29, 2022
divyegala
pushed a commit
to viclafargue/cuml
that referenced
this pull request
Sep 2, 2022
- [x] All points distance membership vector - [x] All points outlier membership vector - [x] All points probability in some cluster - [x] All points membership vector - [x] Tests Authors: - Tarang Jain (https://github.com/tarang-jain) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#4800
ajschmidt8
pushed a commit
that referenced
this pull request
Feb 13, 2023
- [x] All points distance membership vector - [x] All points outlier membership vector - [x] All points probability in some cluster - [x] All points membership vector - [x] Tests Authors: - Tarang Jain (https://github.com/tarang-jain) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: #4800
jakirkham
pushed a commit
to jakirkham/cuml
that referenced
this pull request
Feb 27, 2023
- [x] All points distance membership vector - [x] All points outlier membership vector - [x] All points probability in some cluster - [x] All points membership vector - [x] Tests Authors: - Tarang Jain (https://github.com/tarang-jain) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#4800
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.