-
-
Notifications
You must be signed in to change notification settings - Fork 8.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use adapter to initialize column matrix. #7912
Conversation
trivialfis
commented
May 17, 2022
- Add an adapter for SparsePage. Might seem weird, but we already have something similar in GPU code and predictor. We can unify the code further by treating SparsePage as an ordinary adapter batch.
- Implement initilaization for column matrix from adapter.
- More precise tests.
This is to implement #7890 . |
src/common/column_matrix.h
Outdated
auto line = batch.GetLine(rid); | ||
for (size_t i = 0; i < line.Size(); ++i) { | ||
auto coo = line.GetElement(i); | ||
if (data::IsValidFunctor {missing}(coo)) { | ||
auto fid = coo.column_idx; | ||
const uint32_t bin_id = row_index[k]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the performance implication of abstracting the element access? Is the overhead acceptable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, wouldn't it be inefficient to create the functor in a tight-loop? Let's create the functor instance outside the loop.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the overhead acceptable?
It's not the bottleneck. The major issue with this function is the boolean vector missing_flags_
, which is not thread-safe. #7208 is likely to rewrite many of these anyway, right now I just want to make sure some of the features are merged before optimization.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
right now I just want to make sure some of the features are merged before optimization.
I'm not against optimization (and like them), just trying to show a big picture of what we need in these data structures before we go nuts with optimization.