Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use empty_like in scatter #8314

Merged
merged 3 commits into from May 24, 2021

Conversation

revans2
Copy link
Contributor

@revans2 revans2 commented May 21, 2021

This prevents things like partition from working with deeply nested arrays.

I marked this as non-breaking, but I am happy to change it to breaking because I removed a detailed API that is not used anywhere else and is flawed.

@revans2 revans2 added bug Something isn't working helps: Spark Functionality that helps Spark RAPIDS non-breaking Non-breaking change labels May 21, 2021
@revans2 revans2 self-assigned this May 21, 2021
@revans2 revans2 requested a review from a team as a code owner May 21, 2021 14:13
@revans2 revans2 added this to PR-WIP in v21.06 Release via automation May 21, 2021
@revans2 revans2 requested review from vuule and codereport May 21, 2021 14:13
@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label May 21, 2021
Comment on lines -62 to -64
std::unique_ptr<cudf::column> make_empty_lists_column(data_type child_type,
rmm::cuda_stream_view stream,
rmm::mr::device_memory_resource* mr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thumbs up. This function should not exist.

v21.06 Release automation moved this from PR-WIP to PR-Reviewer approved May 21, 2021
Copy link
Contributor

@codereport codereport left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@codecov
Copy link

codecov bot commented May 21, 2021

Codecov Report

❗ No coverage uploaded for pull request base (branch-21.06@9a85b3b). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##             branch-21.06    #8314   +/-   ##
===============================================
  Coverage                ?   82.88%           
===============================================
  Files                   ?      105           
  Lines                   ?    17874           
  Branches                ?        0           
===============================================
  Hits                    ?    14814           
  Misses                  ?     3060           
  Partials                ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9a85b3b...fdd2862. Read the comment docs.

@vuule
Copy link
Contributor

vuule commented May 21, 2021

What's the error in make_empty_lists_column?

@revans2
Copy link
Contributor Author

revans2 commented May 21, 2021

What's the error in make_empty_lists_column?

The error is that it is trying to make a deeply nested empty column, and make_empty_lists_column can only handle one level because it uses make_empty_column, and make_empty_column throws an exception when it sees a list column passed in as the type.

@nvdbaranec
Copy link
Contributor

nvdbaranec commented May 21, 2021

What's the error in make_empty_lists_column?

The error is that it is trying to make a deeply nested empty column, and make_empty_lists_column can only handle one level because it uses make_empty_column, and make_empty_column throws an exception when it sees a list column passed in as the type.

It's also that for nested types, the "type" is encoded in the hierarchy structure itself. So you can't properly preserve it without making a deep (even if potentially empty) copy of the whole thing.

@revans2
Copy link
Contributor Author

revans2 commented May 24, 2021

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 63faf2f into rapidsai:branch-21.06 May 24, 2021
v21.06 Release automation moved this from PR-Reviewer approved to Done May 24, 2021
@revans2 revans2 deleted the use_empty_like_in_scatter branch May 24, 2021 15:03
@isVoid
Copy link
Contributor

isVoid commented May 25, 2021

Just leaving a reference here so that I can refer back to this PR easily.
#8255

isVoid added a commit to isVoid/cudf that referenced this pull request May 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working helps: Spark Functionality that helps Spark RAPIDS libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

7 participants