Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix writing of ORC files with empty rowgroups #13466

Merged
merged 2 commits into from
May 30, 2023
Merged

Fix writing of ORC files with empty rowgroups #13466

merged 2 commits into from
May 30, 2023

Conversation

vuule
Copy link
Contributor

@vuule vuule commented May 26, 2023 •
edited
Loading

Description

Issue #13460

Fixes the bug in gpuCompactOrcDataStreams where stream pointer would not get updated for empty row groups.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@vuule vuule added bug Something isn't working cuIO cuIO issue non-breaking Non-breaking change labels May 26, 2023
@vuule vuule self-assigned this May 26, 2023
@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label May 26, 2023
@vuule vuule changed the base branch from branch-23.08 to branch-23.06 May 26, 2023 23:25
@vuule vuule mentioned this pull request May 26, 2023
Closed
@vuule vuule changed the title Bug orc write empty rg Fix writing of ORC files with empty rowgroups May 27, 2023
@vuule vuule marked this pull request as ready for review May 27, 2023 06:27
@vuule vuule requested a review from a team as a code owner May 27, 2023 06:27
@vuule vuule requested review from bdice and mythrocks and removed request for a team May 27, 2023 06:27
mythrocks
mythrocks approved these changes May 27, 2023
View reviewed changes
Copy link
Contributor

@mythrocks mythrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

cpp/src/io/orc/stripe_enc.cu
}
if (t == 0) { streams[ss.column_id][group].data_ptrs[cid] = dst_ptr; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. Outside the length check.

@GregoryKimball
Copy link
Contributor

Thank you @vuule for jumping in and solving this on such short notice. 🙏 🙏

revans2
revans2 approved these changes May 30, 2023
View reviewed changes
Copy link
Contributor

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixed the original Spark query issues.

@raydouglass raydouglass merged commit e159c01 into rapidsai:branch-23.06 May 30, 2023
52 checks passed
@GregoryKimball GregoryKimball mentioned this pull request Jun 27, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@revans2 revans2 revans2 approved these changes

@bdice bdice bdice approved these changes

@mythrocks mythrocks mythrocks approved these changes

@ttnghia ttnghia ttnghia approved these changes

Assignees

@vuule vuule

Labels
bug Something isn't working cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
cuDF/Dask/Numba/UCX
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@vuule @GregoryKimball @revans2 @bdice @mythrocks @ttnghia @raydouglass