-
Notifications
You must be signed in to change notification settings - Fork 876
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rework logic in cudf::strings::split_record to improve performance #12729
Rework logic in cudf::strings::split_record to improve performance #12729
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## branch-23.04 #12729 +/- ##
===============================================
Coverage ? 85.85%
===============================================
Files ? 158
Lines ? 25204
Branches ? 0
===============================================
Hits ? 21638
Misses ? 3566
Partials ? 0 Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
Performance numbers for
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM with some nits (not necessarily a change request though)
/merge |
Description
Updates the
cudf::strings::split_record
logic to match the more optimized code incudf::strings:split
.The optimized code performs much better for longer strings (>64 bytes) by parallelizing over the character bytes to find delimiters before determining split tokens.
This led to refactoring the code so it both APIs can share the optimized code.
Also fixes a bug found when using overlapped delimiters.
Additional tests were added for multi-byte delimiters which can overlap and span multiple adjacent strings.
Closes #12694
Checklist