Skip to content

Remove empty elements from exploded character-ngrams output - #15371

Merged
rapids-bot[bot] merged 1 commit into
NVIDIA:branch-24.06from
davidwendt:fix-char-ngram
Apr 4, 2024
Merged

Remove empty elements from exploded character-ngrams output#15371
rapids-bot[bot] merged 1 commit into
NVIDIA:branch-24.06from
davidwendt:fix-char-ngram

Conversation

@davidwendt

@davidwendt davidwendt commented Mar 21, 2024

Copy link
Copy Markdown
Contributor

Description

Fixes character_ngrams function to not include empty entries when as_list=False. That is, the exploded view (non-list result) should not contain empty or NA elements.

This PR changes the nvtext::generate_character_ngrams() API to return a lists column instead of a flat strings column. The python code had been converting the return object into lists column and then exploding it if as_list=False. Returning as a list column simplifies the logic and prevents the double conversion. There is almost no impact to the nvtext code since the offsets for the output lists column were already being generated.

All tests were updated to expect the new result. Also changed some exception types from cudf::logic_error to std::invalid_argument as appropriate.

Continues work of abandoned PR #14685

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@davidwendt davidwendt added bug Something isn't working 2 - In Progress Currently a work in progress libcudf Affects libcudf (C++/CUDA) code. strings strings issues (C++ and Python) breaking Breaking change labels Mar 21, 2024
@davidwendt davidwendt self-assigned this Mar 21, 2024
@github-actions github-actions Bot added the Python Affects Python cuDF API. label Mar 21, 2024
@davidwendt davidwendt added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Mar 25, 2024
@davidwendt
davidwendt marked this pull request as ready for review March 25, 2024 14:32
@davidwendt
davidwendt requested review from a team as code owners March 25, 2024 14:32
@davidwendt

Copy link
Copy Markdown
Contributor Author

/merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 - Ready for Review Ready for review by team breaking Breaking change bug Something isn't working libcudf Affects libcudf (C++/CUDA) code. Python Affects Python cuDF API. strings strings issues (C++ and Python)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants