Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix memcheck error found in nvtext tokenize functions #13649

Merged

Conversation

davidwendt
Copy link
Contributor

Description

Fixes memcheck error found by the nightly build in the nvtext characters_tokenizer utility function.

[ RUN      ] TextNgramsTokenizeTest.Tokenize
========= Invalid __global__ read of size 1 bytes
=========     at 0x2360 in void cub::CUB_101702_610_860_NS::DeviceScanKernel<cub::CUB_101702_610_860_NS::DeviceScanPolicy<int>::Policy600, thrust::cuda_cub::transform_input_iterator_t<int, thrust::counting_iterator<int, thrust::use_default, thrust::use_default, thrust::use_default>, nvtext::detail::strings_tokenizer>, int *, cub::CUB_101702_610_860_NS::ScanTileState<int, (bool)1>, thrust::plus<int>, cub::CUB_101702_610_860_NS::NullType, int>(T2, T3, T4, int, T5, T6, T7)
=========     by thread (5,0,0) in block (0,0,0)
=========     Address 0x7f67a0200a65 is out of bounds
=========     and is 1 bytes after the nearest allocation at 0x7f67a0200a00 of size 101 bytes
=========     Saved host backtrace up to driver entry point at kernel launch time
=========     Host Frame: [0x30b492]
=========                in /usr/lib/x86_64-linux-gnu/libcuda.so.1
=========     Host Frame: [0x1488c]
=========                in /conda/envs/rapids/lib/libcudart.so.11.0
=========     Host Frame:cudaLaunchKernel [0x6c318]
=========                in /conda/envs/rapids/lib/libcudart.so.11.0
=========     Host Frame:nvtext::detail::ngrams_tokenize(cudf::strings_column_view const&, int, cudf::string_scalar const&, cudf::string_scalar const&, rmm::cuda_stream_view, rmm::mr::device_memory_resource*) [0x2693cc9]
=========                in /conda/envs/rapids/lib/libcudf.so

This error was introduced by changes in #13480

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@davidwendt davidwendt added bug Something isn't working 3 - Ready for Review Ready for review by team libcudf Affects libcudf (C++/CUDA) code. strings strings issues (C++ and Python) non-breaking Non-breaking change labels Jun 30, 2023
@davidwendt davidwendt self-assigned this Jun 30, 2023
@davidwendt davidwendt requested a review from a team as a code owner June 30, 2023 12:33
@davidwendt
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 62c4f99 into rapidsai:branch-23.08 Jun 30, 2023
58 checks passed
@davidwendt davidwendt deleted the nvtext-tokenize-memcheck branch June 30, 2023 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team bug Something isn't working libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change strings strings issues (C++ and Python)
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

3 participants