Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix orc reader assert on create data_type #8174

Merged

Conversation

davidwendt
Copy link
Contributor

The following error shows in the gtests/ORC_TEST when run under libcudf built with Debug.

[ RUN      ] OrcWriterNumericTypeTest/0.SingleColumn
ORC_TEST: ../include/cudf/types.hpp:267: cudf::data_type::data_type(cudf::type_id, int32_t): Assertion `id == type_id::DECIMAL32 || id == type_id::DECIMAL64' failed.
Aborted (core dumped)

The assert occurs in the data_type constructor meant for fixed-point types only.

explicit data_type(type_id id, int32_t scale) : _id{id}, _fixed_point_scale{scale}
{
assert(id == type_id::DECIMAL32 || id == type_id::DECIMAL64);
}

The orc reader implementation is using this ctor for all types and just passing 0 for the scale value when not a fixed-point type.
The assert only occurs in a debug build.

This PR changes the logic to call the appropriate data_type constructor depending on the type-id of the parsed column.

@davidwendt davidwendt added bug Something isn't working 3 - Ready for Review Ready for review by team libcudf Affects libcudf (C++/CUDA) code. cuIO cuIO issue non-breaking Non-breaking change labels May 6, 2021
@davidwendt davidwendt self-assigned this May 6, 2021
@davidwendt davidwendt requested a review from a team as a code owner May 6, 2021 15:32
@davidwendt davidwendt requested review from trxcllnt, ttnghia, vuule and devavret and removed request for ttnghia and trxcllnt May 6, 2021 15:32
@codecov
Copy link

codecov bot commented May 6, 2021

Codecov Report

Merging #8174 (01a1756) into branch-0.20 (51336df) will decrease coverage by 0.03%.
The diff coverage is n/a.

❗ Current head 01a1756 differs from pull request most recent head dd4c525. Consider uploading reports for the commit dd4c525 to get more accurate results
Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.20    #8174      +/-   ##
===============================================
- Coverage        82.88%   82.85%   -0.04%     
===============================================
  Files              103      104       +1     
  Lines            17668    17881     +213     
===============================================
+ Hits             14645    14816     +171     
- Misses            3023     3065      +42     
Impacted Files Coverage Δ
python/cudf/cudf/core/tools/datetimes.py 80.42% <0.00%> (-4.11%) ⬇️
python/cudf/cudf/core/abc.py 87.50% <0.00%> (-3.99%) ⬇️
python/cudf/cudf/core/column/decimal.py 91.04% <0.00%> (-1.89%) ⬇️
python/cudf/cudf/core/column/datetime.py 88.03% <0.00%> (-1.88%) ⬇️
python/cudf/cudf/core/column/struct.py 94.73% <0.00%> (-1.56%) ⬇️
python/cudf/cudf/utils/dtypes.py 82.20% <0.00%> (-1.24%) ⬇️
python/dask_cudf/dask_cudf/groupby.py 91.28% <0.00%> (-0.88%) ⬇️
python/cudf/cudf/core/series.py 91.17% <0.00%> (-0.56%) ⬇️
python/cudf/cudf/core/index.py 92.52% <0.00%> (-0.55%) ⬇️
python/cudf/cudf/core/column/column.py 88.20% <0.00%> (-0.44%) ⬇️
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 611cabd...dd4c525. Read the comment docs.

@vuule vuule requested a review from rgsl888prabhu May 7, 2021 18:24
@vuule
Copy link
Contributor

vuule commented May 7, 2021

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 245d8c1 into rapidsai:branch-0.20 May 7, 2021
@davidwendt davidwendt deleted the debug-assert-orc-reader branch May 7, 2021 20:10
rapids-bot bot pushed a commit that referenced this pull request Jul 12, 2021
The fix from #8174 got reverted in rework of `orc/reader_impl.cu` in #8599 
This PR reinstates the original fix to prevent an assert in debug mode in `gtests/ORC_TEST`

Authors:
  - David Wendt (https://github.com/davidwendt)

Approvers:
  - Karthikeyan (https://github.com/karthikeyann)
  - Conor Hoekstra (https://github.com/codereport)

URL: #8706
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team bug Something isn't working cuIO cuIO issue libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants