New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chunk: change offset type to int64 #10348
Conversation
Codecov Report
@@ Coverage Diff @@
## master #10348 +/- ##
================================================
+ Coverage 77.6755% 77.6768% +0.0012%
================================================
Files 411 411
Lines 85440 85427 -13
================================================
- Hits 66366 66357 -9
+ Misses 14113 14109 -4
Partials 4961 4961 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/run-all-tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
What problem does this PR solve?
The max-length of a string field can be 6M, a typical batch size for Chunk is 1024, which is 1K. That is to say, the memory offset of a string column can be 6GB, which exceeds int32
What is changed and how it works?
change offset type from int32 to int64
Check List
Tests
Related changes