-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[C++] "iso_calendar" kernel returns incorrect results for array length > 32 #38655
Labels
Component: Python
Critical Fix
Bugfixes for security vulnerabilities, crashes, or invalid data.
Type: bug
Milestone
Comments
3 tasks
pyarrow.compute.iso_calendar
returns incorrect results and segfaults for multiple repeated timestampspyarrow.compute.iso_calendar
returns incorrect results and segfaults for multiple repeated timestamps
pyarrow.compute.iso_calendar
returns incorrect results and segfaults for multiple repeated timestamps
Thanks! I can't reproduce the segfault, but it's of course obviously wrong anyway. It seems the repeated values don't matter, I can also reproduce this with a date range, and it starts to happens once above 32 values:
cc @rok |
Oh, that is interesting. I'll take a look this week. |
jorisvandenbossche
added a commit
that referenced
this issue
Jan 23, 2024
…rray length > 32 (#39360) ### Rationale for this change When defining `StructArray`'s field builders for `ISOCalendar` we don't pre-allocate memory and then use unsafe append. This causes the resulting array to be at most 32 rows long. ### What changes are included in this PR? This introduces required memory pre-allocation in the `ISOCalendar` c++ kernel. ### Are these changes tested? This adds a test for the Python wrapper. ### Are there any user-facing changes? Fixes the behavior of `iso_calendar` kernel. * Closes: #38655 Lead-authored-by: Rok Mihevc <rok@mihevc.org> Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this issue
Feb 19, 2024
… for array length > 32 (apache#39360) ### Rationale for this change When defining `StructArray`'s field builders for `ISOCalendar` we don't pre-allocate memory and then use unsafe append. This causes the resulting array to be at most 32 rows long. ### What changes are included in this PR? This introduces required memory pre-allocation in the `ISOCalendar` c++ kernel. ### Are these changes tested? This adds a test for the Python wrapper. ### Are there any user-facing changes? Fixes the behavior of `iso_calendar` kernel. * Closes: apache#38655 Lead-authored-by: Rok Mihevc <rok@mihevc.org> Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
raulcd
pushed a commit
that referenced
this issue
Feb 20, 2024
…rray length > 32 (#39360) ### Rationale for this change When defining `StructArray`'s field builders for `ISOCalendar` we don't pre-allocate memory and then use unsafe append. This causes the resulting array to be at most 32 rows long. ### What changes are included in this PR? This introduces required memory pre-allocation in the `ISOCalendar` c++ kernel. ### Are these changes tested? This adds a test for the Python wrapper. ### Are there any user-facing changes? Fixes the behavior of `iso_calendar` kernel. * Closes: #38655 Lead-authored-by: Rok Mihevc <rok@mihevc.org> Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
zanmato1984
pushed a commit
to zanmato1984/arrow
that referenced
this issue
Feb 28, 2024
… for array length > 32 (apache#39360) ### Rationale for this change When defining `StructArray`'s field builders for `ISOCalendar` we don't pre-allocate memory and then use unsafe append. This causes the resulting array to be at most 32 rows long. ### What changes are included in this PR? This introduces required memory pre-allocation in the `ISOCalendar` c++ kernel. ### Are these changes tested? This adds a test for the Python wrapper. ### Are there any user-facing changes? Fixes the behavior of `iso_calendar` kernel. * Closes: apache#38655 Lead-authored-by: Rok Mihevc <rok@mihevc.org> Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
thisisnic
pushed a commit
to thisisnic/arrow
that referenced
this issue
Mar 8, 2024
… for array length > 32 (apache#39360) ### Rationale for this change When defining `StructArray`'s field builders for `ISOCalendar` we don't pre-allocate memory and then use unsafe append. This causes the resulting array to be at most 32 rows long. ### What changes are included in this PR? This introduces required memory pre-allocation in the `ISOCalendar` c++ kernel. ### Are these changes tested? This adds a test for the Python wrapper. ### Are there any user-facing changes? Fixes the behavior of `iso_calendar` kernel. * Closes: apache#38655 Lead-authored-by: Rok Mihevc <rok@mihevc.org> Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Component: Python
Critical Fix
Bugfixes for security vulnerabilities, crashes, or invalid data.
Type: bug
Describe the bug, including details regarding any error messages, version, and platform.
Component(s)
Python
The text was updated successfully, but these errors were encountered: