Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix panics for out-of-range timestamps #481

Merged
merged 1 commit into from
Apr 11, 2024

Conversation

jonmmease
Copy link
Collaborator

The PR removes some panics that could happen in datetime conversions. In particular, we were using arrow's unary function to map a custom function across the elements of an arrow array. For performance reasons, the function will be run on the underlying arrow memory even when the corresponding element is null, so the function must be valid for any possible value of the type. For the Date/timestamp types, this causes problems because not all valid integers represent value Dates or Timestamps. This PR switches to using the try_unary function which only evaluates the input function on non-null elements. In addition, I removed a bunch of unwrap with proper error propagation.

@jonmmease jonmmease merged commit 1bae34f into main Apr 11, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant