Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sql-lab]Pyarrow to pandas fails when timestamp is out of bounds #13661

Closed
cabo40 opened this issue Mar 17, 2021 · 4 comments
Closed

[sql-lab]Pyarrow to pandas fails when timestamp is out of bounds #13661

cabo40 opened this issue Mar 17, 2021 · 4 comments
Labels
inactive Inactive for >= 30 days sqllab Namespace | Anything related to the SQL Lab

Comments

@cabo40
Copy link
Contributor

cabo40 commented Mar 17, 2021

When querying from a database with a timestamp column with [us] granularity and missing data as zero dates the query cannot be completed

Expected results

The query result with full precision of dates

Actual results

pyarrow.lib.ArrowInvalid: Casting from timestamp[us] to timestamp[ns] would result in out of bounds timestamp

How to reproduce the bug

Tested with a DB2 database, select a timestamp from 1900-01-01 in sql-lab

Environment

(please complete the following information):

  • superset version: 0.38.0
  • python version: 3.7.9
  • node.js version: 16.14.0
@cabo40 cabo40 added the #bug Bug report label Mar 17, 2021
@junlincc junlincc added v0.38 need:more-info Requires more information from author and removed #bug Bug report labels Mar 18, 2021
@cabo40
Copy link
Contributor Author

cabo40 commented Mar 27, 2021

Hi @junlincc . I am very very sorry for not answering sooner.

I tested with a fresh installation and could reproduce by querying in sql lab on the examples database:

SELECT TIMESTAMP '01-01-1000';

result:
image

@junlincc junlincc removed the need:more-info Requires more information from author label Apr 8, 2021
@junlincc junlincc changed the title Pyarrow to pandas fails when timestamp is out of bounds [sql-lab]Pyarrow to pandas fails when timestamp is out of bounds Apr 8, 2021
@junlincc
Copy link
Member

junlincc commented Apr 8, 2021

Thanks for the additional details and contributing a fix PR, we will get it reviewed asap!
@cabo40 🙏

@junlincc junlincc added sqllab Namespace | Anything related to the SQL Lab and removed v0.38 labels Apr 8, 2021
@cabo40
Copy link
Contributor Author

cabo40 commented Apr 12, 2021

Thanks to you for reviewing and contributing this great tool!

@stale
Copy link

stale bot commented May 2, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale bot added the inactive Inactive for >= 30 days label May 2, 2022
@cabo40 cabo40 closed this as completed May 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
inactive Inactive for >= 30 days sqllab Namespace | Anything related to the SQL Lab
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants