New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.to() fails with large dates due to dateutil timestamp overflow #991
Comments
Thanks @matthuisman for the report. @jadchaar, @krisfremen and I will take a look at this soon and see what we can do. |
I am unable to reproduce this on my 64-bit macOS machine, so I think it is limited to Linux and windows. I think this is where the exception is triggered (our Lines 176 to 178 in 7c9632c
Therefore, we may be able to wrap this in a try/except and in the except block, get the timestamp, normalize it (using Are you able to reproduce this on your end @anishnya? We should probably get this reproduced either on a local machine or on the CI builds so we can ensure our patch works as expected once we attempt a fix. |
The first print will print fine - it's just the "to" that fails. |
@matthuisman dateutil has seen some recent work on it, but there was about a year stretch where there were zero commits to dateutil. I'm not too sure how likely it is dateutil will fix this issue and if they do fix it, when that fix will be merged in. We've had some internal discussions (well before this issue came up) about dropping the dateutil dependency from Arrow because of the inconsistency of dateutil's maintenance, but we haven't made a decision yet. |
I've tried on my local machine as well (macOS 64bit) and haven't been able to reproduce this issue as well. I'll try on a Windows machine as well and provide a future update. |
I think the issue is the same timestamp bug that requires the below lines in constants.py: You should be able to reproduce on mac with something like the below
|
Finding the max timestamp in a platform agnostic manner has proven to be very difficult (no standard ways to do this in the Python standard library or in dateutil). So the MAX_TIMESTAMP is used as a rought guide for when to compute the normalized timestamp, but it is imperfect because datetime seems to have trouble with its own max timestamp (which is what we use in >>> datetime.fromtimestamp(datetime.max.timestamp())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: year 0 is out of range |
Can't reproduce on my machine. >>> import arrow
>>> date = arrow.get('3100-01-01T07:00:00Z')
>>> date
<Arrow [3100-01-01T07:00:00+00:00]>
>>> print(date.to('local'))
3100-01-01T07:00:00+00:00
>>> import platform
>>> platform.uname()
uname_result(system='Linux', node='Z490', release='5.8.0-63-generic', version='#71-Ubuntu SMP Tue Jul 13 15:59:12 UTC 2021', machine='x86_64') |
how about this
or
|
Issue Description
(I know the issue is actually inside dateutil, but dateutil isn't written to support larger timestamps like arrow is)
64bit
32bit (user with a Raspberry Pi 3B+ initially reported this to me - 2050 isn't that far away)
Due to the timestamp being too large here:
https://github.com/dateutil/dateutil/blob/master/dateutil/tz/tz.py#L259
If you hack _naive_is_dst to something like below
it works as intended.
So maybe arrow needs to do it's own astimezone() so it can use it's normalise timestamp function.
For my workaround, I simply updated the dateutil code to use arrows normalize_timestamp
matthuisman/slyguy.addons@cccc92a
System Info
The text was updated successfully, but these errors were encountered: