Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

date_range produces wrong values when overflowing #14187

Closed
jorisvandenbossche opened this issue Sep 8, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@jorisvandenbossche
Copy link
Member

commented Sep 8, 2016

In the following case, you now get a RuntimeWarning. But shouldn't we rather error on this?

In [23]: pd.date_range('1/1/2000', periods=100000, freq='D')
/home/joris/scipy/pandas/pandas/tseries/index.py:1944: RuntimeWarning: overflow encountered in long_scalars
  e = b + np.int64(periods) * stride
/home/joris/scipy/pandas/pandas/tseries/index.py:1954: RuntimeWarning: overflow encountered in long_scalars
  data = np.arange(b, e, stride, dtype=np.int64)
Out[23]: 
DatetimeIndex([          '2000-01-01 00:00:00',
                         '2000-01-02 00:00:00',
                         '2000-01-03 00:00:00',
                         '2000-01-04 00:00:00',
                         '2000-01-05 00:00:00',
                         '2000-01-06 00:00:00',
                         '2000-01-07 00:00:00',
                         '2000-01-08 00:00:00',
                         '2000-01-09 00:00:00',
                         '2000-01-10 00:00:00',
               ...
               '1689-03-17 00:25:26.290448384',
               '1689-03-18 00:25:26.290448384',
               '1689-03-19 00:25:26.290448384',
               '1689-03-20 00:25:26.290448384',
               '1689-03-21 00:25:26.290448384',
               '1689-03-22 00:25:26.290448384',
               '1689-03-23 00:25:26.290448384',
               '1689-03-24 00:25:26.290448384',
               '1689-03-25 00:25:26.290448384',
               '1689-03-26 00:25:26.290448384'],
              dtype='datetime64[ns]', length=100000, freq='D')

In [24]: pd.__version__
Out[24]: '0.19.0rc1+2.gd8cd33b'

In [25]: np.__version__
Out[25]: '1.11.1'

@jreback

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2016

might be now showing because of the np.errstate changes. yes an overflow is not good here (in general we can/should catch the overflow in an offset and just return a datetime, rather than a Timestamp) But here I think its an error as we can't represent that range.

@jreback jreback added this to the Next Major Release milestone Sep 8, 2016

@jreback

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2016

something like this would work I think

try:
    with np.errstate(over='raise'):
         e = b + .....
except FloatingPointError:
    raise ValueError("cannot create a date range like that")

@jreback jreback modified the milestones: Next Minor Release, Next Major Release Apr 19, 2017

@jreback jreback modified the milestones: Interesting Issues, Next Major Release Nov 26, 2017

@jreback jreback modified the milestones: Next Major Release, 0.23.0 Feb 19, 2018

@jreback jreback referenced this issue Feb 19, 2018

Closed

Fix wraparound/overflow in date_range #19740

4 of 4 tasks complete

@jreback jreback modified the milestones: 0.23.0, Next Major Release Apr 14, 2018

@jreback jreback modified the milestones: Contributions Welcome, 0.24.0 Oct 26, 2018

@jreback jreback referenced this issue Oct 26, 2018

Merged

BUG: Fix date_range overflow #23345

4 of 4 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.