datetime operations spanning MINYEAR give bad results

[markleander]

BPO	7150
Nosy	@amauryfa, @mdickinson, @abalkin
Files	datetimemodule.c.svndiff issue7150.diff issue7150a.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/abalkin'
closed_at = <Date 2010-05-27.22:08:58.131>
created_at = <Date 2009-10-16.09:45:28.820>
labels = ['type-bug', 'library']
title = 'datetime operations spanning MINYEAR give bad results'
updated_at = <Date 2010-05-27.22:08:58.129>
user = 'https://bugs.python.org/markleander'

bugs.python.org fields:

activity = <Date 2010-05-27.22:08:58.129>
actor = 'belopolsky'
assignee = 'belopolsky'
closed = True
closed_date = <Date 2010-05-27.22:08:58.131>
closer = 'belopolsky'
components = ['Library (Lib)']
creation = <Date 2009-10-16.09:45:28.820>
creator = 'mark.leander'
dependencies = []
files = ['15194', '16372', '17466']
hgrepos = []
issue_num = 7150
keywords = ['patch']
message_count = 15.0
messages = ['94132', '94384', '94449', '100102', '100104', '100105', '100107', '100118', '102945', '103718', '106509', '106511', '106529', '106531', '106618']
nosy_count = 5.0
nosy_names = ['amaury.forgeotdarc', 'mark.dickinson', 'belopolsky', 'pythonhacker', 'mark.leander']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue7150'
versions = ['Python 2.6', 'Python 2.7', 'Python 3.2', 'Python 3.3']

[markleander]

The datetime module documentation would imply that operations that cause
dates to fall outside the MINYEAR--MAXYEAR range should raise
OverflowError. The interpreter session below shows that this is not
always the case, and that such operations may cause bogus and
inconsistent results.

Python 2.6.3 (r263rc1:75186, Oct  2 2009, 20:40:30) [MSC v.1500 32 bit
(Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> t0=datetime.datetime(1,1,1)
>>> d1, d2, d3 = map(datetime.timedelta, range(1,4))
# The following is expected and accoring to the docs:
>>> t0-d1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: date value out of range

# The following is completely bogus:
>>> t0-d2
datetime.datetime(1, 0, 255, 0, 0)

# The two following behaving differently may be very confusing,
# the second one is correct
>>> t0-d2+d3
datetime.datetime(1, 8, 15, 0, 0)
>>> t0+d3-d2
datetime.datetime(1, 1, 2, 0, 0)
>>>

[pythonhacker]

The issue is present in Python 3.0 and 2.5 as well.

Python 2.5.1 (r251:54863, Jul 17 2008, 13:21:31)
[GCC 4.3.1 20080708 (Red Hat 4.3.1-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> t0=datetime.datetime(1,1,1)
>>> d1,d2,d3=map(datetime.timedelta, range(1,4))
>>> t0-d1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: date value out of range
>>> t0-d2
datetime.datetime(1, 0, 255, 0, 0)

I think this is bug in datetime for all Python versions

[pythonhacker]

The problem seems to be in the "normalize_date" function in
datetimemodule.c. It is checking for a valid year range, but not
checking for a valid month or day range.

I have a patch which fixes this problem. It checks for month range
(1<=m<=12) and day range(1<=d<=31). Here is Python with the patch.

anand@anand-laptop:~/projects/python/py3k$ ./python
Python 3.2a0 (py3k:75627, Oct 25 2009, 14:28:21) 
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "/home/anand/.pythonrc", line 2, in <module>
    import readline
ImportError: No module named readline
>>> import datetime
>>> t0=datetime.datetime(1,1,1)
>>> d1,d2,d3=map(datetime.timedelta, range(1,4))
>>> t0-d1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: date value out of range
>>> t0-d2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: date value out of range
>>> t0-d3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: date value out of range
>>> d0=datetime.timedelta(0)
>>> t0-d0
datetime.datetime(1, 1, 1, 0, 0)
>>> 

Svn diff is attached.

[amauryfa]

The patch seems OK, but needs tests.

[AlexanderBelopolsky]

Given

    assert(*m > 0);
    assert(*d > 0);

at the end of normalize_y_m_d(), it looks like at lest 1 <=*month and 1 <=*day are redundant.

A closer look also reveals

    assert(1 <= *m && *m <= 12);

in the middle of normalize_y_m_d(). This seems to leave only *day <=31 possibly relevant.

I suspect that out of bounds day surviving normalize_y_m_d() is a logical error in that function that needs to be fixed and an assert() added at the end. The proposed patch appears to cure the symptom rather than the actual flaw.

[AlexanderBelopolsky]

Aha! My reliance on asserts() was misguided. With the debug build:

>>> t0-d2
Assertion failed: (ordinal >= 1), function ord_to_ymd, file /Users/sasha/Work/python-svn/trunk/Modules/datetimemodule.c, line 269.
Abort trap

Should we reclassify this bug as a crash?

[amauryfa]

No, because normally distributions do not use debug builds.
but that's the reason why tests are needed: they must pass with asserts enabled.

[AlexanderBelopolsky]

I am attaching my variant of the patch including additional unit tests.

Note that my changes make normalize_y_m_d() and normalize_date() the same, but I am leaving the current structure intact to make reviewer's job easier.

[pythonhacker]

Can someone update this issue ? Is the 2nd patch tested... ?

[AlexanderBelopolsky]

My patch includes unit tests and I tested it on Mac OS X. Anand, what kind of testing do you have in mind?

[abalkin]

I've untabified my last patch and added a NEWS entry. I believe it is ready for commit review. Mark?

[mdickinson]

I'll take a look at this patch later today.