datetime.datetime operator methods are not subclass-friendly

[stingray]

BPO	2267
Nosy	@amauryfa, @mdickinson, @abalkin
Files	datetime.diff: patch against revision 61343

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/abalkin'
closed_at = <Date 2013-01-12.11:38:19.014>
created_at = <Date 2008-03-10.19:09:50.839>
labels = ['extension-modules', 'type-bug']
title = 'datetime.datetime operator methods are not subclass-friendly'
updated_at = <Date 2013-01-12.11:38:19.014>
user = 'https://bugs.python.org/stingray'

bugs.python.org fields:

activity = <Date 2013-01-12.11:38:19.014>
actor = 'belopolsky'
assignee = 'belopolsky'
closed = True
closed_date = <Date 2013-01-12.11:38:19.014>
closer = 'belopolsky'
components = ['Extension Modules']
creation = <Date 2008-03-10.19:09:50.839>
creator = 'stingray'
dependencies = []
files = ['9651']
hgrepos = []
issue_num = 2267
keywords = ['patch']
message_count = 14.0
messages = ['63446', '63452', '63453', '63454', '63456', '107410', '107414', '107416', '107417', '107418', '108060', '108061', '125979', '179783']
nosy_count = 4.0
nosy_names = ['amaury.forgeotdarc', 'mark.dickinson', 'belopolsky', 'stingray']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue2267'
versions = ['Python 3.4']

[stingray]

The datetime.datetime class overrides some arithmetic operations for it
to be able to add or subtract timedeltas. However, the result of A + B
operation, where A is instance of a subclass of datetime and B is
timedelta instance will be always the instance of base datetime.

This is extremely annoying and requires to override arithmetic operators
and writing a lots of rubbish to replace the datetime base object with
type(self)

[abalkin]

This is trivial to implement (see attached) and IMHO is a good idea.

The next question, however is whether similar changes should be made to
timedelta arithmetics. Timedelta case is not so clearcut because of the
usual dilemma of what the type of a+b should be when a and b are
instances of two different subclasses of timedelta.

[stingray]

I just checked the astimezone method - it also does this.

As with timedelta... well, it's not critical for me now but it worth
thinking about :)

[amauryfa]

Recently, the similar bpo-1562 "Decimal can't be subclassed useful" was
rejected. In the discussion I found a reference to a former post, which
precisely deals with datetime and timedelta:
http://mail.python.org/pipermail/python-list/2005-January/300791.html

The main argument is that the """base class has no idea what
requirements may exist for invoking a subclass's constructor"""

All python types behave this way: int, float, lists.

[abalkin]

Invoking a subclass's constructor is only an issue when subclass adds
data members. In this case, arithmetic methods need to be overridden.
Note that my patch does not make __add__ and friends invoke subclass'
constructor, only subclass' tp_alloc.

Existing code already does this in some cases. For example,

>>> class d(datetime): pass
... 
>>> d.strptime('20080310', '%Y%m%d')
d(2008, 3, 10, 0, 0)
>>> d.now()
d(2008, 3, 10, 20, 27, 6, 303147)

I think date/datetime present a particularly compelling case for
departing from the general rule. These classes are minimal by design
and need to be extended in many applications.

[abalkin]

I would like to take another shot at this. The link in Amaury's closing comment no longer works, so here is the relevant post:

"""
Returning same type as self for arithmetic in subclasses

Tim Peters tim.peters at gmail.com
Sat Jan 8 02:09:27 CET 2005

[Max M]
> """
> I subclass datetime and timedelta
> 
> >>> dt = myDatetime(1970,1,1)
> >>> type(dt)
> <class 'dtime.myDatetime'>
> 
> >>> td = myTimedelta(hours=1)
> >>> type(td)
> <class 'dtime.myTimedelta'>
> 
> But when I do arithmetic with these classes, they return datetime and
> timedelta,
...
> >>> new_time = dt + td
> >>> new_time
> datetime.datetime(1970, 1, 1, 1, 0)
> 
> >>> type(new_time)
> <type 'datetime.datetime'>

Yes, and all builtin Python types work that way. For example,
int.__add__ or float.__add__ applied to a subclass of int or float
will return an int or float; similarly for a subclass of str. This
was Guido's decision, based on that an implementation of any method in
a base class has no idea what requirements may exist for invoking a
subclass's constructor. For example, a subclass may restrict the
values of constructor arguments, or require more arguments than a base
class constructor; it may permute the order of positional arguments in
the base class constructor; it may even be "a feature" that a subclass
constructor gives a different meaning to an argument it shares with
the base class constructor. Since there isn't a way to guess, Python
does a safe thing instead.

where I want them to return myDatetime and myTimedelta

So I wondered if there was a simlpler way to coerce the result into my
desired types rather than overwriting the __add__, __sub__ etc. methods?

Generally speaking, no. But I'm sure someone will torture you with a
framework that purports to make it easy <wink>.
""" http://mail.python.org/pipermail/python-list/2005-January/925838.html

As I explained in my previous post, the same argument, "base class has no idea what requirements may exist for invoking a subclass's constructor", applies to class methods, but they nevertheless consistently construct subclass instances:

>>> class d(datetime): pass
>>> d.utcfromtimestamp(0)
d(1970, 1, 1, 0, 0)
>>> d.fromtimestamp(0)
d(1969, 12, 31, 19, 0)
>>> d.combine(date(1,1,1), time(1,1))
d(1, 1, 1, 1, 1)

Similar example for the date class:

>>> class Date(date): pass
>>> Date.fromordinal(1)
Date(1, 1, 1)

In my view it is hard to justify that for a Date instance d, and integer days, Date.fromordinal(d.toordinal() + days) happily produces a Date instance, but d + timedelta(days) returns a basic date instance.

[mdickinson]

If you want to challenge Guido's design decision, I think python-dev would be the place to do it.

[abalkin]

If you want to challenge Guido's design decision, I think
python-dev would be the place to do it.

Do you have a link supporting that it was "Guido's design decision"? This decision must have been made around class/type unification, but I don't remember reading about it in Guido's essays.

I don't want to make a fool of myself by coming to python-dev with this unprepared. :-)