March 3rd 1am US/Eastern equals one hour later when converted to UTC

[pjh5]

Minimal repro:

>>> from datetime import datetime, timedelta
>>> from dateutil.tz import tz
>>> ONE_HOUR = timedelta(hours=1)
>>> utc = tz.tzutc()
>>> d = datetime(2020, 3, 8, 1, 1, tzinfo=tz.gettz("US/Eastern"))
>>> d == (d + ONE_HOUR)
False
>>> d.astimezone(utc) == (d + ONE_HOUR).astimezone(utc)
True

I thought that the last equality check above should always be False, for all timezone-aware datetimes, no matter what timezones the datetime is converted to.

I checked that I see the correct behavior (at least what I expected) when I set d to an hour earlier (if d is March 3rd, 2020 at 00:01 then the final check is False). I did not check other timezones or dates.

I'm guessing that this is probably related to daylight savings time somehow, as DST in 2020 starts on March 3rd at 2am.

Versions:

> python --version
Python 3.6.3
> python -c 'import dateutil; print(dateutil.__version__)'
2.8.1

Note that these following equalities are also true.

>>> d == d.astimezone(utc)
True
>>> (d + ONE_HOUR) == d.astimezone(utc)
True
>>> d == (d + ONE_HOUR)
False

So both d and d + ONE_HOUR are both equal to d.astimezone(utc) but somehow still not equal to each other.

[pganssle]

@pjh5 I have a blog post on this very subject that was inspired by #338, which has a few other layers of indirection.

Basically the problem is this premise:

I thought that the last equality check above should always be False, for all timezone-aware datetimes, no matter what timezones the datetime is converted to.

It is possible to create aware, non-existent datetimes through arithmetic or just using the constructor. If you want them to represent a specific datetime in the timeline, you need to resolve them to some value. We have a convenience function tz.resolve_imaginary that handles one common use case: shifting forwards by the amount into the "imaginary" component you are. Normally I would recommend just converting to UTC and doing your arithmetic there (see this other blog post), but sometimes you want logic where of you land on a 'missing time' you shift over by the amount that's missing, but if you skip over a missing time, you don't want to do anything (e.g. you don't care that Y is 2 hours after X, just that adding 2 hours to X gives you a real time).

I don't think there's any action that needs to be taken here. Thanks for the report! (After all, if #338 hadn't happened, this would have been fuel for a talk and a blog post.)

[pjh5]

@pganssle thanks for the explanation, but now I have a follow up question.

In the example above, d is a true existing time. It is only when I add an hour to it that it become non-existant.

>>> d = datetime(2020, 3, 8, 1, 1, tzinfo=tz.gettz("US/Eastern"))
>>> tz.datetime_exists(d)
True
>>> tz.datetime_exists(d + ONE_HOUR)
False

So why doesn't adding timedeltas always preserve the existance of a datetime? I'd expect d + ONE_HOUR to correctly equal 3am local time instead of the non-existing 2am.

[pganssle]

@pjh5 That is basically down to how datetime arithmetic works, see the second blog post I referenced for more details. The TL;DR is that "addition" does not mean "add a certain amount of time in UTC", it means "move the clock/calendar forward by this amount and attach the same time zone".