Properly handle UTC conversion in date2num. #6262

Merged
merged 3 commits into from Jul 26, 2016

Conversation

Projects
None yet
7 participants
Contributor

pganssle commented Apr 2, 2016

Fixes #3896.

I believe this is a much better way to do it than the current method where we're trying to pull the utcoffset. The issue in #3896 actually has more to do with the way pytz handles normalization and the way that pandas stores TimeStamp objects. As you can see from the test I added, a normal list of datetime objects with pytz time zones won't trigger the bug, but rather than pulling in pandas just for the test, I mocked out a datetime object that basically does the same thing pandas is doing.

mdboom added the needs_review label Apr 2, 2016

@pganssle pganssle commented on an outdated diff Apr 2, 2016

lib/matplotlib/tests/test_dates.py
@@ -20,6 +22,8 @@
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
+from numpy import array
@pganssle

pganssle Apr 2, 2016

Contributor

Oops, this was an artifact of an earlier test strategy. I'll remove it now.

Contributor

pganssle commented Apr 2, 2016

It may be worth adding a test that makes sure this works during a DST->STD transition, where there is a fold, but I suspect that in general that would mostly be a test of the time zone object's ability to handle ambiguous dates (see the extensive discussion at dateutil/dateutil#225 and the various linked discussions for more information than you could ever want about this), so I don't see any significant need for that. The best advice for people is that if they want to use date2num, they may as well just convert to UTC anyway, since date2num gives you an ordinal UTC time anyway.

Owner

tacaswell commented Apr 2, 2016

I am glad someone who is not me understands dates 😄 .

We do have some pandas-optional tests floating around already (I think mostly in test_axes) and install pandas on both travis and appveyor so if you want to use pandas you can.

Contributor

pganssle commented Apr 2, 2016

OK, I can add a pandas-specific test. Any idea why the Appveyor build is failing? I got that same failure locally on linux, but since I didn't touch anything related to that test, I figured it was just some weird version-specific problem.

Owner

jenshnielsen commented Apr 2, 2016

The appveyor is ses is likely due too #5950

@pganssle pganssle Add pandas-based test.
66c0ed7

@pganssle pganssle and 1 other commented on an outdated diff Apr 2, 2016

lib/matplotlib/dates.py
- if td_remainder > 0:
- base += td_remainder / SEC_PER_DAY
+ # Append the seconds as a fraction of a day
+ base += _total_seconds(dt - rdt) / SEC_PER_DAY
@pganssle

pganssle Apr 2, 2016

Contributor

Note: this _total_seconds function was added for Python 2.6 compatibility, but based on the CI it seems that you've dropped Python 2.6 support. Have there been enough Python 2.6-breaking changes that it makes sense to drop this compatibility artifact?

@tacaswell

tacaswell Apr 2, 2016

Owner

We are still testing 2.6 for the 1.5.x branch but other wise yes, remove
the 2.6 related stuff. We just will not backport this to 1.5.x (there
should not be a 1.5.2 release).

On Sat, Apr 2, 2016 at 5:31 PM Paul Ganssle notifications@github.com
wrote:

In lib/matplotlib/dates.py
#6262 (comment):

  •    if td_remainder > 0:
    
  •        base += td_remainder / SEC_PER_DAY
    
  •    # Append the seconds as a fraction of a day
    
  •    base += _total_seconds(dt - rdt) / SEC_PER_DAY
    

Note: this _total_seconds function was added for Python 2.6
compatibility, but based on the CI it seems that you've dropped Python 2.6
support. Have there been enough Python 2.6-breaking changes that it makes
sense to drop this compatibility artifact?


You are receiving this because you commented.

Reply to this email directly or view it on GitHub
https://github.com/matplotlib/matplotlib/pull/6262/files/66c0ed7a019fb734de3615302923839bc692ba12#r58300209

@pganssle pganssle Remove Python 2.6 compatibility function
3e6e8d1
Contributor

pganssle commented Apr 9, 2016

Do y'all want me to rebase this so it passes the appveyor tests, or should I just leave it as is?

@efiring efiring commented on the diff May 2, 2016

lib/matplotlib/dates.py
@@ -212,47 +212,31 @@ def _to_ordinalf(dt):
days, preserving hours, minutes, seconds and microseconds. Return value
is a :func:`float`.
"""
-
- if hasattr(dt, 'tzinfo') and dt.tzinfo is not None:
- delta = dt.tzinfo.utcoffset(dt)
- if delta is not None:
- dt -= delta
+ # Convert to UTC
+ tzi = getattr(dt, 'tzinfo', None)
+ if tzi is not None:
+ dt = dt.astimezone(UTC)
@efiring

efiring May 2, 2016

Owner

I don't think Python 2.7 datetime.date has the astimezone method.

@pganssle

pganssle May 2, 2016

Contributor

date also doesn't have tzinfo, so the astimezone line will never fire.

Though, actually, datetime.datetime.timetz() returns an object that has tzinfo and not astimezone. I'm not really sure how to deal with that, though. Are time objects an acceptable input here? I think the offset returned will be None if you try and get the utcoffset of a non-fixed-offset bare time object.

@efiring efiring commented on the diff May 16, 2016

lib/matplotlib/dates.py
base = float(dt.toordinal())
- if isinstance(dt, datetime.datetime):
- # Get a datetime object at midnight in the same time zone as dt.
- cdate = dt.date()
- midnight_time = datetime.time(0, 0, 0, tzinfo=dt.tzinfo)
+
+ # If it's sufficiently datetime-like, it will have a `date()` method
+ cdate = getattr(dt, 'date', lambda: None)()
+ if cdate is not None:
@efiring

efiring May 16, 2016

Owner

Based on the line above, I don't think cdate will ever be None--it can return None.

@WeatherGod

WeatherGod May 16, 2016

Member

it is calling the attribute it obtains before assigning to cdate.

On Mon, May 16, 2016 at 3:29 PM, Eric Firing notifications@github.com
wrote:

In lib/matplotlib/dates.py
#6262 (comment):

 base = float(dt.toordinal())
  • if isinstance(dt, datetime.datetime):
  •    # Get a datetime object at midnight in the same time zone as dt.
    
  •    cdate = dt.date()
    
  •    midnight_time = datetime.time(0, 0, 0, tzinfo=dt.tzinfo)
    
  • If it's sufficiently datetime-like, it will have a date() method

  • cdate = getattr(dt, 'date', lambda: None)()
  • if cdate is not None:

Based on the line above, I don't think cdate will ever be None--it can
return None.


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/matplotlib/matplotlib/pull/6262/files/3e6e8d1324e4aac4c0cf08dcffab451e1ee5f525#r63410423

@pganssle

pganssle May 20, 2016

Contributor

I can be more explicit about this if you'd prefer.

Contributor

pganssle commented Jul 25, 2016

Pinging - any update on this PR?

tacaswell closed this Jul 25, 2016

tacaswell reopened this Jul 25, 2016

@tacaswell tacaswell added needs_review and removed needs_review labels Jul 25, 2016

@tacaswell tacaswell merged commit 0a31b36 into matplotlib:master Jul 26, 2016

2 of 3 checks passed

continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.7%) to 70.344%
Details

tacaswell removed the needs_review label Jul 26, 2016

@tacaswell tacaswell added a commit that referenced this pull request Jul 26, 2016

@tacaswell tacaswell Merge pull request #6262 from pganssle/fix_date2num_dst
FIX: Properly handle UTC conversion in date2num
64756ee
Owner

tacaswell commented Jul 26, 2016

backported to v2.x as 64756ee

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment