pd.to_datetime("2015-11-18 15:30:00+05:30").tz_localize('UTC').tz_convert('Asia/Kolkata') returns '2015-11-18 16:30:00+0530' #11708

Closed
PShiw opened this Issue Nov 26, 2015 · 8 comments

Comments

Projects
None yet
4 participants

PShiw commented Nov 26, 2015

running "pd.to_datetime("2015-11-18 15:30:00+05:30").tz_localize('UTC').tz_convert('Asia/Kolkata')" returns following o/p

Timestamp('2015-11-18 16:30:00+0530', tz='Asia/Kolkata')

It should have returned as Timestamp('2015-11-18 15:30:00+0530', tz='Asia/Kolkata') as no daylight saving is applicable in India.

Contributor

jreback commented Nov 26, 2015

pls show your versions of pytz and dateutil

PShiw commented Nov 26, 2015

I am using Python 3.4.3

pytz Version:- 2015.7

dateutil version :- 2.4.2

Contributor

rockg commented Nov 27, 2015

This is somewhat similar to #11481. As you see below, the tzinfo.utcoffset gets intepreted as 4:30 (I put in print statements to help debug) and so your subsequent operations are off. This is a bug as far as I'm concerned.

>>> pd.to_datetime("2015-11-18 15:30:00+05:30")
tzinfo.utcoffset = 4:30:00
Timestamp('2015-11-18 11:00:00')

jreback added this to the Next Major Release milestone Nov 27, 2015

Contributor

rockg commented Nov 27, 2015

And notice that if I remove the "0" in front of the offset, it works correctly as it goes into the dateutil parser (again the extra output is for debugging purposes).

>>> pd.to_datetime("2015-11-18 15:30:00+5:30")
ValueError: Error parsing datetime string "2015-11-18 15:30:00+5:30" at position 20
using parse_date 2015-11-18 15:30:00+5:30
Timestamp('2015-11-18 10:00:00')

PShiw commented Nov 28, 2015

@rockg

Please note that "o" in time stamp was returned by Pandas only. For details refer below code.

pd.to_datetime(1419031147, utc=True, unit = 's').tz_convert('Asia/Kolkata')
Timestamp('2014-12-20 04:49:07+0530', tz='Asia/Kolkata')

Member

sinhrks commented Nov 29, 2015

Needs to look more, but the below line looks problematic. offset_minute should be added to total, not subtracted.

After the change, we get:

pd.to_datetime("2015-11-18 15:30:00+05:30").tz_localize('UTC').tz_convert('Asia/Kolkata')
# Timestamp('2015-11-18 15:30:00+0530', tz='Asia/Kolkata')
Contributor

rockg commented Nov 29, 2015

I agree that does look suspicious and that you should be adding and not subtracting. I guess we have no tests with a tzoffset that contains minutes?

Contributor

rockg commented Dec 5, 2015

Further confirmed with an even better example:

With subtraction:

In [2]: pd.to_datetime("2015-11-18 15:45:00+05:45")
tzinfo.utcoffset = 4:15:00
Out[2]: Timestamp('2015-11-18 11:30:00')

With addition:

In [2]: pd.to_datetime("2015-11-18 15:45:00+05:45")
tzinfo.utcoffset = 5:45:00
Out[2]: Timestamp('2015-11-18 10:00:00')

I'll open up a PR.

@jreback jreback modified the milestone: 0.18.0, Next Major Release Dec 6, 2015

jreback closed this in #11774 Dec 7, 2015

@jreback jreback added a commit to mroeschke/pandas that referenced this issue Apr 8, 2017

@jreback jreback DOC: add doc-strings to tz_convert/tz_localize in tslib.pyx
TST: more tests, xref #15823, xref #11708
a7ee890

@jreback jreback added a commit that referenced this issue Apr 8, 2017

@mroeschke @jreback mroeschke + jreback BUG: Correct Timestamp localization with tz near DST (#11481) (#15934)
* BUG: Timestamp doesn't respect tz DST

closes #11481
closes #15777

* DOC: add doc-strings to tz_convert/tz_localize in tslib.pyx
TST: more tests, xref #15823, xref #11708
f35209e

@linebp linebp added a commit to linebp/pandas that referenced this issue Apr 17, 2017

@mroeschke @linebp mroeschke + linebp BUG: Correct Timestamp localization with tz near DST (#11481) (#15934)
* BUG: Timestamp doesn't respect tz DST

closes #11481
closes #15777

* DOC: add doc-strings to tz_convert/tz_localize in tslib.pyx
TST: more tests, xref #15823, xref #11708
8743970
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment