Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Regression: DatetimeIndex ignores timezones #11736
In 0.17.0, it's now ignoring timezones:
This slipped thru since the tests are very simple and don't cover many cases:
I'll attempt a patch later but if someone can give pointers where the breaking change happened I appreciate.
so there are actually quite a number of tests, but not in formatting; in fact this has nothing to do with formatting.
In 0.17.1, this 'works'
but is actually 'wrong'
is the correct result. However, this parsing of a tz-aware then 'setting' its value via the tz arg is very very odd.
This is why
E.g. what exactly do you mean by having a tz-aware date AND providing a tz? should it be localized, renamed, or converted?
I think the
Basically, pre 0.17.x behavior was correct - it doesn't throw away the tz info neither tries to covert when fed a TZ-aware ISO string. So, if I had analogous function using standard datetime API, it would be equivalent to:
Which is sensible behavior for such ISO datetime parser .
0.17.0 behavior is to ignore the TZ and create a naive datetime series, which is bad because now we've lost tz information from the string. And I've now learned 0.17.1 behavior is even worse, since it's converting and keeping the tz info. Either you convert and turn into naive (drop tz info) or you localize (add the tz info), but never a mix of both.
I see. There are tests here (https://github.com/pydata/pandas/blob/master/pandas/tests/test_series.py#L112) but are not covering parsing an ISO datetime string either.
The parsing seems to be happening here (https://github.com/pydata/pandas/blob/master/pandas/tseries/tools.py#L279), I'll come up with some test cases.
@hcarvalhoalves no, the pre 0.17.x behavior is wrong. It is VERY confusing. This is why the main entry point is
I don't know if passing a
Since I have never seen this I doubt there are tests (or I would have had to resolve this when I added this feature).
You are essentially 'naming' a tz. This is not a normal things to do. Either you have a string that has a tz in it with an offset, in which case you convert to UTC, THEN convert it, or you construct it in local time, then localize it.
There are NO other operations.
I see. You mean the expected usage is this?
If this is really intended and not a regression I'll close this specific issue - and another issue could be opened to not have DatetimeIndex fail silently when there's an offset in the string.
no I would leave this open. I think this path is not very well tested and needs exercising / clarification.
My point though is if you have text data,