Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error while deleting/dropping a series/dataframe entry with timezone aware timestamps. #21761

Closed
elboerto opened this issue Jul 6, 2018 · 7 comments

Comments

Projects
None yet
5 participants
@elboerto
Copy link

commented Jul 6, 2018

Code Sample, a copy-pastable example if possible

import pandas as pd
start = pd.Timestamp('20170101', tz='Europe/Berlin')
end = pd.Timestamp('20180101', tz='Europe/Berlin')
index = pd.date_range(start, end, freq='15min')
data = [1 for x in range(len(index))]
series = pd.Series(index=index, data=data)
dataframe = pd.DataFrame(series)
timestamp = pd.Timestamp('201701011515', tz='Europe/Berlin')

# Working
series[timestamp]
dataframe.loc[timestamp]

# Errors
del series[timestamp]
dataframe.drop(timestamp)
series.drop(timestamp)

Problem description

It not possible to delete/drop a series/dataframe entry with timezone aware timestamps, while accessing them is possible.

Expected Output

Deletion of the respective entry.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.4.final.0 python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 142 Stepping 9, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None pandas: 0.23.1 pytest: 3.6.2 pip: 10.0.1 setuptools: 39.2.0 Cython: None numpy: 1.14.5 scipy: 1.1.0 pyarrow: None xarray: None IPython: 6.4.0 sphinx: None patsy: None dateutil: 2.7.3 pytz: 2018.5 blosc: None bottleneck: None tables: None numexpr: None feather: None matplotlib: 2.2.2 openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: None html5lib: None sqlalchemy: None pymysql: None psycopg2: None jinja2: None s3fs: None fastparquet: None pandas_gbq: None pandas_datareader: None
@gfyoung

This comment has been minimized.

Copy link
Member

commented Jul 6, 2018

@elboerto : Can you try your code against master ? I can't seem to replicate this there.

@gfyoung gfyoung added the Timeseries label Jul 6, 2018

@gnikonorov

This comment has been minimized.

Copy link

commented Jul 6, 2018

I was able to reproduce this issue with master's code. I have just installed pandas for development with anaconda as outlined here and received the following output when I ran the code:

(pandas-dev) ✔ ~/projects/pandas-gnikonorov/test [issue_21761|…1]
18:58 $ cat test_21761.py
import pandas as pd
start = pd.Timestamp('20170101', tz='Europe/Berlin')
end = pd.Timestamp('20180101', tz='Europe/Berlin')
index = pd.date_range(start, end, freq='15min')
data = [1 for x in range(len(index))]
series = pd.Series(index=index, data=data)
dataframe = pd.DataFrame(series)
timestamp = pd.Timestamp('201701011515', tz='Europe/Berlin')

# Working
series[timestamp]
dataframe.loc[timestamp]

# Errors
del series[timestamp]
dataframe.drop(timestamp)
series.drop(timestamp)
(pandas-dev) ✔ ~/projects/pandas-gnikonorov/test [issue_21761|…1]
18:58 $ python test_21761.py
Traceback (most recent call last):
  File "test_21761.py", line 15, in <module>
    del series[timestamp]
  File "/home/gnikonorov/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/pandas/core/generic.py", line 2740, in __delitem__
    self._data.delete(key)
  File "/home/gnikonorov/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/pandas/core/internals.py", line 4785, in delete
    self.axes[0] = self.axes[0].delete(loc)
  File "/home/gnikonorov/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/pandas/core/indexes/datetimes.py", line 2225, in delete
    return DatetimeIndex(new_dates, name=self.name, freq=freq, tz=self.tz)
  File "/home/gnikonorov/anaconda3/envs/pandas-dev/lib/python3.6/site-packages/pandas/core/indexes/datetimes.py", line 445, in __new__
    ambiguous=ambiguous)
  File "pandas/_libs/tslibs/conversion.pyx", line 977, in pandas._libs.tslibs.conversion.tz_localize_to_utc
pytz.exceptions.AmbiguousTimeError: Cannot infer dst time from '2017-10-29 02:00:00', try using the 'ambiguous' argument
(pandas-dev) ✘-1 ~/projects/pandas-gnikonorov/test [issue_21761|…1]
18:58 $

I would like to attempt to fix this issue, if possible.

@gfyoung

This comment has been minimized.

Copy link
Member

commented Jul 6, 2018

@gnikonorov : As I said earlier, I am (un)fortunately unable to replicate those errors. What version of pandas are you using i.e. what is your output when you run:

import pandas as pd
pd.__version__
@gnikonorov

This comment has been minimized.

Copy link

commented Jul 7, 2018

@gfyoung, I am running v0.24.0.dev0+243.g30eb48c.

(pandas-dev) ✘-1 ~/projects/pandas-gnikonorov [issue_21761|…2]
20:23 $ python
Python 3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 17:14:51)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> print(pandas.__version__)
0.24.0.dev0+243.g30eb48c
>>>
@gfyoung

This comment has been minimized.

Copy link
Member

commented Jul 7, 2018

@gnikonorov : Hmmm...that's quite weird.

cc @jreback @jorisvandenbossche @TomAugspurger : Are any of you able to replicate this by any chance?

@gfyoung gfyoung added Can't Repro and removed Testing labels Jul 7, 2018

@elboerto

This comment has been minimized.

Copy link
Author

commented Jul 9, 2018

Version

>>> import pandas
>>> print(pandas.__version__)
0.24.0.dev0+237.g3273309d2

Output:

(pandas-dev) C:\Users\Robert\Documents\Github\pandas-elboerto>python test/test_21761.py
Traceback (most recent call last):
  File "test/test_21761.py", line 15, in <module>
    del series[timestamp]
  File "C:\Users\Robert\AppData\Local\conda\conda\envs\pandas-dev\lib\site-packages\pandas\core\generic.py", line 2740, in __delitem__
    self._data.delete(key)
  File "C:\Users\Robert\AppData\Local\conda\conda\envs\pandas-dev\lib\site-packages\pandas\core\internals.py", line 4785, in delete
    self.axes[0] = self.axes[0].delete(loc)
  File "C:\Users\Robert\AppData\Local\conda\conda\envs\pandas-dev\lib\site-packages\pandas\core\indexes\datetimes.py", line 2225, in delete
    return DatetimeIndex(new_dates, name=self.name, freq=freq, tz=self.tz)
  File "C:\Users\Robert\AppData\Local\conda\conda\envs\pandas-dev\lib\site-packages\pandas\core\indexes\datetimes.py", line 445, in __new__
    ambiguous=ambiguous)
  File "pandas/_libs/tslibs/conversion.pyx", line 977, in pandas._libs.tslibs.conversion.tz_localize_to_utc
    elif is_dst:
pytz.exceptions.AmbiguousTimeError: Cannot infer dst time from '2017-10-29 02:00:00', try using the 'ambiguous' argument
``
@TomAugspurger

This comment has been minimized.

Copy link
Contributor

commented Jul 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.