Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent behaviour of .ix between list and scalar key with (missing) ints #20753

Closed
toobaz opened this Issue Apr 20, 2018 · 2 comments

Comments

Projects
None yet
3 participants
@toobaz
Copy link
Member

commented Apr 20, 2018

Code Sample, a copy-pastable example if possible

In [2]: pd.Series(range(3)).ix[5]
/usr/bin/ipython3:1: DeprecationWarning: 
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
  #! /bin/sh
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/home/nobackup/repo/pandas/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3017             try:
-> 3018                 return self._engine.get_loc(key)
   3019             except KeyError:

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5729)()
    138 
--> 139     cpdef get_loc(self, object val):
    140         if is_definitely_invalid_key(val):

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5575)()
    160         try:
--> 161             return self.mapping.get_item(val)
    162         except (TypeError, ValueError):

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15398)()
    957 
--> 958     cpdef get_item(self, int64_t val):
    959         cdef khiter_t k

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15342)()
    963         else:
--> 964             raise KeyError(val)
    965 

KeyError: 5

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
/home/nobackup/repo/pandas/pandas/core/indexing.py in _get_label(self, label, axis)
    131             try:
--> 132                 return self.obj._xs(label, axis=axis)
    133             except:

/home/nobackup/repo/pandas/pandas/core/generic.py in xs(self, key, axis, level, drop_level)
   2991         else:
-> 2992             loc = self.index.get_loc(key)
   2993 

/home/nobackup/repo/pandas/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3019             except KeyError:
-> 3020                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   3021 

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5729)()
    138 
--> 139     cpdef get_loc(self, object val):
    140         if is_definitely_invalid_key(val):

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5575)()
    160         try:
--> 161             return self.mapping.get_item(val)
    162         except (TypeError, ValueError):

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15398)()
    957 
--> 958     cpdef get_item(self, int64_t val):
    959         cdef khiter_t k

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15342)()
    963         else:
--> 964             raise KeyError(val)
    965 

KeyError: 5

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-2-bba00cd402c6> in <module>()
----> 1 pd.Series(range(3)).ix[5]

/home/nobackup/repo/pandas/pandas/core/indexing.py in __getitem__(self, key)
    118 
    119             key = com._apply_if_callable(key, self.obj)
--> 120             return self._getitem_axis(key, axis=axis)
    121 
    122     def _get_label(self, label, axis=None):

/home/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1104                     return self._get_loc(key, axis=axis)
   1105 
-> 1106             return self._get_label(key, axis=axis)
   1107 
   1108     def _getitem_iterable(self, key, axis=None):

/home/nobackup/repo/pandas/pandas/core/indexing.py in _get_label(self, label, axis)
    132                 return self.obj._xs(label, axis=axis)
    133             except:
--> 134                 return self.obj[label]
    135         elif isinstance(label, tuple) and isinstance(label[axis], slice):
    136             raise IndexingError('no slices here, handle elsewhere')

/home/nobackup/repo/pandas/pandas/core/series.py in __getitem__(self, key)
    753         key = com._apply_if_callable(key, self)
    754         try:
--> 755             result = self.index.get_value(self, key)
    756 
    757             if not is_scalar(result):

/home/nobackup/repo/pandas/pandas/core/indexes/base.py in get_value(self, series, key)
   3051         try:
   3052             return self._engine.get_value(s, k,
-> 3053                                           tz=getattr(series.dtype, 'tz', None))
   3054         except KeyError as e1:
   3055             if len(self) > 0 and self.inferred_type in ['integer', 'boolean']:

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_value (pandas/_libs/index.c:4856)()
    103         return val in self.mapping
    104 
--> 105     cpdef get_value(self, ndarray arr, object key, object tz=None):
    106         """
    107         arr : 1-dimensional ndarray

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_value (pandas/_libs/index.c:4539)()
    111             void* data_ptr
    112 
--> 113         loc = self.get_loc(key)
    114         if PySlice_Check(loc) or cnp.PyArray_Check(loc):
    115             return arr[loc]

/home/nobackup/repo/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5575)()
    159 
    160         try:
--> 161             return self.mapping.get_item(val)
    162         except (TypeError, ValueError):
    163             raise KeyError(val)

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15398)()
    956                                        sizeof(uint32_t)) # flags
    957 
--> 958     cpdef get_item(self, int64_t val):
    959         cdef khiter_t k
    960         k = kh_get_int64(self.table, val)

/home/nobackup/repo/pandas/pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item (pandas/_libs/hashtable.c:15342)()
    962             return self.table.vals[k]
    963         else:
--> 964             raise KeyError(val)
    965 
    966     cpdef set_item(self, int64_t key, Py_ssize_t val):

KeyError: 5

In [3]: pd.Series(range(3)).ix[[5]]
/usr/bin/ipython3:1: DeprecationWarning: 
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
  #! /bin/sh
Out[3]: 
5   NaN
dtype: float64

Problem description

Already fixed, opening the issue just for xref from the test/whatsnew.

Expected Output

An error

Output of pd.show_versions()

INSTALLED VERSIONS

commit: d04b746
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-6-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: it_IT.UTF-8

pandas: 0.23.0.dev0+754.gd04b7464d
pytest: 3.5.0
pip: 9.0.1
setuptools: 39.0.1
Cython: 0.25.2
numpy: 1.14.1
scipy: 0.19.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.5.6
patsy: 0.5.0
dateutil: 2.7.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.0dev
tables: 3.3.0
numexpr: 2.6.1
feather: 0.3.1
matplotlib: 2.0.0
openpyxl: 2.3.0
xlrd: 1.0.0
xlwt: 1.3.0
xlsxwriter: 0.9.6
lxml: 4.1.1
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: 0.2.1

@jorisvandenbossche

This comment has been minimized.

Copy link
Member

commented Apr 20, 2018

This already is like that for a long time (with which I don't want to say that it is clearly an inconsistency with .loc), and given that .ix is deprecated, not sure if it is worth to fix it.

@jorisvandenbossche

This comment has been minimized.

Copy link
Member

commented Apr 20, 2018

However, getitem has the same problem, and this is probably something more worth to fix:

In [15]: s = pd.Series(range(3))

In [16]: s[[5]]
Out[16]: 
5   NaN
dtype: float64

toobaz added a commit to toobaz/pandas that referenced this issue Apr 20, 2018

toobaz added a commit to toobaz/pandas that referenced this issue Apr 22, 2018

toobaz added a commit to toobaz/pandas that referenced this issue Apr 22, 2018

@jreback jreback added this to the 0.23.0 milestone May 1, 2018

TomAugspurger added a commit that referenced this issue May 1, 2018

Emit warning for missing labels in Multiindex.loc[[...]] (and more) (#…
…20770)

* API: emit warning to raise KeyError in the future for missing keys also for MultiIndex

closes #17758
closes #20748
closes #20753
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.