Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexing MultiIndex with NDFrame indexer fails if index of indexer does not contain 0 #15424

Closed
toobaz opened this issue Feb 16, 2017 · 7 comments
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex
Milestone

Comments

@toobaz
Copy link
Member

toobaz commented Feb 16, 2017

Code Sample, a copy-pastable example if possible

In [2]: d = pd.DataFrame([[1, 1, 3],
   ...:                   [1, 2, 4],
   ...:                   [2, 2, 5]], columns=['a', 'b', 'c'])

In [3]: d.set_index(['a', 'b']).loc[pd.Series([1, 2])]
Out[3]: 
     c
a b   
1 1  3
  2  4
2 2  5

In [4]: d.set_index(['a', 'b']).loc[pd.Series([1, 2], index=[2,0])]
Out[4]: 
     c
a b   
1 1  3
  2  4
2 2  5

In [5]: d.set_index(['a', 'b']).loc[pd.Series([1, 2], index=[2,3])]
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-5-a23ad44827f6> in <module>()
----> 1 d.set_index(['a', 'b']).loc[pd.Series([1, 2], index=[2,3])]

/home/pietro/nobackup/repo/pandas/pandas/core/indexing.py in __getitem__(self, key)
   1339         else:
   1340             key = com._apply_if_callable(key, self.obj)
-> 1341             return self._getitem_axis(key, axis=0)
   1342 
   1343     def _is_scalar_access(self, key):

/home/pietro/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1527             if isinstance(labels, MultiIndex):
   1528                 if (not isinstance(key, tuple) and len(key) > 1 and
-> 1529                         not isinstance(key[0], tuple)):
   1530                     if isinstance(key, ABCSeries):
   1531                         # GH 14730

/home/pietro/nobackup/repo/pandas/pandas/core/series.py in __getitem__(self, key)
    604         key = com._apply_if_callable(key, self)
    605         try:
--> 606             result = self.index.get_value(self, key)
    607 
    608             if not is_scalar(result):

/home/pietro/nobackup/repo/pandas/pandas/indexes/base.py in get_value(self, series, key)
   2304         try:
   2305             return self._engine.get_value(s, k,
-> 2306                                           tz=getattr(series.dtype, 'tz', None))
   2307         except KeyError as e1:
   2308             if len(self) > 0 and self.inferred_type in ['integer', 'boolean']:

/home/pietro/nobackup/repo/pandas/pandas/index.pyx in pandas.index.IndexEngine.get_value (pandas/index.c:3992)()

/home/pietro/nobackup/repo/pandas/pandas/index.pyx in pandas.index.IndexEngine.get_value (pandas/index.c:3689)()

/home/pietro/nobackup/repo/pandas/pandas/index.pyx in pandas.index.IndexEngine.get_loc (pandas/index.c:4688)()

/home/pietro/nobackup/repo/pandas/pandas/src/hashtable_class_helper.pxi in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:13370)()

/home/pietro/nobackup/repo/pandas/pandas/src/hashtable_class_helper.pxi in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:13308)()

KeyError: 0

Problem description

The index of the indexer should not matter at all... and indeed there is no bug when indexing a (non-Multi)Index:

In [6]: d.set_index(['a']).loc[pd.Series([1, 2], index=[2,3])]
Out[6]: 
   b  c
a      
1  1  3
1  2  4
2  2  5

Expected Output

The same as d.set_index(['a', 'b']).loc[pd.Series([1, 2])].

A PR is on its way.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.5.2.final.0 python-bits: 64 OS: Linux OS-release: 4.7.0-1-amd64 machine: x86_64 processor: byteorder: little LC_ALL: None LANG: it_IT.utf8 LOCALE: it_IT.UTF-8

pandas: 0.19.0+464.g0b28e5a
pytest: None
pip: 8.1.2
setuptools: 28.0.0
Cython: 0.23.4
numpy: 1.12.0
scipy: 0.18.1
xarray: None
IPython: 5.1.0.dev
sphinx: 1.4.8
patsy: 0.3.0-dev
dateutil: 2.5.3
pytz: 2015.7
blosc: None
bottleneck: 1.2.0
tables: 3.2.2
numexpr: 2.6.0
feather: None
matplotlib: 2.0.0rc2
openpyxl: None
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: None
bs4: 4.5.1
html5lib: 0.999
httplib2: 0.9.1
apiclient: 1.5.2
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.8
s3fs: None
pandas_datareader: 0.2.1

@toobaz toobaz mentioned this issue Feb 16, 2017
5 tasks
@jreback jreback added Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex labels Feb 16, 2017
@jreback
Copy link
Contributor

jreback commented Feb 16, 2017

ok this looks like a bug. IIRC we had a similar issue (maybe was yours)?

@jreback jreback added the Bug label Feb 16, 2017
@jreback jreback added this to the 0.20.0 milestone Feb 16, 2017
@toobaz
Copy link
Member Author

toobaz commented Feb 16, 2017

There was #14730 (this bug might actually be a side effect of its fix, but I didn't check).

@jreback
Copy link
Contributor

jreback commented Feb 16, 2017

@toobaz ok, can you add those examples as confirmation tests (they look similar). if they pass, then add that issue as well in the whatsnew note.

@toobaz
Copy link
Member Author

toobaz commented Feb 16, 2017

I'm lost... have you seen the PR? (#15425)

@jreback
Copy link
Contributor

jreback commented Feb 16, 2017

of course that is where they should be added

@toobaz
Copy link
Member Author

toobaz commented Feb 16, 2017

OK, I asked because that issue is already in the whatsnew note

So: you suggest I add to the tests I already wrote the ones in this bug report, right? (except for the data content, they are exactly the same...)

@jreback
Copy link
Contributor

jreback commented Feb 16, 2017

ahh ok then

@toobaz toobaz mentioned this issue Mar 8, 2017
4 tasks
AnkurDedania pushed a commit to AnkurDedania/pandas that referenced this issue Mar 21, 2017
…xers

closes pandas-dev#15424
closes pandas-dev#15434

Author: Pietro Battiston <me@pietrobattiston.it>

Closes pandas-dev#15425 from toobaz/mi_indexing and squashes the following commits:

2ba2d5d [Pietro Battiston] Updated comment
900e3ce [Pietro Battiston] whatsnew
8467b57 [Pietro Battiston] Tests for previous commit
17209f3 [Pietro Battiston] BUG: support indexing MultiIndex with 1-D array
7606114 [Pietro Battiston] Whatsnew
0b719f5 [Pietro Battiston] Test for previous commit
1f2f385 [Pietro Battiston] BUG: Fix indexing MultiIndex with Series with 0 not index
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants