New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: ValueError when indexing using range only when length >= 1,000,000 #11652

Closed
nekobon opened this Issue Nov 19, 2015 · 3 comments

Comments

Projects
None yet
2 participants
@nekobon

nekobon commented Nov 19, 2015

From SO (http://stackoverflow.com/questions/33814223/strange-error-in-pandas-indexing-with-range-when-length-1-000-000)

Pandas raises a ValueError when assigning multiple values to a Series (or DataFrame) using range(x) where x > 1. This error is raised only when its length is one million or larger.

import pandas as pd

for x in [5, 999999, 1000000]:
    s = pd.Series(index=range(x))
    print('series length =', len(s))
    # assigning value with range(1), always works
    s.loc[range(1)] = 42 
    # reading values with range(x>1), always works
    _ = s.loc[range(2)] 
    # assigning values with range(x>1), fails only when len >= 1 million
    s.loc[range(2)] = 42 

Output:

series length = 5
series length = 999999
series length = 1000000
Traceback (most recent call last):
  File "<stdin>", line 9, in <module>
  File "/home/nekobon/.env_exp/lib/python3.4/site-packages/pandas/core/indexing.py", line 114, in __setitem__
    indexer = self._get_setitem_indexer(key)
  File "/home/nekobon/.env_exp/lib/python3.4/site-packages/pandas/core/indexing.py", line 109, in _get_setitem_indexer
    return self._convert_to_indexer(key, is_setter=True)
  File "/home/nekobon/.env_exp/lib/python3.4/site-packages/pandas/core/indexing.py", line 1042, in _convert_to_indexer
    return labels.get_loc(obj)
  File "/home/nekobon/.env_exp/lib/python3.4/site-packages/pandas/core/index.py", line 1692, in get_loc
    return self._engine.get_loc(_values_from_object(key))
  File "pandas/index.pyx", line 137, in pandas.index.IndexEngine.get_loc (pandas/index.c:3979)
  File "pandas/index.pyx", line 145, in pandas.index.IndexEngine.get_loc (pandas/index.c:3680)
  File "pandas/index.pyx", line 464, in pandas.index._bin_search (pandas/index.c:9124)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Tested on pandas 0.17.0 and python 3.4.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Nov 19, 2015

Contributor

odd, only happens on py3.

Contributor

jreback commented Nov 19, 2015

odd, only happens on py3.

@nekobon

This comment has been minimized.

Show comment
Hide comment
@nekobon

nekobon Nov 19, 2015

range(2) is a list in py2.x. s.loc[list[range(2)]] = 42 works fine with py3, too.
It seems to be failing only with range object.

nekobon commented Nov 19, 2015

range(2) is a list in py2.x. s.loc[list[range(2)]] = 42 works fine with py3, too.
It seems to be failing only with range object.

jreback added a commit to jreback/pandas that referenced this issue Nov 19, 2015

@jreback jreback added this to the 0.17.1 milestone Nov 19, 2015

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Nov 19, 2015

Contributor

note that this was never supported, but fixed in #11653

Contributor

jreback commented Nov 19, 2015

note that this was never supported, but fixed in #11653

jreback added a commit that referenced this issue Nov 20, 2015

yarikoptic added a commit to neurodebian/pandas that referenced this issue Dec 3, 2015

Merge tag 'v0.17.1' into debian
Version 0.17.1

* tag 'v0.17.1': (168 commits)
  add nbviewer link
  Revert "DOC: fix sponsor notice"
  DOC: a few touchups
  DOC: fix sponsor notice
  DOC: warnings and remove HTML
  COMPAT: compat of scalars on all platforms, xref pandas-dev#11638
  DOC: fix build errors/warnings
  DOC: whatsnew edits
  DOC: fix link syntax
  DOC: update release.rst / whatsnew edits
  BUG: fix col iteration in DataFrame.round, pandas-dev#11611
  DOC: Clarify foramtting
  BUG: pandas-dev#11638 return correct dtype for int and float
  BUG: pandas-dev#11637 fix to_csv incorrect output.
  DOC: sponsor notice
  BUG: indexing with a range , pandas-dev#11652
  Fix link to numexpr
  ENH: fixup tilde expansion, xref pandas-dev#11438
  ENH: tilde expansion for write output formatting functions, pandas-dev#11438
  DOC: fix up doc-string creations in generic.py
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment