Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataFrame.iloc raises ValueError: buffer source array is read-only when applied to read-only indices #17192

Closed
lesteve opened this issue Aug 7, 2017 · 1 comment

Comments

@lesteve
Copy link
Contributor

commented Aug 7, 2017

Snippet reproducing the problem:

import numpy as np
import pandas as pd

df = pd.DataFrame({'data': np.ones(100, dtype='float64')})
indices = np.array([1, 3, 6])
indices.flags.writeable = False
df.iloc[indices]

This seems like a variation of #10043 (fixed by #10070). The difference here is that the indices is read-only not the numpy arrays inside the DataFrame. For completeness this problem was seen in scikit-learn as a by-product of debugging scikit-learn/scikit-learn#9483. We can very likely work-around the problem on the scikit-learn side by copying the indices if they are read-only.

Full stack-trace:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-fb86e43fb729> in <module>()
      5 indices = np.array([1, 3, 6])
      6 indices.flags.writeable = False
----> 7 df.iloc[indices]

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/indexing.py in __getitem__(self, key)
   1326         else:
   1327             key = com._apply_if_callable(key, self.obj)
-> 1328             return self._getitem_axis(key, axis=0)
   1329 
   1330     def _is_scalar_access(self, key):

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1736         # a list of integers
   1737         elif is_list_like_indexer(key):
-> 1738             return self._get_list_axis(key, axis=axis)
   1739 
   1740         # a single integer

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/indexing.py in _get_list_axis(self, key, axis)
   1713         """
   1714         try:
-> 1715             return self.obj.take(key, axis=axis, convert=False)
   1716         except IndexError:
   1717             # re-raise with different error message

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/generic.py in take(self, indices, axis, convert, is_copy, **kwargs)                                                                                                 
   1926         new_data = self._data.take(indices,
   1927                                    axis=self._get_block_manager_axis(axis),
-> 1928                                    convert=True, verify=True)
   1929         result = self._constructor(new_data).__finalize__(self)
   1930 

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/internals.py in take(self, indexer, axis, verify, convert)
   4009         new_labels = self.axes[axis].take(indexer)
   4010         return self.reindex_indexer(new_axis=new_labels, indexer=indexer,
-> 4011                                     axis=axis, allow_dups=True)
   4012 
   4013     def merge(self, other, lsuffix='', rsuffix=''):

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/internals.py in reindex_indexer(self, new_axis, indexer, axis, fill_value, allow_dups, copy)
   3895             new_blocks = [blk.take_nd(indexer, axis=axis, fill_tuple=(
   3896                 fill_value if fill_value is not None else blk.fill_value,))
-> 3897                 for blk in self.blocks]
   3898 
   3899         new_axes = list(self.axes)

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/internals.py in <listcomp>(.0)
   3895             new_blocks = [blk.take_nd(indexer, axis=axis, fill_tuple=(
   3896                 fill_value if fill_value is not None else blk.fill_value,))
-> 3897                 for blk in self.blocks]
   3898 
   3899         new_axes = list(self.axes)

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/internals.py in take_nd(self, indexer, axis, new_mgr_locs, fill_tuple)
   1044             fill_value = fill_tuple[0]
   1045             new_values = algos.take_nd(values, indexer, axis=axis,
-> 1046                                        allow_fill=True, fill_value=fill_value)
   1047 
   1048         if new_mgr_locs is None:

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/algorithms.py in take_nd(arr, indexer, axis, out, fill_value, mask_info, allow_fill)
   1469     func = _get_take_nd_function(arr.ndim, arr.dtype, out.dtype, axis=axis,
   1470                                  mask_info=mask_info)
-> 1471     func(arr, indexer, out, fill_value)
   1472 
   1473     if flip_order:

pandas/_libs/algos_take_helper.pxi in pandas._libs.algos.take_2d_axis0_float64_float64 (pandas/_libs/algos.c:110417)()

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/_libs/algos.cpython-36m-x86_64-linux-gnu.so in View.MemoryView.memoryview_cwrapper (pandas/_libs/algos.c:124730)()

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/_libs/algos.cpython-36m-x86_64-linux-gnu.so in View.MemoryView.memoryview.__cinit__ (pandas/_libs/algos.c:120965)()

ValueError: buffer source array is read-only

Pandas version information:

INSTALLED VERSIONS ------------------https://github.com/scikit-learn/scikit-learn/issues/9483 commit: None python: 3.6.2.final.0 python-bits: 64 OS: Linux OS-release: 4.4.0-83-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8

pandas: 0.20.3
pytest: 3.1.3
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.26
numpy: 1.13.1
scipy: 0.19.1
xarray: None
IPython: 6.1.0
sphinx: 1.6.2
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None

@chris-b1

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2017

It'd be nice if this were addressed in Cython, xref cython/cython#1605

But until then could do something similar to #10070

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.