Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataFrame.iloc raises ValueError: buffer source array is read-only when applied to read-only indices #17192

lesteve opened this issue Aug 7, 2017 · 1 comment


Copy link

commented Aug 7, 2017

Snippet reproducing the problem:

import numpy as np
import pandas as pd

df = pd.DataFrame({'data': np.ones(100, dtype='float64')})
indices = np.array([1, 3, 6])
indices.flags.writeable = False

This seems like a variation of #10043 (fixed by #10070). The difference here is that the indices is read-only not the numpy arrays inside the DataFrame. For completeness this problem was seen in scikit-learn as a by-product of debugging scikit-learn/scikit-learn#9483. We can very likely work-around the problem on the scikit-learn side by copying the indices if they are read-only.

Full stack-trace:

ValueError                                Traceback (most recent call last)
<ipython-input-6-fb86e43fb729> in <module>()
      5 indices = np.array([1, 3, 6])
      6 indices.flags.writeable = False
----> 7 df.iloc[indices]

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in __getitem__(self, key)
   1326         else:
   1327             key = com._apply_if_callable(key, self.obj)
-> 1328             return self._getitem_axis(key, axis=0)
   1330     def _is_scalar_access(self, key):

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in _getitem_axis(self, key, axis)
   1736         # a list of integers
   1737         elif is_list_like_indexer(key):
-> 1738             return self._get_list_axis(key, axis=axis)
   1740         # a single integer

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in _get_list_axis(self, key, axis)
   1713         """
   1714         try:
-> 1715             return self.obj.take(key, axis=axis, convert=False)
   1716         except IndexError:
   1717             # re-raise with different error message

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in take(self, indices, axis, convert, is_copy, **kwargs)                                                                                                 
   1926         new_data = self._data.take(indices,
   1927                                    axis=self._get_block_manager_axis(axis),
-> 1928                                    convert=True, verify=True)
   1929         result = self._constructor(new_data).__finalize__(self)

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in take(self, indexer, axis, verify, convert)
   4009         new_labels = self.axes[axis].take(indexer)
   4010         return self.reindex_indexer(new_axis=new_labels, indexer=indexer,
-> 4011                                     axis=axis, allow_dups=True)
   4013     def merge(self, other, lsuffix='', rsuffix=''):

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in reindex_indexer(self, new_axis, indexer, axis, fill_value, allow_dups, copy)
   3895             new_blocks = [blk.take_nd(indexer, axis=axis, fill_tuple=(
   3896                 fill_value if fill_value is not None else blk.fill_value,))
-> 3897                 for blk in self.blocks]
   3899         new_axes = list(self.axes)

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in <listcomp>(.0)
   3895             new_blocks = [blk.take_nd(indexer, axis=axis, fill_tuple=(
   3896                 fill_value if fill_value is not None else blk.fill_value,))
-> 3897                 for blk in self.blocks]
   3899         new_axes = list(self.axes)

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in take_nd(self, indexer, axis, new_mgr_locs, fill_tuple)
   1044             fill_value = fill_tuple[0]
   1045             new_values = algos.take_nd(values, indexer, axis=axis,
-> 1046                                        allow_fill=True, fill_value=fill_value)
   1048         if new_mgr_locs is None:

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/core/ in take_nd(arr, indexer, axis, out, fill_value, mask_info, allow_fill)
   1469     func = _get_take_nd_function(arr.ndim, arr.dtype, out.dtype, axis=axis,
   1470                                  mask_info=mask_info)
-> 1471     func(arr, indexer, out, fill_value)
   1473     if flip_order:

pandas/_libs/algos_take_helper.pxi in pandas._libs.algos.take_2d_axis0_float64_float64 (pandas/_libs/algos.c:110417)()

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/_libs/ in View.MemoryView.memoryview_cwrapper (pandas/_libs/algos.c:124730)()

/volatile/le243287/miniconda3/lib/python3.6/site-packages/pandas/_libs/ in View.MemoryView.memoryview.__cinit__ (pandas/_libs/algos.c:120965)()

ValueError: buffer source array is read-only

Pandas version information:

INSTALLED VERSIONS ------------------ commit: None python: python-bits: 64 OS: Linux OS-release: 4.4.0-83-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8

pandas: 0.20.3
pytest: 3.1.3
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.26
numpy: 1.13.1
scipy: 0.19.1
xarray: None
IPython: 6.1.0
sphinx: 1.6.2
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None


This comment has been minimized.

Copy link

commented Aug 7, 2017

It'd be nice if this were addressed in Cython, xref cython/cython#1605

But until then could do something similar to #10070

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
3 participants
You can’t perform that action at this time.