Skip to content

Why does recarray assignment fail: "cannot call setfield on an object array"? (Trac #2006) #2599

Closed
numpy-gitbot opened this Issue Oct 19, 2012 · 3 comments

3 participants

@numpy-gitbot

Original ticket http://projects.scipy.org/numpy/ticket/2006 on 2011-12-23 by trac user kevin000, assigned to unknown.

Using a recarray with member assignment fails if any column uses the object data type. I don't understand why this fails, especially since the dictionary style assignment works.

Brief example showing what recarray assignments succeed or fail

import numpy as np

recarray with integer datatypes

dt = np.dtype([('foo', 'i8'), ('bar', 'i8')])
r = np.zeros((1,3), dtype=dt).view(np.recarray)
r['foo'] = np.array([1, 2, 3]) # OK
r.foo = np.array([1, 2, 3]) # OK

recarray with an object datatype

dt = np.dtype([('foo', 'i8'), ('bar', 'O8')])
r = np.zeros((1,3), dtype=dt).view(np.recarray)
r['foo'] = np.array([1, 2, 3]) # OK
r.foo = np.array([1, 2, 3]) # RuntimeError

For the relevant C code search for "cannot call setfield on an object array" in this file:

https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/methods.c

Thanks!

Kevin

@cmarquardt

The same problem occurs in 1.6.2 and 1.7.0b2.

The (big) change that introduced this behavior is 2635398 and apparently has introduced the error condition on purpose; there's even a test checking that using the setfield() method on an (ordinary) object array throws a RuntimeError.

As modifying object sub-arrays using the dictionary approach works fine, the easiest solution seems to be to replace the setfield() method in the setattr() method of recarray with ndarray's setitem() method; I just attached a patch against the 1.7.0b2 distribution of numpy which runns successfully through the same tests as the original distribution does.

I know way too little about numpy internals to judge if this patch is consistent with the numpy conventions etc.; but keeping this bug in recarrays somehow make the whole recarray class superfluous if not fixed IMHO.

Thanks for considering the patch (which is below),

Christian.

*** numpy-1.7.0b2.orig/numpy/core//records.py   2012-09-04 23:31:31.000000000 +0200
--- numpy-1.7.0b2/numpy/core//records.py        2012-12-02 23:20:00.757234770 +0100
***************
*** 451,457 ****
              res = fielddict[attr][:2]
          except (TypeError, KeyError):
              raise AttributeError("record array has no attribute %s" % attr)
!         return self.setfield(val, *res)
  
      def __getitem__(self, indx):
          obj = ndarray.__getitem__(self, indx)
--- 451,457 ----
              res = fielddict[attr][:2]
          except (TypeError, KeyError):
              raise AttributeError("record array has no attribute %s" % attr)
!         return ndarray.__setitem__(self, attr, val)
  
      def __getitem__(self, indx):
          obj = ndarray.__getitem__(self, indx)

@charris
NumPy member
charris commented Feb 21, 2014

There are a bunch of tickets that look related to this. The patch has not been applied as of 1.9-devel.

@charris
NumPy member
charris commented Feb 21, 2014

Easy fix if the patch is good and we want that behavior.

@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 4, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
907c541
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 4, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
b7fece9
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 4, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
4e3841c
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 4, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
8171464
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 4, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
34bdab2
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 5, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
00a31fe
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 5, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
432ddcd
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 5, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
f3903e3
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 5, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
958bf63
@jaimefrio jaimefrio pushed a commit that closed this issue Jun 5, 2015
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
e2cd6fa
@jaimefrio jaimefrio closed this in e2cd6fa Jun 5, 2015
@cmarquardt cmarquardt added a commit to cmarquardt/Python-packages that referenced this issue Oct 19, 2015
@cmarquardt cmarquardt Removed the numpy-path.h script.
numpy-patch fixed numpy/numpy#2599,
the latter being properly solved in numpy v1.10.0. As homebrew
has been updated to that version, the manual patch isn't
required any more.
e014534
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Oct 26, 2015
@ahaldane ahaldane BUG: revert view safety checks
Because of slowdowns caused by the view safety checks introduced in
 #5548 they are removed here for 1.10. The plan is to reintroduce
a better version of the checks in 1.11.

Many changes made since #5548 are left in place, as they will be
needed in the future, and fix some minor bugs.

This re-opens #2599
585e2ec
@ahaldane ahaldane added a commit to ahaldane/numpy that referenced this issue Oct 26, 2015
@ahaldane ahaldane BUG: revert view safety checks
Because of slowdowns caused by the view safety checks introduced in
 #5548 they are removed here for 1.10. The plan is to reintroduce
a better version of the checks in 1.11.

Many changes made since #5548 are left in place, as they will be
needed in the future, and fix some minor bugs.

This re-opens #2599
6602a20
@daniel-perry daniel-perry added a commit to daniel-perry/numpy that referenced this issue Jun 2, 2016
@ahaldane ahaldane ENH: structured datatype safety checks
Previously views of structured arrays containing objects were completely
disabled.  This commit adds more lenient check for whether an object-array
view is allowed, and adds similar checks to getfield/setfield

Fixes #2346. Fixes #3256. Fixes #2599. Fixes #3253. Fixes #3286.
Fixes #5762.
a984090
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.