'Nifti1Header' object has no attribute '_structarr' upon copy

[yarikoptic]

As reported originally on nipy-user
https://groups.google.com/forum/?fromgroups=#!topic/nipy-user/tDl4w2oN75U

    header_ = evds.a.imghdr.copy()
  File "/usr/lib/pymodules/python2.6/nibabel/nifti1.py", line 561, in copy
    self.binaryblock,
  File "/usr/lib/pymodules/python2.6/nibabel/wrapstruct.py", line 210, in binaryblock
    return self._structarr.tostring()
AttributeError: 'Nifti1Header' object has no attribute '_structarr'

and it is probably related to the trigger -- storing of this header into hdf5 using our PyMVPA hdf5 backend under nibabel 1.2.0 and then reloading it with nibabel 1.4.0, so per se it might not even be nibabel's issue... but that yet to be proven ;)

[BloodD]

I have met this error too~

[matthew-brett]

@yarikoptic @BloodD - can you make some test case that I could look at?

[BloodD]

@matthew-brett

----------------------------code------------------------------------------------------
import mvpa2.base.hdf5 as hd
from mvpa2.datasets.base import Dataset
from mvpa2.datasets.mri import map2nifti

#load a hdf5 matrix file or hdf5 with attribute 'a'
file = hd.h5load("fdt_matrix2.T.hdf5")
#create a Dataset instance
test_data = Dataset([])
#copy the attribute 'a' to test_data.a
test_data.a = file.a
#map the data_test to nifti file
img = map2nifti(test_data)

#then error comes

------------------------------bug---------------------------------------------------------
AttributeError Traceback (most recent call last)
.... in ()
----> 1 img = map2nifti(data)
/usr/local/neurosoft/epd-7.2.1/lib/python2.7/site-packages/mvpa2/datasets/mri.pyc in map2nifti(dataset, data, imghdr, imgtype)
130 # use of None for the affine should cause to pull it from
131 # the header
--> 132 return imgtype(_get_xyzt_shaped(dsarray), None, imghdr)
133 else:
134 raise ValueError(
/usr/local/neurosoft/epd-7.2.1/lib/python2.7/site-packages/nibabel-1.3.0-py2.7.egg/nibabel/spatialimages.pyc in init(self, data, affine, header, extra, file_map)
311 extra = {}
312 self.extra = extra
--> 313 self._header = self.header_class.from_header(header)
314 # if header not specified, get data type from input array
315 if header is None:
/usr/local/neurosoft/epd-7.2.1/lib/python2.7/site-packages/nibabel-1.3.0-py2.7.egg/nibabel/analyze.pyc in from_header(klass, header, check)
394 # own type, return copy
395 if type(header) == klass:
--> 396 obj = header.copy()
397 if check:
398 obj.check_fix()
/usr/local/neurosoft/epd-7.2.1/lib/python2.7/site-packages/nibabel-1.3.0-py2.7.egg/nibabel/nifti1.pyc in copy(self)
559 '''
560 return self.class(
--> 561 self.binaryblock,
562 self.endianness,
563 False,
/usr/local/neurosoft/epd-7.2.1/lib/python2.7/site-packages/nibabel-1.3.0-py2.7.egg/nibabel/wrapstruct.pyc in binaryblock(self)
208 2
209 '''
--> 210 return self._structarr.tostring()
211
212 def write_to(self, fileobj):
AttributeError: 'Nifti1Header' object has no attribute '_structarr'

[yarikoptic]

Hi @matthew-brett

sorry for lagging with investigating this... as you probably have
guessed - it is related to our custom break apart/put together of
the instances while storing them in hdf5: if objects do not provide
reduce then we are harvesting attributes... and assigning them
back upon reconstruction... as you might guess, with version change of
nibabel, and change of internal storage -- things go hairy ... ;-) I am
yet to look into this one in particular, but altogether in might be nice
if nibabel's classes provided "proper" reduce's?

On Thu, 17 Jan 2013, Dang Xiaobin wrote:

[1]@matthew-brett

----------------------------code------------------------------------------------------
import mvpa2.base.hdf5 as hd
from mvpa2.datasets.base import Dataset
from mvpa2.datasets.mri import map2nifti

#load a hdf5 matrix file or hdf5 with attribute 'a'
file = hd.h5load("fdt_matrix2.T.hdf5")
#create a Dataset instance
test_data = Dataset([])
#copy the attribute 'a' to test_data.a
test_data.a = file.a
#map the data_test to nifti file
img = map2nifti(test_data)

Yaroslav O. Halchenko
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

[yarikoptic]

ok -- here is PyMVPA-free example illustrating the problem dealing with serialized (pickled) objects across nibabel releases:

yoh@orphan:~/deb$ PYTHONPATH=$PWD/nibabel-1.1 python -c 'import nibabel as nib; from cPickle import dump; ni=nib.load("1.nii.gz").get_header();  dump(ni, open("/tmp/1.dat", "w")); print "nibabel version : %s" % nib.__version__'; ls -l /tmp/1.dat
nibabel version : 1.1.0
-rw-r--r-- 1 yoh yoh 3955 Jan 18 12:51 /tmp/1.dat

yoh@orphan:~/deb$ PYTHONPATH=$PWD/nibabel-1.2 python -c 'from cPickle import load; load(open("/tmp/1.dat")).copy()'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/yoh/deb/nibabel-1.2/nibabel/nifti1.py", line 561, in copy
    self.binaryblock,
  File "/home/yoh/deb/nibabel-1.2/nibabel/wrapstruct.py", line 210, in binaryblock
    return self._structarr.tostring()
AttributeError: 'Nifti1Header' object has no attribute '_structarr'

[bcipolli]

@yarikoptic I am hitting this same issue, using nibabel v2.0.0 and trying to access data from http://data.pymvpa.org/datasets/hyperalignment_tutorial_data/ .

Did you find a workaround for this?

[yarikoptic]

@bcipolli unfortunately I must just ask your forgiveness for our somewhat obnoxious way to serialize everything and anything into HDF5 -- it simply often doesn't work across versions (save with one, load with another) of external (or "internal" ;)) modules providing classes without proper serialization methods.

So the only current remedy to still stick to previous nibabel 1.3.0 while loading this dataset... hopefully we provide a workaround some time soonish

[bcipolli]

@yarikoptic I really appreciate the note. I was able to get this to work with nibabel v1.0, but not v1.2 nor v1.3. Just FYI :)

[yarikoptic]

@bcipolli that is strange:

hopa:/tmp
$> wget -c http://data.pymvpa.org/datasets/hyperalignment_tutorial_data/hyperalignment_tutorial_data.hdf5.gz
--2015-02-21 22:53:55--  http://data.pymvpa.org/datasets/hyperalignment_tutorial_data/hyperalignment_tutorial_data.hdf5.gz
Resolving data.pymvpa.org (data.pymvpa.org)... 129.170.30.193
Connecting to data.pymvpa.org (data.pymvpa.org)|129.170.30.193|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14252520 (14M) [application/x-gzip]
Saving to: ‘hyperalignment_tutorial_data.hdf5.gz’

hyperalignment_tutorial_data.hdf5.gz   100%[=============================================================================>]  13.59M  3.35MB/s   in 4.3s   

2015-02-21 22:53:59 (3.18 MB/s) - ‘hyperalignment_tutorial_data.hdf5.gz’ saved [14252520/14252520]


#2 !61 [0].....................................:Sat Feb 21 22:53:59:.
hopa:/tmp
$> apt-cache policy python-nibabel
python-nibabel:
  Installed: 1.3.0-2
  Candidate: 1.3.0-2
  Version table:
     2.0.0-1~nd80+1 0
        500 http://neuro.debian.net/debian-devel/ jessie/main amd64 Packages
 *** 1.3.0-2 0
        900 http://http.debian.net/debian/ jessie/main amd64 Packages
        600 http://http.debian.net/debian/ sid/main amd64 Packages
        100 /var/lib/dpkg/status
     1.3.0-1~nd70+1 0
        500 http://neuro.debian.net/debian/ jessie/main amd64 Packages

#3 !62 [0].....................................:Sat Feb 21 22:54:06:.
hopa:/tmp
$> python -c 'from mvpa2.base.hdf5 import h5load; print h5load("hyperalignment_tutorial_data.hdf5.gz")'  | head
[Dataset(array([[-0.01163818, -0.01163818,  0.00453591, ...,  0.06999348,
         0.26156655,  0.03828176],
       [-0.00443326, -0.00443326,  0.0033065 , ..., -0.00524773,
        -0.00151256, -0.03147409],
       [-0.01418352, -0.01418352,  0.00432815, ..., -0.00598793,
        -0.07479344, -0.15036383],
       ..., 
       [-0.00061614, -0.00061614,  0.00977637, ..., -0.00502579,
         0.05680206,  0.0594268 ],
       [ 0.0056519 ,  0.0056519 ,  0.00902873, ..., -0.00473931,
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IOError: [Errno 32] Broken pipe

#4 !63 [0].....................................:Sat Feb 21 22:54:19:.
hopa:/tmp
$> md5sum hyperalignment_tutorial_data.hdf5.gz 
e701480c76c07ea060d4223019a84acc  hyperalignment_tutorial_data.hdf5.gz

[yarikoptic]

what is your pymvpa version btw? mine is 2.3.1-2 and IIRC we had some fixes related to workarounds in 2.3.0

[bcipolli]

2.3.1. The dataset loads fine for me, but I can't access dataset[0].a.imghdr (except for when I have nibabel v1.0)

[effigies]

Some deep thread necromancy, but is this still an issue? A minimal reproducible example would be great. I see @yarikoptic's suggestion of providing __reduce__ (or __reduce_ex__ I guess...), but without a round-trip test that fails, it would be a pain.