ValueError: Object arrays cannot be loaded when allow_pickle=False

[dragonbook]

Hi, Rana,

Thank you for your excellent work. I found an error when running

bash ./scripts/shrec/train.sh

Below is the program output:

------------ Options -------------
arch: mconvnet
batch_size: 16
beta1: 0.9
checkpoints_dir: ./checkpoints
continue_train: False
dataroot: datasets/shrec_16
dataset_mode: classification
epoch_count: 1
export_folder: 
fc_n: 100
flip_edges: 0.2
gpu_ids: [0]
init_gain: 0.02
init_type: normal
is_train: True
lr: 0.0002
lr_decay_iters: 50
lr_policy: lambda
name: shrec16
ncf: [64, 128, 256, 256]
ninput_edges: 750
niter: 100
niter_decay: 100
no_vis: False
norm: group
num_aug: 20
num_groups: 16
num_threads: 3
phase: train
pool_res: [600, 450, 300, 180]
print_freq: 10
resblocks: 1
run_test_freq: 1
save_epoch_freq: 1
save_latest_freq: 250
scale_verts: False
seed: None
serial_batches: False
slide_verts: 0.2
verbose_plot: False
which_epoch: latest
-------------- End ----------------
loaded mean / std from cache
#training meshes = 480
---------- Network initialized -------------
[Network] Total number of parameters : 1.323 M
-----------------------------------------------
Traceback (most recent call last):
  File "train.py", line 23, in <module>
    for i, data in enumerate(dataset):
  File "/home/maiqi/yalong/project/more-personal/deep-3d/MeshCNN/data/__init__.py", line 33, in __iter__
    for i, data in enumerate(self.dataloader):
  File "/home/maiqi/yalong/software/anaconda3/envs/meshcnn/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 637, in __next__
    return self._process_next_batch(batch)
  File "/home/maiqi/yalong/software/anaconda3/envs/meshcnn/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 658, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
ValueError: Traceback (most recent call last):
  File "/home/maiqi/yalong/software/anaconda3/envs/meshcnn/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/maiqi/yalong/software/anaconda3/envs/meshcnn/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 138, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/maiqi/yalong/project/more-personal/deep-3d/MeshCNN/data/classification_data.py", line 27, in __getitem__
    mesh = Mesh(file=path, opt=self.opt, hold_history=False, export_folder=self.opt.export_folder)
  File "/home/maiqi/yalong/project/more-personal/deep-3d/MeshCNN/models/layers/mesh.py", line 16, in __init__
    fill_mesh(self, file, opt)
  File "/home/maiqi/yalong/project/more-personal/deep-3d/MeshCNN/models/layers/mesh_prepare.py", line 21, in fill_mesh
    mesh2fill.ve = mesh_data['ve']
  File "/home/maiqi/yalong/software/anaconda3/envs/meshcnn/lib/python3.6/site-packages/numpy/lib/npyio.py", line 262, in __getitem__
    pickle_kwargs=self.pickle_kwargs)
  File "/home/maiqi/yalong/software/anaconda3/envs/meshcnn/lib/python3.6/site-packages/numpy/lib/format.py", line 692, in read_array
    raise ValueError("Object arrays cannot be loaded when "
ValueError: Object arrays cannot be loaded when allow_pickle=False

---

Machine OS: ubuntu 16.04
conda environments:

#
# Name                    Version                   Build  Channel
astroid                   2.2.5                    py36_0  
blas                      1.0                         mkl  
ca-certificates           2019.1.23                     0  
certifi                   2019.3.9                 py36_0  
cffi                      1.12.3           py36h2e261b9_0  
cudatoolkit               9.2                           0  
cudnn                     7.3.1                 cuda9.2_0  
cycler                    0.10.0                   py36_0  
cython                    0.29.7           py36he6710b0_0  
dbus                      1.13.6               h746ee38_0  
expat                     2.2.6                he6710b0_0  
fontconfig                2.13.0               h9420a91_0  
freetype                  2.9.1                h8a8886c_1  
glib                      2.56.2               hd408876_0  
gst-plugins-base          1.14.0               hbbd80ab_1  
gstreamer                 1.14.0               hb453b48_1  
icu                       58.2                 h9c2bf20_1  
intel-openmp              2019.3                      199  
isort                     4.3.17                   py36_0  
jpeg                      9b                   h024ee3a_2  
kiwisolver                1.1.0            py36he6710b0_0  
lazy-object-proxy         1.3.1            py36h14c3975_2  
libedit                   3.1.20181209         hc058e9b_0  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 8.2.0                hdf63c60_1  
libgfortran-ng            7.3.0                hdf63c60_0  
libpng                    1.6.37               hbc83047_0  
libstdcxx-ng              8.2.0                hdf63c60_1  
libtiff                   4.0.10               h2733197_2  
libuuid                   1.0.3                h1bed415_2  
libxcb                    1.13                 h1bed415_1  
libxml2                   2.9.9                he19cac6_0  
matplotlib                3.0.3            py36h5429711_0  
mccabe                    0.6.1                    py36_1  
mkl                       2019.3                      199  
mkl_fft                   1.0.12           py36ha843d7b_0  
mkl_random                1.0.2            py36hd81dba3_0  
ncurses                   6.1                  he6710b0_1  
ninja                     1.9.0            py36hfd86e86_0  
numpy                     1.16.3           py36h7e9f1db_0  
numpy-base                1.16.3           py36hde5b4d6_0  
olefile                   0.46                     py36_0  
openssl                   1.1.1b               h7b6447c_1  
pcre                      8.43                 he6710b0_0  
pillow                    6.0.0            py36h34e0f95_0  
pip                       19.1                     py36_0  
protobuf                  3.7.1                    pypi_0    pypi
pycparser                 2.19                     py36_0  
pylint                    2.3.1                    py36_0  
pyparsing                 2.4.0                      py_0  
pyqt                      5.9.2            py36h05f1152_2  
python                    3.6.8                h0371630_0  
python-dateutil           2.8.0                    py36_0  
pytorch                   1.0.1           cuda92py36h65efead_0  
pytz                      2019.1                     py_0  
qt                        5.9.7                h5867ecd_1  
readline                  7.0                  h7b6447c_5  
setuptools                41.0.1                   py36_0  
sip                       4.19.8           py36hf484d3e_0  
six                       1.12.0                   py36_0  
sqlite                    3.28.0               h7b6447c_0  
tensorboardx              1.6                      pypi_0    pypi
tk                        8.6.8                hbc83047_0  
torchvision               0.2.2                      py_3    pytorch
tornado                   6.0.2            py36h7b6447c_0  
typed-ast                 1.3.4            py36h7b6447c_0  
wheel                     0.33.1                   py36_0  
wrapt                     1.11.1           py36h7b6447c_0  
xz                        5.2.4                h14c3975_4  
zlib                      1.2.11               h7b6447c_3  
zstd                      1.3.7                h0b5b093_0 

Any advice? thanks.

[ranahanocka]

Hi,
I also ran into this issue with the newest numpy (it seems to be a bug with numpy).
Can you downgrade to numpy 1.15.0 and also delete the datasets/shrec_data folder and re-download? let me know if it works

[dragonbook]

Hi,
It works now, thank you!

[ranahanocka]

great! thanks for letting me know. I'll look into if there is a work around for using the newest numpy.

[lsqshr]

It works if I explicityly add allow_pickle=True

mesh_data = np.load(load_path, encoding='latin1', allow_pickle=True) I guess the newest numpy turned this off by default.

[ranahanocka]

Nice. Added the fix in this commit. Thanks @lsqshr !

BTW - looks it came from a change in numpy 16.3

when loading *.npy or *.npz files now requires an explicit opt-in

numpy release notes