Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix imdb.load_data for np v1.16.3 #12714

Merged
merged 1 commit into from Apr 23, 2019

Conversation

Projects
None yet
2 participants
@MarkDaoust
Copy link
Contributor

commented Apr 22, 2019

Summary

Numpy 1.16.3 changed the default value for the allow_pickle flag.

So this load call breaks.

https://nvd.nist.gov/vuln/detail/CVE-2019-6446

https://github.com/numpy/numpy/blob/v1.16.2/numpy/lib/npyio.py#L288
https://github.com/numpy/numpy/blob/v1.16.3/numpy/lib/npyio.py#L292

np.__version__ = 1.16.3
keras.__version__ = 2.2.4
keras.datasets.imdb.load_data()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-154d5969a0d5> in <module>
----> 1 k.datasets.imdb.load_data()

~/venv3/lib/python3.6/site-packages/keras/datasets/imdb.py in load_data(path, num_words, skip_top, maxlen, seed, start_char, oov_char, index_from, **kwargs)
     57                     file_hash='599dadb1135973df5b59232a0e9a887c')
     58     with np.load(path) as f:
---> 59         x_train, labels_train = f['x_train'], f['y_train']
     60         x_test, labels_test = f['x_test'], f['y_test']
     61 

~/venv3/lib/python3.6/site-packages/numpy/lib/npyio.py in __getitem__(self, key)
    260                 return format.read_array(bytes,
    261                                          allow_pickle=self.allow_pickle,
--> 262                                          pickle_kwargs=self.pickle_kwargs)
    263             else:
    264                 return self.zip.read(key)

~/venv3/lib/python3.6/site-packages/numpy/lib/format.py in read_array(fp, allow_pickle, pickle_kwargs)
    690         # The array contained Python objects. We need to unpickle the data.
    691         if not allow_pickle:
--> 692             raise ValueError("Object arrays cannot be loaded when "
    693                              "allow_pickle=False")
    694         if pickle_kwargs is None:

ValueError: Object arrays cannot be loaded when allow_pickle=False

Related Issues

PR Overview

  • This PR requires new unit tests [y/n] (make sure tests are included)
  • This PR requires to update the documentation [y/n] (make sure the docs are up-to-date)
  • This PR is backwards compatible [y/n]
  • This PR changes the current API [y/n] (all API changes need to be approved by fchollet)
Fix np.load call for np v1.16.3
Numpy 1.16.3 changed the default value for the allow_pickle flag.

So this load call breaks.

https://nvd.nist.gov/vuln/detail/CVE-2019-6446

https://github.com/numpy/numpy/blob/v1.16.2/numpy/lib/npyio.py#L288
https://github.com/numpy/numpy/blob/v1.16.3/numpy/lib/npyio.py#L292

np.__version__ = 1.16.3
keras.version = 2.2.4
keras.datasets.imdb.load_data()          

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-154d5969a0d5> in <module>
----> 1 k.datasets.imdb.load_data()

~/venv3/lib/python3.6/site-packages/keras/datasets/imdb.py in load_data(path, num_words, skip_top, maxlen, seed, start_char, oov_char, index_from, **kwargs)
     57                     file_hash='599dadb1135973df5b59232a0e9a887c')
     58     with np.load(path) as f:
---> 59         x_train, labels_train = f['x_train'], f['y_train']
     60         x_test, labels_test = f['x_test'], f['y_test']
     61 

~/venv3/lib/python3.6/site-packages/numpy/lib/npyio.py in __getitem__(self, key)
    260                 return format.read_array(bytes,
    261                                          allow_pickle=self.allow_pickle,
--> 262                                          pickle_kwargs=self.pickle_kwargs)
    263             else:
    264                 return self.zip.read(key)

~/venv3/lib/python3.6/site-packages/numpy/lib/format.py in read_array(fp, allow_pickle, pickle_kwargs)
    690         # The array contained Python objects. We need to unpickle the data.
    691         if not allow_pickle:
--> 692             raise ValueError("Object arrays cannot be loaded when "
    693                              "allow_pickle=False")
    694         if pickle_kwargs is None:

ValueError: Object arrays cannot be loaded when allow_pickle=False
```

@MarkDaoust MarkDaoust changed the title Fix np.load call for np v1.16.3 Fix imdb.load_data for np v1.16.3 Apr 22, 2019

tensorflow-copybara pushed a commit to tensorflow/tensorflow that referenced this pull request Apr 22, 2019

Fix imdb.load_data for np v1.16.3
keras-team/keras#12714

Numpy 1.16.3 changed the default value for the allow_pickle flag.

So this load call breaks.

https://nvd.nist.gov/vuln/detail/CVE-2019-6446

https://github.com/numpy/numpy/blob/v1.16.2/numpy/lib/npyio.py#L288
https://github.com/numpy/numpy/blob/v1.16.3/numpy/lib/npyio.py#L292

`np.__version__` = 1.16.3
`keras.__version__` = 2.2.4
keras.datasets.imdb.load_data()

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-154d5969a0d5> in <module>
----> 1 k.datasets.imdb.load_data()

~/venv3/lib/python3.6/site-packages/keras/datasets/imdb.py in load_data(path, num_words, skip_top, maxlen, seed, start_char, oov_char, index_from, **kwargs)
     57                     file_hash='599dadb1135973df5b59232a0e9a887c')
     58     with np.load(path) as f:
---> 59         x_train, labels_train = f['x_train'], f['y_train']
     60         x_test, labels_test = f['x_test'], f['y_test']
     61

~/venv3/lib/python3.6/site-packages/numpy/lib/npyio.py in __getitem__(self, key)
    260                 return format.read_array(bytes,
    261                                          allow_pickle=self.allow_pickle,
--> 262                                          pickle_kwargs=self.pickle_kwargs)
    263             else:
    264                 return self.zip.read(key)

~/venv3/lib/python3.6/site-packages/numpy/lib/format.py in read_array(fp, allow_pickle, pickle_kwargs)
    690         # The array contained Python objects. We need to unpickle the data.
    691         if not allow_pickle:
--> 692             raise ValueError("Object arrays cannot be loaded when "
    693                              "allow_pickle=False")
    694         if pickle_kwargs is None:

ValueError: Object arrays cannot be loaded when allow_pickle=False
```
PiperOrigin-RevId: 244705570
@fchollet

This comment has been minimized.

Copy link
Collaborator

commented Apr 23, 2019

Thanks for the PR. I will apply the same change to the other datasets that need it.

@fchollet fchollet merged commit 9416f36 into keras-team:master Apr 23, 2019

1 check failed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.