Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tf.keras.utils.get_file inconsistent behavior with keras.utils.get_file #38831

Closed
ietz opened this issue Apr 23, 2020 · 6 comments
Closed

tf.keras.utils.get_file inconsistent behavior with keras.utils.get_file #38831

ietz opened this issue Apr 23, 2020 · 6 comments
Assignees
Labels
comp:keras Keras related issues stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.1 for tracking issues in 2.1 release TF 2.2 Issues related to TF 2.2 type:bug Bug

Comments

@ietz
Copy link

ietz commented Apr 23, 2020

System information

  • OS Platform and Distribution: Colab
  • TensorFlow version: v2.2.0-rc3-0-gaad398b5e9 2.2.0-rc3
  • Python version: 3.6.9

Describe the current behavior
tensorflow.keras.utils.get_file by default saves any file to a subdirectory of ~/.keras. This behavior is specified here.

Describe the expected behavior
The function should behave the same as keras.utils.get_file, which uses the path specified in the KERAS_HOME environment variable instead of ~/.keras if it is set. This behavior is specified here

Standalone code to reproduce the issue
https://colab.research.google.com/drive/1pfS-BgACkgkCQ9nXT_iz123lvdlTk9Hg

Other info / logs
This is relevant especially for cases where the user is not able to specify the fname parameter manually to set an absolute path. One example for this mentioned in #33501 are the weights downloaded by tensorflow.keras.applications models. In my case, the user home directory is write-protected on the compute nodes of our HPC cluster and the fallback /tmp also should not be used.
The issue keras-team/keras#11923 set the behavior in keras.

@amahendrakar
Copy link
Contributor

Was able to reproduce the issue with TF v2.1, TF v2.2.0-rc3 and TF-nightly. Please find the attached gist. Thanks!

@amahendrakar amahendrakar added comp:keras Keras related issues TF 2.1 for tracking issues in 2.1 release TF 2.2 Issues related to TF 2.2 labels Apr 23, 2020
@jvishnuvardhan jvishnuvardhan added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jun 5, 2020
@jvishnuvardhan jvishnuvardhan assigned fchollet and unassigned qlzh727 Jun 5, 2020
@yan12125
Copy link

This is still an issue with the latest Tensorflow. By running the Colab notebook mentioned above (https://colab.research.google.com/drive/1pfS-BgACkgkCQ9nXT_iz123lvdlTk9Hg), it's shown that Tensorflow v2.5.0-0-ga4dfb8d1a71 2.5.0 still ignores the environment variable KERAS_HOME. Furthermore, there is no way to change the download location without modifying Python codes - the get_file function always use ~/.keras or /tmp/.keras [1]. The issue prevents downloading tf datasets to a custom location. For example,

from tensorflow.keras.datasets import cifar10
cifar10.load_data()

Always downloads the CIFAR-10 dataset to one of the two locations mentioned above.

[1] https://github.com/tensorflow/tensorflow/blob/v2.4.0/tensorflow/python/keras/utils/data_utils.py#L224-L231

@tensorflow tensorflow deleted a comment from tensorflowbutler Jun 15, 2021
@sachinprasadhs
Copy link
Contributor

While running the above mentioned colab, I'm getting error AttributeError: module 'keras.utils' has no attribute 'get_file'.

@yan12125
Copy link

AttributeError: module 'keras.utils' has no attribute 'get_file'

Yeah, the latest version of Keras is synced with Tensorflow, and keras/utils/__init__.py no longer has the line from .data_utils import get_file [1]. I can call keras.utils.data_utils.get_file, but the latter does not have the expected behavior, either.

To reproduce the expected behavior, I need to run pip install 'keras<2.4' first. Here are results on my machine:

$ ipython
Python 3.9.5 (default, May 24 2021, 12:50:35) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.24.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import os
   ...: os.environ['KERAS_HOME'] = '/tmp/my_keras_home'
   ...: 
   ...: import tensorflow
   ...: import keras
   ...: 
   ...: print('Tensorflow', tensorflow.version.GIT_VERSION, tensorflow.version.VERSION)
   ...: print(tensorflow.keras.utils.get_file('example.html', 'http://example.com/'))
   ...: print()
   ...: 
   ...: print('Keras', keras.__version__)
   ...: print(keras.utils.data_utils.get_file('example.html', 'http://example.com/'))
Using TensorFlow backend.
Tensorflow unknown 2.5.0
/tmp/.keras/datasets/example.html

Keras 2.3.1
/tmp/my_keras_home/datasets/example.html

Somehow placing !pip install 'keras<2.4' at the first line of the Colab results in mysterious errors.

[1] keras-team/keras@b5cb82c

@tensorflowbutler
Copy link
Member

Hi There,

This is a stale issue. As you are using an older version of tensorflow, we are checking to see if you still need help on this issue. Please test the issue with the latest TensorFlow (TF2.7 and tf-nightly). If the issue still persists with the newer versions of TF, please feel free to open it in keras-team/keras repository by providing details about the issue and a standalone code to reproduce the issue. Thanks!

Please note that Keras development has moved to a separate Keras-team/keras repository to focus entirely on only Keras. Thanks!

@google-ml-butler
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:keras Keras related issues stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.1 for tracking issues in 2.1 release TF 2.2 Issues related to TF 2.2 type:bug Bug
Projects
None yet
Development

No branches or pull requests

8 participants