Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spurious logging from GCS access #38

Closed
rsepassi opened this issue Feb 3, 2019 · 1 comment
Closed

Spurious logging from GCS access #38

rsepassi opened this issue Feb 3, 2019 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@rsepassi
Copy link
Contributor

rsepassi commented Feb 3, 2019

Short description
Spurious logging from GCS access when using tfds.load. From metadata file access internally.

Environment information

  • Operating System: Ubuntu 16
  • Python version: Python 3
  • tensorflow-datasets/tfds-nightly version: tfds-nightly
  • tensorflow/tensorflow-gpu/tf-nightly/tf-nightly-gpu version: tf-nightly

Reproduction instructions

import tensorflow as tf
tf.io.gfile.exists("gs://tfds-data")

Link to logs

2019-02-03 02:03:50.095060: I tensorflow/core/platform/cloud/retrying_utils.cc:73] The operation failed and will be automatically retried in 1.92628 seconds (attempt 10 out of 10), caused by: Unavailable: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Couldn't resolve host 'metadata'
2019-02-03 02:03:52.022467: W tensorflow/core/platform/cloud/google_auth_provider.cc:157] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "Not found: Could not locate the credentials file.". Retrieving token from GCE failed with "Aborted: All 10 retry attempts failed. The last failure: Unavailable: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Couldn't resolve host 'metadata'".

Additional context

Clearly a problem with TensorFlow. But would be nice to not have these logs dumped (10 retries). They go away on subsequent access to GCS. Seems to be just on first access. And nothing crashes or breaks, just wait for the 10 retries to be done and move on. Annoying.

One alternative for now may be to use requests to use the HTTP API for GCS access to the TFDS bucket (similar to #36).

@rsepassi rsepassi added the bug Something isn't working label Feb 3, 2019
@rsepassi rsepassi changed the title Spurious Spurious logging from GCS access Feb 3, 2019
@rsepassi rsepassi self-assigned this Feb 6, 2019
@rsepassi
Copy link
Contributor Author

rsepassi commented Feb 6, 2019

We've switched to using the HTTP API for access to the public bucket (fix for #36) so aren't seeing this anymore. It's a TF bug.

@rsepassi rsepassi closed this as completed Feb 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant