Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixed the version of tensorflow-datasets for the testing #262

Merged
merged 2 commits into from
Apr 1, 2023

Conversation

vulkomilev
Copy link
Contributor

@vulkomilev vulkomilev commented Mar 1, 2023

What does this pull request do?

fixes an issue that cloud arise due to the package tensorflow-datasets being too outdated

How did you test this change?

pytest model_card_toolkit

Why this change was needed?

the test (pytest model_card_toolkit) failed with the following log
`============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-7.1.3, pluggy-0.13.1 -- /home//anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /home//open_source/model-card-toolkit, configfile: pyproject.toml
plugins: typeguard-2.11.1
collecting ... collected 78 items

model_card_toolkit/core_test.py::TfxTest::test_session PASSED [ 1%]
model_card_toolkit/core_test.py::CoreTest::test_export_format PASSED [ 2%]
model_card_toolkit/core_test.py::CoreTest::test_export_format_before_scaffold_assets PASSED [ 3%]
model_card_toolkit/core_test.py::CoreTest::test_export_format_with_customized_template_and_output_name PASSED [ 5%]
model_card_toolkit/core_test.py::CoreTest::test_init_with_store_model_uri_not_found PASSED [ 6%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets PASSED [ 7%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_empty_source PASSED [ 8%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_invalid_tfdv_source PASSED [ 10%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_invalid_tfma_source PASSED [ 11%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_json PASSED [ 12%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_source0 PASSED [ 14%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_source1 PASSED [ 15%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_source2 PASSED [ 16%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_source3 PASSED [ 17%]
model_card_toolkit/core_test.py::CoreTest::test_scaffold_assets_with_store PASSED [ 19%]
model_card_toolkit/core_test.py::CoreTest::test_session PASSED [ 20%]
model_card_toolkit/core_test.py::CoreTest::test_update_model_card_with_valid_model_card PASSED [ 21%]
model_card_toolkit/core_test.py::CoreTest::test_update_model_card_with_valid_model_card_as_proto PASSED [ 23%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_copy_from_proto_and_to_proto_with_all_fields PASSED [ 24%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_copy_from_proto_success PASSED [ 25%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_copy_from_proto_with_invalid_proto PASSED [ 26%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_from_invalid_json PASSED [ 28%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_from_invalid_json_vesion PASSED [ 29%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_from_json_and_to_json_with_all_fields PASSED [ 30%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_from_json_overwrites_previous_fields PASSED [ 32%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_from_json_to_proto PASSED [ 33%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_from_proto_to_json PASSED [ 34%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_merge_from_json_dict_and_str PASSED [ 35%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_merge_from_json_does_not_overwrite_all_fields PASSED [ 37%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_merge_from_proto_and_to_proto_with_all_fields PASSED [ 38%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_merge_from_proto_success PASSED [ 39%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_merge_from_proto_with_invalid_proto PASSED [ 41%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_to_proto_success PASSED [ 42%]
model_card_toolkit/model_card_test.py::ModelCardTest::test_to_proto_with_invalid_field PASSED [ 43%]
model_card_toolkit/documentation/examples/cats_vs_dogs_test.py::CatsVsDogsTest::test_get_data FAILED [ 44%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_annotate_dataset_feature_statistics_plots PASSED [ 46%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_annotate_eval_results_plots PASSED [ 47%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_extract_graph_data_from_dataset_feature_statistics PASSED [ 48%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_extract_graph_data_from_slicing_metrics PASSED [ 50%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_stringify_slice_key0 PASSED [ 51%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_stringify_slice_key1 PASSED [ 52%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_stringify_slice_key2 PASSED [ 53%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_stringify_slice_key3 PASSED [ 55%]
model_card_toolkit/utils/graphics_test.py::GraphicsTest::test_stringify_slice_key4 PASSED [ 56%]
model_card_toolkit/utils/json_util_test.py::JsonUtilTest::test_json_update_latest_version_should_be_identity_function PASSED [ 57%]
model_card_toolkit/utils/json_util_test.py::JsonUtilTest::test_json_update_succeeds PASSED [ 58%]
model_card_toolkit/utils/json_util_test.py::JsonUtilTest::test_json_update_validation_error PASSED [ 60%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_annotate_eval_results_metrics PASSED [ 61%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_filter_features PASSED [ 62%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_filter_metrics PASSED [ 64%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_generate_model_card_for_model PASSED [ 65%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_generate_model_card_for_model_with_invalid_db PASSED [ 66%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_generate_model_card_for_model_with_invalid_model PASSED [ 67%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_generate_model_card_for_model_with_model_not_found PASSED [ 69%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_get_metrics_artifacts_for_model PASSED [ 70%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_get_metrics_artifacts_for_model_model_with_model_not_found PASSED [ 71%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_get_metrics_artifacts_for_model_with_invalid_db PASSED [ 73%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_get_metrics_artifacts_for_model_with_invalid_model PASSED [ 74%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_get_stats_artifacts_for_model PASSED [ 75%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_get_stats_artifacts_for_model_with_invalid_db PASSED [ 76%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_get_stats_artifacts_for_model_with_invalid_model PASSED [ 78%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_get_stats_artifacts_for_model_with_model_not_found PASSED [ 79%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_read_metrics_eval_result PASSED [ 80%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_read_metrics_eval_result_with_invalid_uri PASSED [ 82%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_read_stats_proto PASSED [ 83%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_read_stats_proto_with_invalid_split PASSED [ 84%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_read_stats_proto_with_invalid_uri PASSED [ 85%]
model_card_toolkit/utils/tfx_util_test.py::TfxUtilsTest::test_read_stats_protos PASSED [ 87%]
model_card_toolkit/utils/validation_test.py::ValidationTest::test_template_test_files_cats_vs_dogs PASSED [ 88%]
model_card_toolkit/utils/validation_test.py::ValidationTest::test_template_test_files_considerations PASSED [ 89%]
model_card_toolkit/utils/validation_test.py::ValidationTest::test_template_test_files_full PASSED [ 91%]
model_card_toolkit/utils/validation_test.py::ValidationTest::test_template_test_files_train_data PASSED [ 92%]
model_card_toolkit/utils/validation_test.py::ValidationTest::test_validate_json_schema PASSED [ 93%]
model_card_toolkit/utils/validation_test.py::ValidationTest::test_validate_json_schema_invalid_dict PASSED [ 94%]
model_card_toolkit/utils/validation_test.py::ValidationTest::test_validate_json_schema_invalid_version PASSED [ 96%]
model_card_toolkit/utils/testdata/testdata_utils_test.py::TestDataUtilsTest::test_get_tfx_pipeline_metadata_store_metrics_artifact_exists PASSED [ 97%]
model_card_toolkit/utils/testdata/testdata_utils_test.py::TestDataUtilsTest::test_get_tfx_pipeline_metadata_store_model_uri_exists PASSED [ 98%]
model_card_toolkit/utils/testdata/testdata_utils_test.py::TestDataUtilsTest::test_get_tfx_pipeline_metadata_store_stats_artifact_exists PASSED [100%]

=================================== FAILURES ===================================
_________________________ CatsVsDogsTest.test_get_data _________________________

self = <model_card_toolkit.documentation.examples.cats_vs_dogs_test.CatsVsDogsTest testMethod=test_get_data>

def test_get_data(self):
data = cats_vs_dogs.get_data()

model_card_toolkit/documentation/examples/cats_vs_dogs_test.py:13:


model_card_toolkit/documentation/examples/cats_vs_dogs.py:36: in get_data
validation_ds = tfds.load(
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/load.py:328: in load
dbuilder.download_and_prepare(**download_and_prepare_kwargs)
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/dataset_builder.py:432: in download_and_prepare
self._download_and_prepare(
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/dataset_builder.py:1135: in _download_and_prepare
split_generators = self._split_generators( # pylint: disable=unexpected-keyword-arg
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/image_classification/cats_vs_dogs.py:72: in _split_generators
path = dl_manager.download(_URL)
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py:549: in download
return _map_promise(self._download, url_or_urls)
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py:777: in _map_promise
res = tf.nest.map_structure(lambda p: p.get(), all_promises) # Wait promises
../../anaconda3/lib/python3.8/site-packages/tensorflow/python/util/nest.py:917: in map_structure
structure[0], [func(*x) for x in entries],
../../anaconda3/lib/python3.8/site-packages/tensorflow/python/util/nest.py:917: in
structure[0], [func(*x) for x in entries],
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py:777: in
res = tf.nest.map_structure(lambda p: p.get(), all_promises) # Wait promises
../../anaconda3/lib/python3.8/site-packages/promise/promise.py:512: in get
return self._target_settled_value(_raise=True)
../../anaconda3/lib/python3.8/site-packages/promise/promise.py:516: in _target_settled_value
return self._target()._settled_value(_raise)
../../anaconda3/lib/python3.8/site-packages/promise/promise.py:226: in _settled_value
reraise(type(raise_val), raise_val, self._traceback)
../../anaconda3/lib/python3.8/site-packages/six.py:703: in reraise
raise value
../../anaconda3/lib/python3.8/site-packages/promise/promise.py:844: in handle_future_result
resolve(future.result())
../../anaconda3/lib/python3.8/concurrent/futures/_base.py:432: in result
return self.__get_result()
../../anaconda3/lib/python3.8/concurrent/futures/_base.py:388: in __get_result
raise self._exception
../../anaconda3/lib/python3.8/concurrent/futures/thread.py:57: in run
result = self.fn(*self.args, **self.kwargs)
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/downloader.py:209: in _sync_download
with _open_url(url, verify=verify) as (response, iter_content):
../../anaconda3/lib/python3.8/contextlib.py:113: in enter
return next(self.gen)
../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/downloader.py:271: in _open_with_requests
_assert_status(response)


response = <Response [404]>

def _assert_status(response: requests.Response) -> None:
  """Ensure the URL response is 200."""
  if response.status_code != 200:
  raise DownloadError('Failed to get url {}. HTTP code: {}.'.format(
        response.url, response.status_code))

E tensorflow_datasets.core.download.downloader.DownloadError: Failed to get url https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip. HTTP code: 404.

../../anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/downloader.py:302: DownloadError
----------------------------- Captured stdout call -----------------------------
�[1mDownloading and preparing dataset 786.68 MiB (download: 786.68 MiB, generated: Unknown size, total: 786.68 MiB) to /home//tensorflow_datasets/cats_vs_dogs/4.0.0...�[0m
----------------------------- Captured stderr call -----------------------------
2023-02-25 14:49:45.489191: W tensorflow/core/platform/cloud/google_auth_provider.cc:184] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "NOT_FOUND: Could not locate the credentials file.". Retrieving token from GCE failed with "FAILED_PRECONDITION: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Could not resolve host: metadata".

Dl Completed...: 0 url [00:00, ? url/s]

Dl Size...: 0 MiB [00:00, ? MiB/s]�[A
Dl Completed...: 0%| | 0/1 [00:00<?, ? url/s]

Dl Size...: 0 MiB [00:00, ? MiB/s]�[A
Dl Size...: 0 MiB [00:00, ? MiB/s]

Dl Completed...: 0%| | 0/1 [00:00<?, ? url/s]
=============================== warnings summary ===============================
../../anaconda3/lib/python3.8/site-packages/hdfs/config.py:15
/home//anaconda3/lib/python3.8/site-packages/hdfs/config.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import load_source

../../anaconda3/lib/python3.8/site-packages/h5py/init.py:46
/home//anaconda3/lib/python3.8/site-packages/h5py/init.py:46: DeprecationWarning: np.typeDict is a deprecated alias for np.sctypeDict.
from ._conv import register_converters as _register_converters

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED model_card_toolkit/documentation/examples/cats_vs_dogs_test.py::CatsVsDogsTest::test_get_data
================== 1 failed, 77 passed, 2 warnings in 50.98s ===================`

hanneshapke
hanneshapke previously approved these changes Mar 1, 2023
@hanneshapke
Copy link
Collaborator

@vulkomilev Can you please leave a comment in the pr why the pinning of the version was needed. Thank you! Good to know in case we check-in later.

@hanneshapke hanneshapke merged commit 14f7865 into tensorflow:main Apr 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants