intro-swivel.ipynb fails on TF import

[ivanmkc]

Logs: https://pantheon.corp.google.com/cloud-build/builds/ebea2843-3345-4a7e-a4e4-5c625613d42d?project=python-docs-samples-tests

Error in intro-swivel.ipynb when importing tensorflow:

Step #3: Traceback (most recent call last):
Step #3:   File "/workspace/.cloud-build/execute_notebook_cli.py", line 34, in <module>
Step #3:     ExecuteNotebook.execute_notebook(
Step #3:   File "/workspace/.cloud-build/ExecuteNotebook.py", line 84, in execute_notebook
Step #3:     raise execution_exception
Step #3:   File "/workspace/.cloud-build/ExecuteNotebook.py", line 53, in execute_notebook
Step #3:     pm.execute_notebook(
Step #3:   File "/builder/home/.local/lib/python3.9/site-packages/papermill/execute.py", line 122, in execute_notebook
Step #3:     raise_for_execution_errors(nb, output_path)
Step #3:   File "/builder/home/.local/lib/python3.9/site-packages/papermill/execute.py", line 234, in raise_for_execution_errors
Step #3:     raise error
Step #3: papermill.exceptions.PapermillExecutionError: 
Step #3: ---------------------------------------------------------------------------
Step #3: Exception encountered at "In [14]":
Step #3: ---------------------------------------------------------------------------
Step #3: AttributeError                            Traceback (most recent call last)
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in _dep_map(self)
Step #3:    3015         try:
Step #3: -> 3016             return self.__dep_map
Step #3:    3017         except AttributeError:
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in __getattr__(self, attr)
Step #3:    2812         if attr.startswith('_'):
Step #3: -> 2813             raise AttributeError(attr)
Step #3:    2814         return getattr(self._provider, attr)
Step #3: 
Step #3: AttributeError: _DistInfoDistribution__dep_map
Step #3: 
Step #3: During handling of the above exception, another exception occurred:
Step #3: 
Step #3: AttributeError                            Traceback (most recent call last)
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in _parsed_pkg_info(self)
Step #3:    3006         try:
Step #3: -> 3007             return self._pkg_info
Step #3:    3008         except AttributeError:
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in __getattr__(self, attr)
Step #3:    2812         if attr.startswith('_'):
Step #3: -> 2813             raise AttributeError(attr)
Step #3:    2814         return getattr(self._provider, attr)
Step #3: 
Step #3: AttributeError: _pkg_info
Step #3: 
Step #3: During handling of the above exception, another exception occurred:
Step #3: 
Step #3: FileNotFoundError                         Traceback (most recent call last)
Step #3: /tmp/ipykernel_15/2420721388.py in <module>
Step #3:       1 import pandas as pd
Step #3: ----> 2 import tensorflow as tf
Step #3:       3 from google.cloud import aiplatform
Step #3:       4 from kfp.v2.google import client
Step #3:       5 from sklearn.metrics.pairwise import cosine_similarity
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/__init__.py in <module>
Step #3:      39 import sys as _sys
Step #3:      40 
Step #3: ---> 41 from tensorflow.python.tools import module_util as _module_util
Step #3:      42 from tensorflow.python.util.lazy_loader import LazyLoader as _LazyLoader
Step #3:      43 
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/__init__.py in <module>
Step #3:      47 from tensorflow.python import distribute
Step #3:      48 # from tensorflow.python import keras
Step #3: ---> 49 from tensorflow.python.feature_column import feature_column_lib as feature_column
Step #3:      50 # from tensorflow.python.layers import layers
Step #3:      51 from tensorflow.python.module import module
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/feature_column/feature_column_lib.py in <module>
Step #3:      20 
Step #3:      21 # pylint: disable=unused-import,line-too-long,wildcard-import,g-bad-import-order
Step #3: ---> 22 from tensorflow.python.feature_column.feature_column import *
Step #3:      23 from tensorflow.python.feature_column.feature_column_v2 import *
Step #3:      24 from tensorflow.python.feature_column.sequence_feature_column import *
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/feature_column/feature_column.py in <module>
Step #3:     145 from tensorflow.python.framework import sparse_tensor as sparse_tensor_lib
Step #3:     146 from tensorflow.python.framework import tensor_shape
Step #3: --> 147 from tensorflow.python.layers import base
Step #3:     148 from tensorflow.python.ops import array_ops
Step #3:     149 from tensorflow.python.ops import check_ops
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/layers/base.py in <module>
Step #3:      18 from __future__ import print_function
Step #3:      19 
Step #3: ---> 20 from tensorflow.python.keras.legacy_tf_layers import base
Step #3:      21 
Step #3:      22 InputSpec = base.InputSpec
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/__init__.py in <module>
Step #3:      23 
Step #3:      24 # See b/110718070#comment18 for more details about this import.
Step #3: ---> 25 from tensorflow.python.keras import models
Step #3:      26 
Step #3:      27 from tensorflow.python.keras.engine.input_layer import Input
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/models.py in <module>
Step #3:      18 from tensorflow.python.framework import ops
Step #3:      19 from tensorflow.python.keras import backend
Step #3: ---> 20 from tensorflow.python.keras import metrics as metrics_module
Step #3:      21 from tensorflow.python.keras import optimizer_v1
Step #3:      22 from tensorflow.python.keras.engine import functional
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/metrics.py in <module>
Step #3:      32 from tensorflow.python.framework import ops
Step #3:      33 from tensorflow.python.framework import tensor_shape
Step #3: ---> 34 from tensorflow.python.keras import activations
Step #3:      35 from tensorflow.python.keras import backend
Step #3:      36 from tensorflow.python.keras.engine import base_layer
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/activations.py in <module>
Step #3:      16 
Step #3:      17 from tensorflow.python.keras import backend
Step #3: ---> 18 from tensorflow.python.keras.layers import advanced_activations
Step #3:      19 from tensorflow.python.keras.utils.generic_utils import deserialize_keras_object
Step #3:      20 from tensorflow.python.keras.utils.generic_utils import serialize_keras_object
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/layers/__init__.py in <module>
Step #3:      20 # pylint: disable=g-bad-import-order
Step #3:      21 # pylint: disable=g-import-not-at-top
Step #3: ---> 22 from tensorflow.python.keras.engine.input_layer import Input
Step #3:      23 from tensorflow.python.keras.engine.input_layer import InputLayer
Step #3:      24 from tensorflow.python.keras.engine.input_spec import InputSpec
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/engine/input_layer.py in <module>
Step #3:      22 from tensorflow.python.keras import backend
Step #3:      23 from tensorflow.python.keras.distribute import distributed_training_utils
Step #3: ---> 24 from tensorflow.python.keras.engine import base_layer
Step #3:      25 from tensorflow.python.keras.engine import keras_tensor
Step #3:      26 from tensorflow.python.keras.engine import node as node_module
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer.py in <module>
Step #3:      46 from tensorflow.python.keras import initializers
Step #3:      47 from tensorflow.python.keras import regularizers
Step #3: ---> 48 from tensorflow.python.keras.engine import base_layer_utils
Step #3:      49 from tensorflow.python.keras.engine import input_spec
Step #3:      50 from tensorflow.python.keras.engine import keras_tensor
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_utils.py in <module>
Step #3:      29 from tensorflow.python.keras.utils import control_flow_util
Step #3:      30 from tensorflow.python.keras.utils import tf_inspect
Step #3: ---> 31 from tensorflow.python.keras.utils import tf_utils
Step #3:      32 from tensorflow.python.ops import array_ops
Step #3:      33 from tensorflow.python.ops import variables as tf_variables
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/keras/utils/tf_utils.py in <module>
Step #3:      20 
Step #3:      21 from tensorflow.python.data.experimental.ops import cardinality
Step #3: ---> 22 from tensorflow.python.distribute.coordinator import cluster_coordinator as coordinator_lib
Step #3:      23 from tensorflow.python.eager import context
Step #3:      24 from tensorflow.python.framework import composite_tensor
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/distribute/coordinator/cluster_coordinator.py in <module>
Step #3:      32 from six.moves import queue
Step #3:      33 
Step #3: ---> 34 from tensorflow.python.distribute import parameter_server_strategy_v2
Step #3:      35 from tensorflow.python.distribute.coordinator import coordinator_context
Step #3:      36 from tensorflow.python.distribute.coordinator import metric_utils
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/distribute/parameter_server_strategy_v2.py in <module>
Step #3:      32 from tensorflow.python.distribute import mirrored_run
Step #3:      33 from tensorflow.python.distribute import multi_worker_util
Step #3: ---> 34 from tensorflow.python.distribute import parameter_server_strategy
Step #3:      35 from tensorflow.python.distribute import ps_values
Step #3:      36 from tensorflow.python.distribute import sharded_variable
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/distribute/parameter_server_strategy.py in <module>
Step #3:      32 from tensorflow.python.distribute import ps_values
Step #3:      33 from tensorflow.python.distribute import values
Step #3: ---> 34 from tensorflow.python.distribute.cluster_resolver import SimpleClusterResolver
Step #3:      35 from tensorflow.python.distribute.cluster_resolver import TFConfigClusterResolver
Step #3:      36 from tensorflow.python.eager import context
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/distribute/cluster_resolver/__init__.py in <module>
Step #3:      29 from tensorflow.python.distribute.cluster_resolver.cluster_resolver import SimpleClusterResolver
Step #3:      30 from tensorflow.python.distribute.cluster_resolver.cluster_resolver import UnionClusterResolver
Step #3: ---> 31 from tensorflow.python.distribute.cluster_resolver.gce_cluster_resolver import GCEClusterResolver
Step #3:      32 from tensorflow.python.distribute.cluster_resolver.kubernetes_cluster_resolver import KubernetesClusterResolver
Step #3:      33 from tensorflow.python.distribute.cluster_resolver.slurm_cluster_resolver import SlurmClusterResolver
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/tensorflow/python/distribute/cluster_resolver/gce_cluster_resolver.py in <module>
Step #3:      26 _GOOGLE_API_CLIENT_INSTALLED = True
Step #3:      27 try:
Step #3: ---> 28   from googleapiclient import discovery  # pylint: disable=g-import-not-at-top
Step #3:      29   from oauth2client.client import GoogleCredentials  # pylint: disable=g-import-not-at-top
Step #3:      30 except ImportError:
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/googleapiclient/discovery.py in <module>
Step #3:      66 from googleapiclient.errors import UnknownApiNameOrVersion
Step #3:      67 from googleapiclient.errors import UnknownFileType
Step #3: ---> 68 from googleapiclient.http import build_http
Step #3:      69 from googleapiclient.http import BatchHttpRequest
Step #3:      70 from googleapiclient.http import HttpMock
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/googleapiclient/http.py in <module>
Step #3:      65 from googleapiclient.errors import UnexpectedBodyError
Step #3:      66 from googleapiclient.errors import UnexpectedMethodError
Step #3: ---> 67 from googleapiclient.model import JsonModel
Step #3:      68 
Step #3:      69 
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/googleapiclient/model.py in <module>
Step #3:      34 from googleapiclient.errors import HttpError
Step #3:      35 
Step #3: ---> 36 _LIBRARY_VERSION = pkg_resources.get_distribution("google-api-python-client").version
Step #3:      37 _PY_VERSION = platform.python_version()
Step #3:      38 
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in get_distribution(dist)
Step #3:     464         dist = Requirement.parse(dist)
Step #3:     465     if isinstance(dist, Requirement):
Step #3: --> 466         dist = get_provider(dist)
Step #3:     467     if not isinstance(dist, Distribution):
Step #3:     468         raise TypeError("Expected string, Requirement, or Distribution", dist)
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in get_provider(moduleOrReq)
Step #3:     340     """Return an IResourceProvider for the named module or requirement"""
Step #3:     341     if isinstance(moduleOrReq, Requirement):
Step #3: --> 342         return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
Step #3:     343     try:
Step #3:     344         module = sys.modules[moduleOrReq]
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in require(self, *requirements)
Step #3:     884         included, even if they were already activated in this working set.
Step #3:     885         """
Step #3: --> 886         needed = self.resolve(parse_requirements(requirements))
Step #3:     887 
Step #3:     888         for dist in needed:
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras)
Step #3:     778 
Step #3:     779             # push the new requirements onto the stack
Step #3: --> 780             new_requirements = dist.requires(req.extras)[::-1]
Step #3:     781             requirements.extend(new_requirements)
Step #3:     782 
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in requires(self, extras)
Step #3:    2732     def requires(self, extras=()):
Step #3:    2733         """List of Requirements needed for this distro if `extras` are used"""
Step #3: -> 2734         dm = self._dep_map
Step #3:    2735         deps = []
Step #3:    2736         deps.extend(dm.get(None, ()))
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in _dep_map(self)
Step #3:    3016             return self.__dep_map
Step #3:    3017         except AttributeError:
Step #3: -> 3018             self.__dep_map = self._compute_dependencies()
Step #3:    3019             return self.__dep_map
Step #3:    3020 
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in _compute_dependencies(self)
Step #3:    3025         reqs = []
Step #3:    3026         # Including any condition expressions
Step #3: -> 3027         for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
Step #3:    3028             reqs.extend(parse_requirements(req))
Step #3:    3029 
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in _parsed_pkg_info(self)
Step #3:    3007             return self._pkg_info
Step #3:    3008         except AttributeError:
Step #3: -> 3009             metadata = self.get_metadata(self.PKG_INFO)
Step #3:    3010             self._pkg_info = email.parser.Parser().parsestr(metadata)
Step #3:    3011             return self._pkg_info
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in get_metadata(self, name)
Step #3:    1405             return ""
Step #3:    1406         path = self._get_metadata_path(name)
Step #3: -> 1407         value = self._get(path)
Step #3:    1408         try:
Step #3:    1409             return value.decode('utf-8')
Step #3: 
Step #3: /usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py in _get(self, path)
Step #3:    1609 
Step #3:    1610     def _get(self, path):
Step #3: -> 1611         with open(path, 'rb') as stream:
Step #3:    1612             return stream.read()
Step #3:    1613 
Step #3: 
Step #3: FileNotFoundError: [Errno 2] No such file or directory: '/builder/home/.local/lib/python3.9/site-packages/google_auth-2.3.3.dist-info/METADATA'

[ivanmkc]

I'd recommend waiting for #140 and #139 to be resolved as they include TF as well.

[andrewferlitsch]

@kweinmeister 139 and 140 are closed. Can we retest this notebook and see if the install issue is still there.

[kweinmeister]

@andrewferlitsch - sure, that makes sense to retry (would that take a new PR making a minor change to re-test)?

[yinghsienwu]

I want to update this notebook for some new features. Let me create a PR.

[andrewferlitsch]

Hi Amy. Look forward to the update. Will process the PR when it comes through.

…

On Tue, Nov 30, 2021 at 1:36 PM Amy Wu ***@***.***> wrote: I want to update this notebook for some new features. Let me create a PR. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#134 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFOVC2CNXYC4BBVU3ZFX54LUOU7XJANCNFSM5HYO5M5Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[yinghsienwu]

This TensorFlow import issue was fixed by Andrew. I submit a PR and this PR is blocked by another downstream error in vertex-ai-notebook-execution-test. See #193