Periodic example tests are failing

[jlewi]

Tests are failing
https://k8s-testgrid.appspot.com/sig-big-data#kubeflow-periodic-examples

Here are the logs
mnist-141008-a08-j8hpc.log.txt

It looks like the problem is installing the Azure libraries.

2020-04-21T14:11:19.446899515Z  executing-notebooks  Collecting azure>=4.0.0
2020-04-21T14:11:19.462923687Z  executing-notebooks    Downloading azure-5.0.0.zip (4.6 kB)
2020-04-21T14:11:19.764094752Z  executing-notebooks      ERROR: Command errored out with exit status 1:
2020-04-21T14:11:19.764133506Z  executing-notebooks       command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u9prdzr6/azure/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u9prdzr6/azure/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-u9prdzr6/azure/pip-egg-info
2020-04-21T14:11:19.764141239Z  executing-notebooks           cwd: /tmp/pip-install-u9prdzr6/azure/
2020-04-21T14:11:19.764145700Z  executing-notebooks      Complete output (24 lines):
2020-04-21T14:11:19.764150263Z  executing-notebooks      Traceback (most recent call last):
2020-04-21T14:11:19.764154553Z  executing-notebooks        File "<string>", line 1, in <module>
2020-04-21T14:11:19.764159501Z  executing-notebooks        File "/tmp/pip-install-u9prdzr6/azure/setup.py", line 60, in <module>
2020-04-21T14:11:19.764164301Z  executing-notebooks          raise RuntimeError(message)
2020-04-21T14:11:19.764168478Z  executing-notebooks      RuntimeError:
2020-04-21T14:11:19.764172635Z  executing-notebooks
2020-04-21T14:11:19.764176710Z  executing-notebooks      Starting with v5.0.0, the 'azure' meta-package is deprecated and cannot be installed anymore.
2020-04-21T14:11:19.764181201Z  executing-notebooks      Please install the service specific packages prefixed by `azure` needed for your application.
2020-04-21T14:11:19.764185534Z  executing-notebooks
2020-04-21T14:11:19.764189599Z  executing-notebooks      The complete list of available packages can be found at:
2020-04-21T14:11:19.764193822Z  executing-notebooks      https://aka.ms/azsdk/python/all
2020-04-21T14:11:19.764198068Z  executing-notebooks
2020-04-21T14:11:19.764202084Z  executing-notebooks      Here's a non-exhaustive list of common packages:
2020-04-21T14:11:19.764206278Z  executing-notebooks
2020-04-21T14:11:19.764210318Z  executing-notebooks      -  azure-mgmt-compute (https://pypi.python.org/pypi/azure-mgmt-compute) : Management of Virtual Machines, etc.
2020-04-21T14:11:19.764214820Z  executing-notebooks      -  azure-mgmt-storage (https://pypi.python.org/pypi/azure-mgmt-storage) : Management of storage accounts.
2020-04-21T14:11:19.764219227Z  executing-notebooks      -  azure-mgmt-resource (https://pypi.python.org/pypi/azure-mgmt-resource) : Generic package about Azure Resource Management (ARM)
2020-04-21T14:11:19.764223639Z  executing-notebooks      -  azure-keyvault-secrets (https://pypi.python.org/pypi/azure-keyvault-secrets) : Access to secrets in Key Vault
2020-04-21T14:11:19.764240871Z  executing-notebooks      -  azure-storage-blob (https://pypi.python.org/pypi/azure-storage-blob) : Access to blobs in storage accounts
2020-04-21T14:11:19.764245590Z  executing-notebooks
2020-04-21T14:11:19.764249640Z  executing-notebooks      A more comprehensive discussion of the rationale for this decision can be found in the following issue:
2020-04-21T14:11:19.764254018Z  executing-notebooks      https://github.com/Azure/azure-sdk-for-python/issues/10646
2020-04-21T14:11:19.764263250Z  executing-notebooks
2020-04-21T14:11:19.764268045Z  executing-notebooks
2020-04-21T14:11:19.764271951Z  executing-notebooks      ----------------------------------------
2020-04-21T14:11:19.796503176Z  executing-notebooks  ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output

It looks like one solution would be to pin the libraries to an earlier version

[issue-label-bot]

Issue-Label Bot is automatically applying the labels:

Label	Probability
kind/bug	0.98

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[jlewi]

I think we might just need to bump the fairing commit used in the examples.
This is the current commit
https://github.com/kubeflow/fairing/blob/7e4c23c47a6b0544a722d04e1f9910c2f283a4f6/requirements.txt#L21

Which is using

azure>=4.0.0

It looks like on master fairing is using specific subpackages just like the error message suggests

[jlewi]

I think we might just need to bump the fairing commit used in the examples.
This is the current commit
https://github.com/kubeflow/fairing/blob/7e4c23c47a6b0544a722d04e1f9910c2f283a4f6/requirements.txt#L21

Which is using

azure>=4.0.0

It looks like on master fairing is using specific subpackages just like the error message suggests

[iancoffey]

A note that I volunteered to help with this on the community meeting :)

[iancoffey]

Bumped this mnist fairing version in -> #795