Enabling TFMA jupyter extension on GPU images requires libcuda

[jlewi]

When we try to enable the Jupyter extension for TFMA in GPU images we run into the following problem

 File "/opt/conda/envs/py2/lib/python2.7/site-packages/notebook/nbextensions.py", line 211, in install_nbextension_python
   m, nbexts = _get_nbextension_metadata(module)
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
   m = import_item(module)
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/traitlets/utils/importstring.py", line 42, in import_item
   return __import__(parts[0])
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow_model_analysis/__init__.py", line 17, in <module>
   from tensorflow_model_analysis import view
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow_model_analysis/view/__init__.py", line 15, in <module>
    from tensorflow_model_analysis.view.widget_view import render_plot
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow_model_analysis/view/widget_view.py", line 21, in <module>
   from tensorflow_model_analysis.api import model_eval_lib
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow_model_analysis/api/model_eval_lib.py", line 24, in <module>
   import tensorflow as tf
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow/__init__.py", line 22, in <module>
   from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 49, in <module>
   from tensorflow.python import pywrap_tensorflow
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
   from tensorflow.python.pywrap_tensorflow_internal import *
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
   _pywrap_tensorflow_internal = swig_import_helper()
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
   _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/install_sources#common_installation_problems
for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.
The command '/bin/bash -c chmod a+rx /tmp/install.sh &&     /tmp/install.sh' returned a non-zero code: 1

It looks like the problem is that enabling the extension requires importing TF which won't work when the installed version of TF is the GPU version.

[jlewi]

Two possible solutoins

1. Run it on GPU during image build
2. Delay enabling of the extension until runtime.

[jlewi]

Looks like just installing the extension tries to import TF

+ jupyter nbextension install --py --system --symlink tensorflow_model_analysis
Traceback (most recent call last):
 File "/opt/conda/envs/py2/bin/jupyter-nbextension", line 11, in <module>
   sys.exit(main())
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/jupyter_core/application.py", line 266, in launch_instance
   return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/notebook/nbextensions.py", line 988, in start
    super(NBExtensionApp, self).start()
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/jupyter_core/application.py", line 255, in start
   self.subapp.start()
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/notebook/nbextensions.py", line 716, in start
    self.install_extensions()
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/notebook/nbextensions.py", line 695, in install_extensions
    **kwargs
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/notebook/nbextensions.py", line 211, in install_nbextension_python
   m, nbexts = _get_nbextension_metadata(module)
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
    m = import_item(module)
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/traitlets/utils/importstring.py", line 42, in import_item
   return __import__(parts[0])
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow_model_analysis/__init__.py", line 16, in <module>
   from tensorflow_model_analysis import view
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow_model_analysis/view/__init__.py", line 15, in <module>
    from tensorflow_model_analysis.view.jupyter_widget_view import render_plot
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow_model_analysis/view/jupyter_widget_view.py", line 21, in <module>
   from tensorflow_model_analysis.api import model_eval_lib
 File "/opt/conda/envs/py2/lib/python2.7/site-packages/tensorflow_model_analysis/api/model_eval_lib.py", line 24, in <module>
   import tensorflow as tf