update Kubeflow Notebooks docs (#3003)

Author: thesuperzapper

content/en/_redirects

@@ -161,6 +161,12 @@ docs/started/requirements/                    /docs/started/getting-started/
 /docs/reference/pytorchjob/*       /docs/components/training
 /docs/reference/tfjob/*            /docs/components/training
 
+# Cleanup of Notebooks Docs
+/docs/components/notebooks/custom-notebook                /docs/components/notebooks/container-images
+/docs/components/notebooks/setup                          /docs/components/notebooks/quickstart-guide
+/docs/components/notebooks/troubleshoot                   /docs/components/notebooks/troubleshooting
+/docs/components/notebooks/why-use-jupyter-notebook       /docs/components/notebooks/overview
+
 # ===============
 # IMPORTANT NOTE:
 # Catch-all redirects should be added at the end of this file as redirects happen from top to bottom

content/en/docs/components/notebooks/_index.md

@@ -1,5 +1,5 @@
 +++
-title = "Notebook Servers"
-description = "Using Jupyter notebooks in Kubeflow"
+title = "Kubeflow Notebooks"
+description = "Documentation for Kubeflow Notebooks"
 weight = 10
 +++

content/en/docs/components/notebooks/container-images.md

@@ -0,0 +1,75 @@
++++
+title = "Container Images"
+description = "About Container Images for Kubeflow Notebooks"
+weight = 30
+                    
++++
+Kubeflow Notebooks natively supports three types of notebooks, [JupyterLab](https://github.com/jupyterlab/jupyterlab), [RStudio](https://github.com/rstudio/rstudio), and [Visual Studio Code (code-server)](https://github.com/cdr/code-server), but any web-based IDE should work.
+Notebook servers run as containers inside a Kubernetes Pod, which means the type of IDE (and which packages are installed) is determined by the Docker image you pick for your server.
+
+## Images
+
+We provide a number of [example container images](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers) to get you started.
+
+### Base Images
+
+These images provide a common starting point for Kubeflow Notebook containers.
+See [custom images](#custom-images) to learn how to extend them with your own packages.
+
+Dockerfile | Registry | Notes
+--- | --- | ---
+[base](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/base) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/base:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/base) | common base image
+[codeserver](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/codeserver) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver) | base [code-server](https://github.com/cdr/code-server) (Visual Studio Code) image
+[jupyter](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter) | base [JupyterLab](https://github.com/jupyterlab/jupyterlab) image
+[rstudio](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/rstudio) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio) | base [RStudio](https://github.com/rstudio/rstudio) image
+
+### Full Images
+
+These images extend the [base images](#base-images) with common packages used by Data Scientists and ML Engineers.
+
+Dockerfile | Registry | Notes
+--- | --- | ---
+[codeserver-python](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/codeserver-python) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python) | code-server (Visual Studio Code) + Conda Python
+[jupyter-pytorch (CPU)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch) | JupyterLab + PyTorch (CPU)
+[jupyter-pytorch (CUDA)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda) | JupyterLab + PyTorch (CUDA)
+[jupyter-pytorch-full (CPU)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch-full) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full) | JupyterLab + PyTorch (CPU) + [common](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch-full/requirements.txt) packages
+[jupyter-pytorch-full (CUDA)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch-full) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full) | JupyterLab + PyTorch (CUDA) + [common](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-pytorch-full/requirements.txt) packages
+[jupyter-scipy](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-scipy) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy) | JupyterLab + [SciPy](https://www.scipy.org/) packages
+[jupyter-tensorflow (CPU)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow) | JupyterLab + TensorFlow (CPU)
+[jupyter-tensorflow (CUDA)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda) | JupyterLab + TensorFlow (CUDA)
+[jupyter-tensorflow-full (CPU)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow-full) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full) | JupyterLab + TensorFlow (CPU) + [common](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow-full/requirements.txt) packages
+[jupyter-tensorflow-full (CUDA)](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow-full) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full) | JupyterLab + TensorFlow (CUDA) + [common](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/jupyter-tensorflow-full/requirements.txt) packages
+[rstudio-tidyverse](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers/rstudio-tidyverse) | [`public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:{TAG}`](https://gallery.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse) | RStudio + [Tidyverse](https://www.tidyverse.org/) packages
+
+### Image Dependency Chart
+
+This flow-chart shows how our notebook container images depend on each other.
+
+<img src="/docs/images/notebook-container-image-chart.png" 
+     alt="A flow-chart showing how notebook container images depend on each other"  
+     class="mt-3 mb-3 border border-info rounded">
+
+## Custom Images
+
+Packages installed by users __after spawning__ a Kubeflow Notebook will only last the lifetime of the pod (unless installed into a PVC-backed directory).
+
+To ensure packages are preserved throughout Pod restarts users will need to either:
+1. [Build custom images that include them](https://github.com/kubeflow/kubeflow/tree/master/components/example-notebook-servers#custom-images), or
+2. Ensure they are installed in a PVC-backed directory
+
+### Image Requirements
+
+For Kubeflow Notebooks to work with a container image, the image must:
+- expose an HTTP interface on port `8888`:
+  - kubeflow sets an environment variable `NB_PREFIX` at runtime with the URL path we expect the container be listening under
+  - kubeflow uses IFrames, so ensure your application sets `Access-Control-Allow-Origin: *` in HTTP response headers
+- run as a user called `jovyan`:
+  - the home directory of `jovyan` should be `/home/jovyan`
+  - the UID of `jovyan` should be `1000`
+- start successfully with an empty PVC mounted at `/home/jovyan`:
+  - kubeflow mounts a PVC at `/home/jovyan` to keep state across Pod restarts
+  
+## Next steps
+
+- Use your container image by specifying it when spawning your notebook server.
+  (See the [quickstart guide](/docs/components/notebooks/quickstart-guide/).)

content/en/docs/components/notebooks/custom-notebook.md

@@ -0,0 +1,58 @@
++++
+title = "Jupyter TensorFlow Examples"
+description = "Examples using Jupyter and TensorFlow in Kubeflow Notebooks"
+weight = 40
+
++++
+
+## Mnist Example
+
+(adapted from [tensorflow/tensorflow - mnist_softmax.py](https://github.com/tensorflow/tensorflow/blob/r1.4/tensorflow/examples/tutorials/mnist/mnist_softmax.py))
+
+1. When creating your notebook server choose a [container image](/docs/components/notebooks/container-images/) which has Jupyter and TensorFlow installed.
+
+2. Use Jupyter's interface to create a new **Python 3** notebook.
+
+3. Copy the following code and paste it into your notebook:
+
+    ```python
+    from tensorflow.examples.tutorials.mnist import input_data
+    mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
+
+    import tensorflow as tf
+
+    x = tf.placeholder(tf.float32, [None, 784])
+
+    W = tf.Variable(tf.zeros([784, 10]))
+    b = tf.Variable(tf.zeros([10]))
+
+    y = tf.nn.softmax(tf.matmul(x, W) + b)
+
+    y_ = tf.placeholder(tf.float32, [None, 10])
+    cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
+
+    train_step = tf.train.GradientDescentOptimizer(0.05).minimize(cross_entropy)
+
+    sess = tf.InteractiveSession()
+    tf.global_variables_initializer().run()
+
+    for _ in range(1000):
+      batch_xs, batch_ys = mnist.train.next_batch(100)
+      sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
+
+    correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
+    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
+    print("Accuracy: ", sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))
+    ```
+
+4. Run the code. You should see a number of `WARNING` messages from TensorFlow, followed by a line showing a training accuracy something like this:
+
+    ```
+    Accuracy:  0.9012
+    ```
+
+## Next steps
+
+- See a [simple example](https://github.com/kubeflow/examples/tree/master/pipelines/simple-notebook-pipeline) of creating Kubeflow pipelines in a Jupyter notebook.
+- Build machine-learning pipelines with the [Kubeflow Pipelines SDK](/docs/components/pipelines/sdk/sdk-overview/).
+- Learn the advanced features available from a Kubeflow notebook, such as [submitting Kubernetes resources](/docs/components/notebooks/submit-kubernetes/) or [building Docker images](/docs/components/notebooks/custom-notebook/). 

content/en/docs/components/notebooks/jupyter-tensorflow-examples.md

@@ -0,0 +1,22 @@
++++
+title = "Overview"
+description = "An overview of Kubeflow Notebooks"
+weight = 5
+                    
++++
+{{% stable-status %}}
+
+## What is Kubeflow Notebooks?
+
+Kubeflow Notebooks provides a way to run web-based development environments inside your Kubernetes cluster by running them inside Pods.
+
+Some key features include:
+- Native support for [JupyterLab](https://github.com/jupyterlab/jupyterlab), [RStudio](https://github.com/jupyterlab/jupyterlab), and [Visual Studio Code (code-server)](https://github.com/cdr/code-server).
+- Users can create notebook containers directly in the cluster, rather than locally on their workstations.
+- Admins can provide standard notebook images for their organization with required packages pre-installed.
+- Access control is managed by Kubeflow's RBAC, enabling easier notebook sharing across the organization.
+
+## Next steps
+
+- Get started with Kubeflow Notebooks using the [quickstart guide](/docs/components/notebooks/quickstart-guide/).
+- Learn how to create your own [container images](/docs/components/notebooks/container-images/).