Skip to content
This repository has been archived by the owner on Jul 10, 2024. It is now read-only.

Commit

Permalink
SUBMARINE-636. Update user documents for notebook
Browse files Browse the repository at this point in the history
### What is this PR for?
Update "docs/userdocs/k8s/notebook.md" and "docs/userdocs/k8s/api/notebook.md"

### What type of PR is it?
[Documentation]

### Todos
* [ ] - Task

### What is the Jira issue?
[SUBMARINE-636](https://issues.apache.org/jira/browse/SUBMARINE-636)

### How should this be tested?
[Travis CI](https://travis-ci.org/github/lowc1012/submarine/builds/733432146)

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Ryan Lo <lowc1012@gmail.com>

Closes #431 from lowc1012/SUBMARINE-636 and squashes the following commits:

a23c71d [Ryan Lo] SUBMARINE-636. Delete jupyter-example.yaml
5fcb731 [Ryan Lo] SUBMARINE-636. Update notebook.md
fc8e9a7 [Ryan Lo] SUBMARINE-636. Update user documents for notebook
  • Loading branch information
lowc1012 authored and jiwq committed Oct 11, 2020
1 parent c611879 commit f896803
Show file tree
Hide file tree
Showing 6 changed files with 92 additions and 100 deletions.
Binary file added docs/assets/created-notebook.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/notebook-form.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/notebook-list.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
37 changes: 21 additions & 16 deletions docs/userdocs/k8s/api/notebook.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,11 @@ curl -X POST -H "Content-Type: application/json" -d '
{
"meta": {
"name": "test-nb",
"namespace": "default"
"namespace": "default",
"ownerId": "e9ca23d68d884d4ebb19d07889727dae"
},
"environment": {
"name": "my-submarine-env"
"name": "notebook-env"
},
"spec": {
"envVars": {
Expand All @@ -56,17 +57,18 @@ curl -X POST -H "Content-Type: application/json" -d '
"notebookId":"notebook_1597931805405_0001",
"name":"test-nb",
"uid":"5a94c01d-6a92-4222-bc66-c610c277546d",
"url":"/notebook/default/test-nb",
"url":"/notebook/default/test-nb/",
"status":"Created",
"createdTime":"2020-08-20T21:58:27.000+08:00",
"deletedTime":null,
"spec":{
"meta":{
"name":"test-nb",
"namespace":"default"
"namespace":"default",
"ownerId":"e9ca23d68d884d4ebb19d07889727dae"
},
"environment":{
"name":"my-submarine-env",
"name":"notebook-env",
"dockerImage":"apache/submarine:jupyter-notebook-0.5.0-SNAPSHOT",
"kernelSpec":{
"name": "team_default_python_3.7",
Expand All @@ -92,12 +94,12 @@ curl -X POST -H "Content-Type: application/json" -d '
}
```

## List notebook instances
## List notebook instances which belong to user
`GET /api/v1/notebook`

**Example Request:**
```sh
curl -X GET http://127.0.0.1:8080/api/v1/notebook
curl -X GET http://127.0.0.1:8080/api/v1/notebook?id={user_id}
```

**Example Response:**
Expand All @@ -112,17 +114,18 @@ curl -X GET http://127.0.0.1:8080/api/v1/notebook
"notebookId":"notebook_1597931805405_0001",
"name":"test-nb",
"uid":"5a94c01d-6a92-4222-bc66-c610c277546d",
"url":"/notebook/default/test-nb",
"url":"/notebook/default/test-nb/",
"status":"Created",
"createdTime":"2020-08-20T21:58:27.000+08:00",
"deletedTime":null,
"spec":{
"meta":{
"name":"test-nb",
"namespace":"default"
"namespace":"default",
"ownerId":"e9ca23d68d884d4ebb19d07889727dae"
},
"environment":{
"name":"my-submarine-env",
"name":"notebook-env",
"dockerImage":"apache/submarine:jupyter-notebook-0.5.0-SNAPSHOT",
"kernelSpec":{
"name": "team_default_python_3.7",
Expand Down Expand Up @@ -168,17 +171,18 @@ curl -X GET http://127.0.0.1:8080/api/v1/notebook/{id}
"notebookId":"notebook_1597931805405_0001",
"name":"test-nb",
"uid":"5a94c01d-6a92-4222-bc66-c610c277546d",
"url":"/notebook/default/test-nb",
"url":"/notebook/default/test-nb/",
"status":"Created",
"createdTime":"2020-08-20T21:58:27.000+08:00",
"deletedTime":null,
"spec":{
"meta":{
"name":"test-nb",
"namespace":"default"
"namespace":"default",
"ownerId":"e9ca23d68d884d4ebb19d07889727dae"
},
"environment":{
"name":"my-submarine-env",
"name":"notebook-env",
"dockerImage":"apache/submarine:jupyter-notebook-0.5.0-SNAPSHOT",
"kernelSpec":{
"name": "team_default_python_3.7",
Expand Down Expand Up @@ -223,17 +227,18 @@ curl -X DELETE http://127.0.0.1:8080/api/v1/notebook/{id}
"notebookId": "notebook_1597931805405_0001",
"name": "test-nb",
"uid": "5a94c01d-6a92-4222-bc66-c610c277546d",
"url": "/notebook/null/null",
"url": "/notebook/default/test-nb/",
"status": "Deleted",
"createdTime": "2020-08-22T14:03:19.000+08:00",
"deletedTime": "2020-08-22T14:46:28+0800",
"spec": {
"meta": {
"name": "test-nb",
"namespace": "default"
"namespace": "default",
"ownerId":"e9ca23d68d884d4ebb19d07889727dae"
},
"environment": {
"name": "my-submarine-env",
"name": "notebook-env",
"dockerImage": "apache/submarine:jupyter-notebook-0.5.0-SNAPSHOT",
"kernelSpec": {
"name": "team_default_python_3.7",
Expand Down
36 changes: 0 additions & 36 deletions docs/userdocs/k8s/jupyter-example.yaml

This file was deleted.

119 changes: 71 additions & 48 deletions docs/userdocs/k8s/notebook.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,54 +17,77 @@ specific language governing permissions and limitations
under the License.
-->

# Notebook Guide
This guide describes how to use Kubeflow's notebook-controller to manage jupyter notebook instances.

## Notebook Controller
The controller creates a StatefulSet to manage the notebook instance, and a Service to expose its port. \
Please refer to the [link](https://github.com/kubeflow/kubeflow/tree/master/components/notebook-controller) for more info.


### Pod Spec
To specify the PodSpec for the jupyter notebook.
```yaml
apiVersion: kubeflow.org/v1alpha1
kind: Notebook
metadata:
name: {NOTEBOOK_NAME}
namespace: {NAMESPACE}
labels:
app: {NOTEBOOK_NAME}
spec:
template:
spec:
containers:
- image: {IMAGE_NAME}
name: {NOTEBOOK_NAME}
env: []
resources:
requests:
cpu: "0.5"
memory: "1.0Gi"
volumes: []
...
..
```
You could refer to this sample [Dockerfile](../../../dev-support/docker-images/jupyter/Dockerfile) for building your own
jupyter docker image and the CR (Notebook) [example](jupyter-example.yaml).
# Notebooks Guide

### Create a notebook instance
```
kubectl apply -f jupyter-example.yaml
```
This guide describes how to use Jupyter notebook in Submarine to launch
and manage Jupyter notebooks.

## Working with notebooks

We recommend using Web UI to manage notebooks.

### Notebooks Web UI

Notebooks can be started from the Web UI. You can click the “Notebook” tab in the \
left-hand panel to manage your notebooks.

#### <img src="../../assets/notebook-list.png" width=70%>

To create a new notebook server, click “New Notebook”. You should see a form for entering \
details of your new notebook server.

- Notebook Name : Name of the notebook server. It should be unique and include no spaces.
- Environment : It defines a set of libraries and docker image.
- CPU and Memory
- GPU (optional)
- EnvVar (optional) : Injects environment variables into the notebook.

**If you’re not sure which environment you need, please choose the environment “notebook-env” \
for the new notebook.**

#### <img src="../../assets/notebook-form.png" width=70%>

You should see your new notebook server. Click the name of your notebook server to connect to it.

#### <img src="../../assets/created-notebook.png" width=70%>

## Experiment with your notebook

The environment “notebook-env” includes Submarine Python SDK which can talk to Submarine Server to \
create experiments, as the example below:

```python
from __future__ import print_function
import submarine
from submarine.experiment.models.environment_spec import EnvironmentSpec
from submarine.experiment.models.experiment_spec import ExperimentSpec
from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
from submarine.experiment.models.experiment_meta import ExperimentMeta
from submarine.experiment.models.code_spec import CodeSpec

# Create Submarine Client
submarine_client = submarine.ExperimentClient()

# Define TensorFlow experiment spec
environment = EnvironmentSpec(image='apache/submarine:tf-dist-mnist-test-1.0')
experiment_meta = ExperimentMeta(name='mnist-dist',
namespace='default',
framework='Tensorflow',
cmd='python /var/tf_dist_mnist/dist_mnist.py --train_steps=100',
env_vars={'ENV1': 'ENV1'})

worker_spec = ExperimentTaskSpec(resources='cpu=1,memory=1024M',
replicas=1)
ps_spec = ExperimentTaskSpec(resources='cpu=1,memory=1024M',
replicas=1)
code_spec = CodeSpec(sync_mode='git', url='https://github.com/apache/submarine.git')

experiment_spec = ExperimentSpec(meta=experiment_meta,
environment=environment,
code=code_spec,
spec={'Ps' : ps_spec,'Worker': worker_spec})

# Create experiment
experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)

## Access the notebook locally
The controller creates a Service which will target TCP port 8888 on jupyter pod and be exposed on port 80 internally. \
You can use the following command to set up port forwarding to the notebook.
```
kubectl port-forward -n ${NAMESPACE} svc/${NOTEBOOK_NAME} 8888:80
```
To access the jupyter notebook, open the following URL in your browser.
```
http://localhost:8888/notebook/${NAMESPACE}/${NOTEBOOK_NAME}
```

0 comments on commit f896803

Please sign in to comment.