Skip to content

Commit

Permalink
docs: Update the Kubeflow example
Browse files Browse the repository at this point in the history
  • Loading branch information
ssheng committed Mar 23, 2023
1 parent bc2744b commit c040505
Show file tree
Hide file tree
Showing 5 changed files with 17 additions and 21 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 4 additions & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,10 @@
"pypi": ("https://pypi.org/project/%s", "%s"), # noqa: WPS323
"wiki": ("https://wikipedia.org/wiki/%s", "%s"), # noqa: WPS323
"github": ("https://github.com/%s", "%s"), # noqa: WPS323
"examples": ("https://github.com/bentoml/BentoML/tree/main/examples/%s", "examples/"), # noqa: WPS323
"examples": (
"https://github.com/bentoml/BentoML/tree/main/examples/%s",
"examples/",
), # noqa: WPS323
}
# custom roles
rst_prolog = """
Expand Down
2 changes: 2 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ working on :)
| [triton/onnx](https://github.com/bentoml/BentoML/tree/main/examples/triton/onnx) | ONNX | YOLOv5 | Triton Inference Server, gRPC, Python SDK (Containerization, Serve, Build) |
| [triton/pytorch](https://github.com/bentoml/BentoML/tree/main/examples/triton/pytorch) | Torchscript | YOLOv5 | Triton Inference Server, gRPC, Python SDK (Containerization, Serve, Build) |
| [triton/tensorflow](https://github.com/bentoml/BentoML/tree/main/examples/triton/tensorflow) | Tensorflow | YOLOv5 | Triton Inference Server, gRPC, Python SDK (Containerization, Serve, Build) |
| [kubeflow](https://github.com/bentoml/BentoML/tree/main/examples/kubeflow) |
XGBoost | Fraud Detection | Kubeflow, Notebook |

## How to contribute

Expand Down
29 changes: 10 additions & 19 deletions examples/kubeflow/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
# Developing BentoML Applications on Kubeflow

Starting with the release of Kubeflow 1.7, BentoML provides a native integration with Kubeflow. This integration allows you to package models trained in Kubeflow notebooks or pipelines as [Bentos](https://docs.bentoml.org/en/latest/concepts/bento.html), and deploy them as microservices in a Kubernetes cluster through BentoML's cloud native components and custom resource definitions (CRDs). This documentation provides a comprehensive guide on how to use BentoML and Kubeflow together to streamline the process of deploying models at scale.
Starting with the release of Kubeflow 1.7, BentoML provides a native integration with Kubeflow. This integration allows you to package models trained in Kubeflow Notebooks or Pipelines as [Bentos](https://docs.bentoml.org/en/latest/concepts/bento.html), and deploy them as microservices in the Kubernetes cluster through BentoML's cloud native components and custom resource definitions (CRDs). This documentation provides a comprehensive guide on how to use BentoML and Kubeflow together to streamline the process of deploying models at scale.

In this example, we will train three fraud detection models using the Kubeflow notebook and the [Kaggle IEEE-CIS Fraud Detection dataset](https://www.kaggle.com/c/ieee-fraud-detection). We will then create a BentoML service that can simultaneously invoke all three models and return a decision on whether a transaction is fraudulent and build it into a Bento. We will showcase two deployment workflows using BentoML's Kubernetes operators: deploying directly from the Bento, and deploying from an OCI image built from the Bento.

This example can also be run from the `notebook.ipynb` included in this directory.

## Prerequisites

Install Kubeflow and BentoML resources to the Kubernetes cluster. See [Kubeflow](https://github.com/kubeflow/manifests) and [BentoML](https://github.com/kubeflow/manifests/tree/master/contrib/bentoml) installation guides for details.
Install Kubeflow and BentoML resources to the Kubernetes cluster. See [Kubeflow](https://github.com/kubeflow/manifests) and [BentoML](https://github.com/kubeflow/manifests/tree/master/contrib/bentoml) manifests installation guides for details.

After BentoML Kubernetes resources are installed successfully, you should have the following CRDs in the namespace.

Expand Down Expand Up @@ -46,7 +44,7 @@ unzip -d ./data/ ieee-fraud-detection.zip && rm ieee-fraud-detection.zip

## Train Models

In this demonstration, we'll train three fraud detection models using the Kaggle IEEE-CIS Fraud Detection dataset. To showcase saving and serving multiple models with Kubeflow and BentoML, we'll split the dataset into three equal-sized chunks and use each chunk to train a separate model. While this approach has no practical benefits, it will help illustrate how to save and serve multiple models with Kubeflow and BentoML.
In this demonstration, we'll train three fraud detection models using the Kaggle IEEE-CIS Fraud Detection dataset. To showcase saving and serving multiple models with Kubeflow and BentoML, we'll split the dataset into three equal-sized chunks and use each chunk to train a separate model. While this approach has no practical benefits, it will help illustrate how to save and serve multiple models with Kubeflow and BentoML. This step can also be run from the `notebook.ipynb` included in this directory.

```python
import pandas as pd
Expand Down Expand Up @@ -240,7 +238,7 @@ Next, we will demonstrate two ways of deployment.

### Deploy with BentoRequest CRD

In this workflow, we will export the Bento to a remote storage. We will then leverage the `yatai-image-build` operator to containerize the Bento and `yatai-deployment` operator deploy the containerized Bento image.
In this workflow, we will export the Bento to a remote storage, and then use the `yatai-image-builder` operator to containerize it. Finally, we'll deploy the containerized Bento image using the `yatai-deployment` operator. Using the `yatai-image-builder` operator to download Bentos from AWS and push the containerized OCI image will require setting up the credentials of S3 bucket and container repository. See the [manifests installation guide](https://github.com/kubeflow/manifests/tree/master/contrib/bentoml) for detailed instructions.

Push the Bento built and saved in the local Bento store to a cloud storage such as AWS S3.

Expand All @@ -251,22 +249,22 @@ bentoml export fraud_detection:o5smnagbncigycvj s3://your_bucket/fraud_detection
Apply the `BentoRequest` and `BentoDeployment` resources as defined in `deployment_from_bentorequest.yaml` included in this example.

```bash
kubectl apply deployment_from_bentorequest.yaml
kubectl apply -f deployment_from_bentorequest.yaml
```

Once the resources are created, the `yatai-image-builder` operator will reconcile the `BentoRequest` resource and spawn a pod to build the container image from the provided Bento defined in the resource. The `yatai-image-builder` operator will push the built image to the container registry specified during the installation and create a `Bento` resource with the same name. At the same time, the `yatai-deployment` operator will reconcile the `BentoDeployment` resource with the provided name and create Kubernetes deployments of API Servers and Runners from the container image specified in the `Bento` resource.
Once the resources are created, the `yatai-image-builder` operator will reconcile the `BentoRequest` resource and spawn a pod to download and build the container image from the provided Bento defined in the resource. The `yatai-image-builder` operator will push the built image to the container registry specified during the installation and create a `Bento` resource with the same name. At the same time, the `yatai-deployment` operator will reconcile the `BentoDeployment` resource with the provided name and create Kubernetes deployments of API Servers and Runners from the container image specified in the `Bento` resource.

### Deploym with Bento CRD

In this workflow, we will build and push the container image from the Bento. We will then leverage the `yatai-deployment` operator to deploy the containerized Bento image.
In this workflow, we'll create an OCI image from a Bento and upload the image to a container repository. After that, we'll use the `yatai-deployment` operator to deploy the Bento OCI image.

Containerize the image through `containerzie` sub-command.
Containerize the image through `containerize` sub-command.

```bash
bentoml containerize fraud_detection:o5smnagbncigycvj -t your-username/fraud_detection:o5smnagbncigycvj
```

Push the containerized Bento image to a remote repository of your choice.
Push the containerized Bento image to a remote repository of your choice.

```bash
docker push your-username/fraud_detection:o5smnagbncigycvj
Expand All @@ -275,7 +273,7 @@ docker push your-username/fraud_detection:o5smnagbncigycvj
Apply the `Bento` and `BentoDeployment` resources as defined in `deployment_from_bento.yaml` file included in this example.

```bash
kubectl apply deployment_from_bento.yaml
kubectl apply -f deployment_from_bento.yaml
```

Once the resources are created, the `yatai-deployment` operator will reconcile the `BentoDeployment` resource with the provided name and create Kubernetes deployments of API Servers and Runners from the container image specified in the `Bento` resource.
Expand All @@ -296,19 +294,12 @@ fraud-detection-runner-2-6d48794b7-xws4j 3/3 Running 0 10s

![image](https://user-images.githubusercontent.com/861225/226584180-056719cf-0579-4bfb-a5e3-115f7a8808b1.png)


Port forward the Fraud Detection service to test locally. You should be able to visit the Swagger page of the service by requesting http://0.0.0.0:8080 while port forwarding.

```bash
kubectl -n kubeflow port-forward svc/fraud-detection 8080:3000 --address 0.0.0.0
```

Delete the `Bento` and `BentoDeployment` resources.

```bash
kubectl delete -f deployment.yaml
```

## Conclusion

Congratulations! You completed the example. Let's recap what we have learned.
Expand Down
2 changes: 1 addition & 1 deletion examples/kubeflow/notebook.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -321,7 +321,7 @@
"source": [
"## Deploy to Kubernetes Cluster\n",
"\n",
"Great work! You have successfully built and tested the Fraud Detection Bento. Next, we will deploy the bento to the Kubeflow's Kubernetes cluster. Proceed to the README of the example for the next steps."
"Great work! You have successfully built and tested the Fraud Detection Bento. Next, we will deploy the bento to the Kubernetes cluster. Proceed to the README of the example for the next steps."
]
}
],
Expand Down

0 comments on commit c040505

Please sign in to comment.