Deploy BentoML service as serverless function on AWS Lambda or Google Cloud Function
Cloud providers offer serverless computing service to help teams deploy a scalable services without worry about hardware configuration and maintains. This benefit may also apply to machine learning as well.
In this example, we will train a sentiment analysis model with SciKit-learn, and then use BentoML to create serverless archive and deploy to AWS lambda service with the Serverless framework.
We will exame the benefits and challenges of deploying machine learning services to serverless services. And then, we will use BentoML to deploy the trained model to AWS Lambda with one command, and take a closer look of the entire process.
Developing sentiment analysis model
We will train a scikit-learn model, and then we will use BentoML to package it. You can follow those steps in the Jupyter Notebook
Prerequisites for deploying model to AWS Lambda
- Install Node.JS. Follow the instructions on Nodejs.org
- Install Serverless framework. You can find instructions here
- AWS account configured on your machine
Deploy to AWS lambda
It is simple to deploy to AWS Lambda with BentoML. After you saved your model as BentoML bundle, you invoke a single command.
You have two ways to invoke that command.
One way is invoke the command inside the training notebook. You add a new cell after you save the model with BentoML.
Make sure you add
! before you call any command
!bentoml deploy /SAVED_PATH --platform aws-lambda --region us-west-2
The second way is invoke the same command from a terminal.
bentoml deploy /ARCHIVE_PATH --platform aws-lambda --region us-west-2
What happens after the deploy command
BentoML does serveal things under the hood that help data scientists to deploy their model services to AWS lambda.
BentoML will first generate a serverless project and store it in the snapshot directory. The snapshot contains your BentoML model archive and configuration files for AWS Lambda
BentoML will invoke AWS to create different services. Those services managed by CloudFormation. When the process complete, you will see the Lambda function in service.
To make a prediction request, you can use CURL command. Copy and paste the following command and update
base on your deployment
curl -i \ --header "Content-Type: application/json" \ --request POST \ --data '["good movie", "bad food"]' \ https://URL
Check deployment status
bentoml check-deployment-status /ARCHIVE_PATH --platform aws-lambda -region us-west-2
Delete deployment from AWS lambda
Delete deployment from AWS lambda is as simple as deploy it. To delete deployment use
bentoml delete-deployment command.
bentoml delete-deployment /ARCHIVE_PATH --platform aws-lambda --region us-west-2