Skip to content
A platform for serving and deploying machine learning models
Python Jupyter Notebook Other
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Delete ISSUE_TEMPLATE.md Apr 12, 2019
bentoml doc: tf_savedmodel_artifact docstring (#408) Nov 28, 2019
docs [DOC] add TensorflowSavedModelArtifact doc (#368) Nov 8, 2019
guides Update project readme and quick start guide notebook (#403) Nov 26, 2019
protos aws lambda deployment v2 (#380) Nov 26, 2019
scripts AWS Lambda Deployment misc improvements (#407) Nov 28, 2019
tests AWS Lambda Deployment misc improvements (#407) Nov 28, 2019
.flake8 Add versioneer.py for managing BentoML version (#253) Aug 7, 2019
.gitattributes Add versioneer.py for managing BentoML version (#253) Aug 7, 2019
.gitignore Implement feature uploading bento repo to S3 (#356) Oct 26, 2019
.pep8speaks.yml
.readthedocs.yml manually order api toc, create separate req for sphinx-doc Jul 10, 2019
.travis.yml Removing Python2.7 form test environments list (#398) Nov 25, 2019
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md Apr 12, 2019
CONTRIBUTING.md Update readme (#184) Jun 24, 2019
DEVELOPMENT.md Update README.md and quick start notebook (#346) Oct 16, 2019
LICENSE Moving to Apache 2.0 license (#148) Jun 3, 2019
MANIFEST.in Include migration files in PyPI distribution (#350) Oct 17, 2019
README.md Update README to include the new LightGBM example (#404) Nov 26, 2019
pylintrc Remove opencv-python module from test dependency (#389) Nov 20, 2019
pyproject.toml Update quick-start guide for coming release & MISC (#295) Sep 17, 2019
pytest.ini Fix BentoService bundle pip package dependencies (#396) Nov 25, 2019
setup.cfg Use shorter release tag prefix (#257) Aug 7, 2019
setup.py AWS Lambda Deployment misc improvements (#407) Nov 28, 2019
tox.ini AWS Lambda Deployment misc improvements (#407) Nov 28, 2019
versioneer.py Add versioneer.py for managing BentoML version (#253) Aug 7, 2019

README.md

pypi status python versions Downloads build status Documentation Status join BentoML Slack

From ML model to production API endpoint with a few lines of code

BentoML

Getting Started | Documentation | Gallery | Contributing | Releases | License | Blog

BentoML makes it easy to serve and deploy machine learning models in the cloud.

It is an open source framework for machine learning teams to build cloud-native prediction API services that are ready for production. BentoML supports most popular ML training frameworks and common deployment platforms including major cloud providers and docker/kubernetes.

👉 Join BentoML Slack community to hear about the latest development updates.


Getting Started

Installation with pip:

pip install bentoml

Defining a prediction service with BentoML:

import bentoml
from bentoml.handlers import DataframeHandler
from bentoml.artifact import SklearnModelArtifact

@bentoml.env(pip_dependencies=["scikit-learn"]) # defining pip/conda dependencies to be packed
@bentoml.artifacts([SklearnModelArtifact('model')]) # defining required artifacts, typically trained models
class IrisClassifier(bentoml.BentoService):

    @bentoml.api(DataframeHandler) # defining prediction service endpoint and expected input format
    def predict(self, df):
        # Pre-processing logic and access to trained mdoel artifacts in API function
        return self.artifacts.model.predict(df)

Train a classifier model with default Iris dataset and pack the trained model with the BentoService IrisClassifier defined above:

from sklearn import svm
from sklearn import datasets

if __name__ == "__main__":
    clf = svm.SVC(gamma='scale')
    iris = datasets.load_iris()
    X, y = iris.data, iris.target
    clf.fit(X, y)

    # Create a iris classifier service
    iris_classifier_service = IrisClassifier()

    # Pack it with the newly trained model artifact
    iris_classifier_service.pack('model', clf)

    # Save the prediction service to a BentoService bundle
    saved_path = iris_classifier_service.save()

A BentoService bundle is a versioned file archive, containing the BentoService you defined, along with trained model artifacts, dependencies and configurations.

Now you can start a REST API server based off the saved BentoService bundle form command line:

bentoml serve {saved_path}

If you are doing this only local machine, visit http://127.0.0.1:5000 in your browser to play around with the API server's Web UI for debbugging and sending test request. You can also send prediction request with curl from command line:

curl -i \
  --header "Content-Type: application/json" \
  --request POST \
  --data '[[5.1, 3.5, 1.4, 0.2]]' \
  http://localhost:5000/predict

Saved BentoService bundle is also structured to work as a docker build context, which can be used to build a docker image for deployment:

docker build -t my_api_server {saved_path}

The saved BentoService bundle can also be loaded directly from command line:

bentoml predict {saved_path} --input='[[5.1, 3.5, 1.4, 0.2]]'

# alternatively:
bentoml predict {saved_path} --input='./iris_test_data.csv'

The saved bundle is pip-installable and can be directly distributed as a PyPI package:

pip install {saved_path}
# Your BentoService class name will become packaged name
import IrisClassifier

installed_svc = IrisClassifier.load()
installed_svc.predict([[5.1, 3.5, 1.4, 0.2]])

Deploy the saved BentoService to cloud services such as AWS Lambda with the bentoml command:

bentoml deployment create my-iris-classifier --bento IrisClassifier:{VERSION} --platform=aws-lambda

To learn more, try out our 5-mins Quick Start notebook using BentoML to turn a trained sklearn model into a containerized REST API server, and then deploy it to AWS Lambda: Download, Google Colab, nbviewer

Examples

FastAI

Scikit-Learn

PyTorch

Keras

XGBoost

LightGBM

H2O

Visit bentoml/gallery repository for more example projects demonstrating how to use BentoML.

Deployment guides:

Feature Highlights

  • Multiple Distribution Format - Easily package your Machine Learning models and preprocessing code into a format that works best with your inference scenario:

    • Docker Image - deploy as containers running REST API Server
    • PyPI Package - integrate into your python applications seamlessly
    • CLI tool - put your model into Airflow DAG or CI/CD pipeline
    • Spark UDF - run batch serving on a large dataset with Spark
    • Serverless Function - host your model on serverless platforms such as AWS Lambda
  • Multiple Framework Support - BentoML supports a wide range of ML frameworks out-of-the-box including Tensorflow, PyTorch, Keras, Scikit-Learn, xgboost, H2O, FastAI and can be easily extended to work with new or custom frameworks

  • Deploy Anywhere - BentoService bundle can be easily deployed with platforms such as Docker, Kubernetes, Serverless, Airflow and Clipper, on cloud platforms including AWS, Google Cloud, and Azure

  • Custom Runtime Backend - Easily integrate your python pre-processing code with high-performance deep learning runtime backend, such as tensorflow-serving

  • Workflow Designed For Teams - The YataiService component in BentoML provides Web UI and APIs for managing and deploying all the models and preidciton services your team has created or deployed, in a centralized service.

Documentation

Full documentation and API references can be found at bentoml.readthedocs.io

Usage Tracking

BentoML library by default reports basic usages using Amplitude. It helps BentoML authors to understand how people are using this tool and improve it over time. You can easily opt-out by running the following command from terminal:

bentoml config set usage_tracking=false

Contributing

Have questions or feedback? Post a new github issue or discuss in our Slack channel: join BentoML Slack

Want to help build BentoML? Check out our contributing guide and the development guide.

Releases

BentoML is under active development and is evolving rapidly. Currently it is a Beta release, we may change APIs in future releases.

Read more about the latest features and changes in BentoML from the releases page.

License

Apache License 2.0

FOSSA Status

You can’t perform that action at this time.