Skip to content
A Python framework for serving and operating machine learning models
Python Shell
Branch: master
Clone or download
lambdaofgod and parano ImageHandler - handling multiple images (#265)
* ImageHandler - handling multiple images

* code style fixes

* swagger schema change
Latest commit bb8b1a4 Aug 19, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Delete ISSUE_TEMPLATE.md Apr 12, 2019
bentoml ImageHandler - handling multiple images (#265) Aug 19, 2019
docs Add BentoML logo image files (#225) Jul 24, 2019
examples Update external facing doc to use Bento instead of BentoArchive (#244) Aug 3, 2019
protos Yatai deployment service implementation MVP (#262) Aug 14, 2019
scripts Release version should follow semantic versioning format (#258) Aug 7, 2019
tests Fix save/load BentoService that has no artifact (#266) Aug 14, 2019
.flake8 Add versioneer.py for managing BentoML version (#253) Aug 7, 2019
.gitattributes Add versioneer.py for managing BentoML version (#253) Aug 7, 2019
.gitignore update development.md (#222) Jul 23, 2019
.pep8speaks.yml Add versioneer.py for managing BentoML version (#253) Aug 7, 2019
.readthedocs.yml manually order api toc, create separate req for sphinx-doc Jul 10, 2019
.travis.yml Auto formatting with python/black (#157) Jun 6, 2019
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md Apr 12, 2019
CONTRIBUTING.md Update readme (#184) Jun 24, 2019
DEVELOPMENT.md update development.md (#222) Jul 23, 2019
LICENSE Moving to Apache 2.0 license (#148) Jun 3, 2019
MANIFEST.in Add versioneer.py for managing BentoML version (#253) Aug 7, 2019
README.md Update external facing doc to use Bento instead of BentoArchive (#244) Aug 3, 2019
pylintrc Add versioneer.py for managing BentoML version (#253) Aug 7, 2019
pyproject.toml Add versioneer.py for managing BentoML version (#253) Aug 7, 2019
setup.cfg Use shorter release tag prefix (#257) Aug 7, 2019
setup.py BentoML Deployment CLI POC (#249) Aug 15, 2019
tox.ini add fastai to py36,py36 deps in tox deps (#208) Jul 16, 2019
versioneer.py Add versioneer.py for managing BentoML version (#253) Aug 7, 2019

README.md

pypi status python versions Downloads build status Documentation Status join BentoML Slack

From a model in jupyter notebook to production API service in 5 minutes

BentoML

Getting Started | Documentation | Examples | Contributing | Releases | License | Blog

BentoML is a python framework for serving and operating machine learning models, making it easy to promote trained models into high performance prediction services.

The framework provides high-level APIs for defining an ML service and packaging its trained model artifacts, preprocessing source code, dependencies, and configurations into a standard file format called Bento - which can be deployed as containerize REST API server, PyPI package, CLI tool, and batch/streaming inference job.

Check out our 5-mins quick start notebook Google Colab Badge using BentoML to productionize a scikit-learn model and deploy it to AWS Lambda.


Getting Started

Installation with pip:

pip install bentoml

Defining a machine learning service with BentoML:

import bentoml
from bentoml.artifact import PickleArtifact
from bentoml.handlers import DataframeHandler

# You can also import your own Python module here and BentoML will automatically
# figure out the dependency chain and package all those Python modules
import my_preproceesing_lib

@bentoml.artifacts([PickleArtifact('model')])
@bentoml.env(pip_dependencies=["scikit-learn"])
class IrisClassifier(bentoml.BentoService):

    @api(DataframeHandler)
    def predict(self, df):
        # Preprocessing prediction request - DataframeHandler parses REST API
        # request or CLI args into pandas Dataframe that can be easily processed
        # into feature vectors that are ready for the trained model
        df = my_preproceesing_lib.process(df)

        # Assess to serialized trained model artifact via self.artifacts
        return self.artifacts.model.predict(df)

After training your ML model, you can pack it with the prediction service IrisClassifier defined above, and save them as a Bento to file system:

from sklearn import svm
from sklearn import datasets

clf = svm.SVC(gamma='scale')
iris = datasets.load_iris()
X, y = iris.data, iris.target
clf.fit(X, y)

# Packaging trained model for serving in production:
saved_path = IrisClassifier.pack(model=clf).save('/tmp/bento')

A Bento is a versioned archive, containing the BentoService you defined, along with trained model artifacts, dependencies and configurations etc. BentoML library can then load in a Bento file and turn it into a high performance prediction service.

For example, you can now start a REST API server based off the saved Bento files:

bentoml serve {saved_path}

Visit http://127.0.0.1:5000 in your browser to play around with the Web UI of the REST API model server, sending testing requests from the UI, or try sending prediction request with curl from CLI:

curl -i \
  --header "Content-Type: application/json" \
  --request POST \
  --data '[[5.1, 3.5, 1.4, 0.2]]' \
  http://localhost:5000/predict

The saved archive can also be used directly from CLI:

bentoml predict {saved_path} --input='[[5.1, 3.5, 1.4, 0.2]]'

# alternatively:
bentoml predict {saved_path} --input='./iris_test_data.csv'

Saved Bento can also be installed and used as a Python PyPI package:

pip install {saved_path}
# Your bentoML model class name will become packaged name
import IrisClassifier

installed_svc = IrisClassifier.load()
installed_svc.predict([[5.1, 3.5, 1.4, 0.2]])

You can also build a docker image for this API server with all dependencies and environments configured automatically by BentoML, and share the docker image with your DevOps team for deployment in production:

docker build -t my_api_server {saved_path}

Try out the full example notebook here on Google Colab.

Examples

More examples can be found under the BentoML/examples directory or the bentoml/gallery repo.

Deployment guides:

Feature Highlights

  • Multiple Distribution Format - Easily package your Machine Learning models and preprocessing code into a format that works best with your inference scenario:

    • Docker Image - deploy as containers running REST API Server
    • PyPI Package - integrate into your python applications seamlessly
    • CLI tool - put your model into Airflow DAG or CI/CD pipeline
    • Spark UDF - run batch serving on a large dataset with Spark
    • Serverless Function - host your model on serverless platforms such as AWS Lambda
  • Multiple Framework Support - BentoML supports a wide range of ML frameworks out-of-the-box including Tensorflow, PyTorch, Keras, Scikit-Learn, xgboost, H2O, FastAI and can be easily extended to work with new or custom frameworks.

  • Deploy Anywhere - BentoML bundled ML service can be easily deployed with platforms such as Docker, Kubernetes, Serverless, Airflow and Clipper, on cloud platforms including AWS, Google Cloud, and Azure.

  • Custom Runtime Backend - Easily integrate your python pre-processing code with high-performance deep learning runtime backend, such as tensorflow-serving.

Documentation

Full documentation and API references can be found at bentoml.readthedocs.io

Usage Tracking

BentoML library by default reports basic usages using Amplitude. It helps BentoML authors to understand how people are using this tool and improve it over time. You can easily opt-out by running the following command from terminal:

bentoml config set usage_tracking=false

Or from your python code:

import bentoml
bentoml.config.set('core', 'usage_tracking', 'false')

We also collect example notebook page views to help us understand the community interests. To opt-out of tracking, delete the ![Impression](http... line in the first markdown cell of our example notebooks.

Contributing

Have questions or feedback? Post a new github issue or join our Slack chat room: join BentoML Slack

Want to help build BentoML? Check out our contributing guide and the development guide.

To make sure you have a pleasant experience, please read the code of conduct. It outlines core values and beliefs and will make working together a happier experience.

Happy hacking!

Releases

BentoML is under active development and is evolving rapidly. Currently it is a Beta release, we may change APIs in future releases.

Read more about the latest features and changes in BentoML from the releases page. and follow the BentoML Community Calendar.

Watch BentoML Github repo for future releases:

gh-watch

License

Apache License 2.0

FOSSA Status

You can’t perform that action at this time.