Prerequisites

• You have installed Python 3.11 and pip. See the Python downloads page to learn more.
• You have a basic understanding of key concepts in BentoML, such as Services. We recommend you read Quickstart first.
• (Optional) We recommend you create a virtual environment for dependency isolation for this project. See the Conda documentation or the Python documentation for details.

Install dependencies

git clone https://github.com/bentoml/BentoResnetTensorflow.git
cd BentoResnet
pip install -r requirements.txt

Import model

Run the following commands to download Resnet V2 Object detection model and import it into BentoML's model store

python import_model.py

# list models in BentoML's model store
bentoml models list

Run the BentoML Service

We have defined a BentoML Service in service.py. Run bentoml serve in your project directory to start the Service.

$ bentoml serve .

2024-01-08T09:07:28+0000 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "service:Resnet" can be accessed at http://localhost:3000/metrics.
2024-01-08T09:07:28+0000 [INFO] [cli] Starting production HTTP BentoServer from "service:Resnet" listening on http://localhost:3000 (Press CTRL+C to quit)
Model resnet loaded device: cuda

The Service is accessible at http://localhost:3000. You can interact with it using the Swagger UI or in other different ways:

CURL

curl -s \
     -X POST \
     -F 'images=@cat1.jpg' \
     http://localhost:3000/classify

Python client

import bentoml
from pathlib import Path

with bentoml.SyncHTTPClient("http://localhost:3000") as client:
    result = client.classify(
        images=[
            Path("cat1.jpg"),
        ],
    )

Deploy to BentoCloud

After the Service is ready, you can deploy the application to BentoCloud for better management and scalability. Sign up if you haven't got a BentoCloud account.

Make sure you have logged in to BentoCloud, then run the following command to deploy it.

bentoml deploy .

Once the application is up and running on BentoCloud, you can access it via the exposed URL.