In this example we will show how we can wrap a complex CPP project by extending the buildsystem defaults provided, which will give us flexibility to configure the required bindings.
If you are looking for a basic implementation of the C++ wrapper, you can get started with the "Single file C++ Example".
You can read about how to configure your environment in the CPP Wrapper documentation page.
In this example we will have full control on naming conventions.
More specifically there are a few key naming conventions that we need to consider:
- Python Module name
- Python Wrapper Class name
- C++ Library Name
As long as we keep these three key naming conventions in mind, we will have full flexibility on the entire build system.
For this project we will choose the following naming conventions:
- Python Module Name:
CustomSeldonPackage - Python Wrapper Class:
MyModelClass - C++ Library Name:
CustomSeldonPackage
As you can see, the name of the Python Module and C++ Library can be the same.
We will first start with the wrapper code of our example. We'll first create our file Main.cpp and we'll explain in detail each section below.
%%writefile Main.cpp
#include "seldon/SeldonModel.hpp"
class MyModelClass : public seldon::SeldonModelBase {
seldon::protos::SeldonMessage predict(seldon::protos::SeldonMessage &data) override {
return data;
}
};
SELDON_BIND_MODULE(CustomSeldonPackage, MyModelClass)Overwriting Main.cpp
In this file we basically have to note the following key points:
- We import
"seldon/SeldonModel.hpp"which is from the Seldon package - We use our custom class name
"MyModelClass" - We extend the
SeldonModelBaseclass which processes the protos for us - We override the
predict()function which provides the raw protos - We register our class as
SELDON_BIND_MODULEpassing the package name and class name
For the build system we have integrated with CMake, as this provides quite a lot of flexibility, and easy integration with external projects.
In this case below are the minimal configurations required in order for everything to work smoothly. The key components to note are:
- We fetch the seldon and pybind11 packages
- We register our C++ library with the name
CustomSeldonMessage - We bind the package with the seldon library
You are able to extend the points below as required.
%%writefile CMakeLists.txt
cmake_minimum_required(VERSION 3.4.1)
project(seldon_custom_model VERSION 0.0.1)
set(CMAKE_CXX_STANDARD 14)
find_package(seldon REQUIRED)
find_package(pybind11 REQUIRED)
pybind11_add_module(
CustomSeldonPackage
Main.cpp)
target_link_libraries(
CustomSeldonPackage PRIVATE
seldon::seldon)Overwriting CMakeLists.txt
The final component is to specify the environment variables.
FOr this we can either pass the env variable as a parameter to the s2i command below, or in this example we'll approach it by the other option which is creating an environment file in the .s2i/environment file.
The environment variable is MODEL_NAME, which should contain the name of your package and model.
In our case it is CustomSeldonPackage.MyModelClass as follows:
!mkdir -p .s2i/%writefile .s2i/environment
MODEL_NAME = CustomSeldonPackage.MyModelClassUsageError: Line magic function `%writefile` not found (But cell magic `%%writefile` exists, did you mean that instead?).
In our case we won't have to pass any custom CMAKE parameters as we can configure everything through the CMakeLists.txt, but if you wish to modify how your C++ wrapper is packaged you can extend the setup.py file by following the details in the CPP Wrapper documentation page.
We can now build our seldon microservice using s2i:
!s2i build . seldonio/s2i-cpp-build:0.0.1 seldonio/advanced-cpp:0.1---> Installing application source...
---> Installing application ...
Looking in links: /whl
Obtaining file:///microservice
Installing collected packages: SeldonPackage
Running setup.py develop for SeldonPackage
Successfully installed SeldonPackage
Collecting pip-licenses
Downloading https://files.pythonhosted.org/packages/08/b4/6e53ab4e82e2b9f8356dd17d7b9e30cba57ba0460186c92cc05e8a1a7f97/pip_licenses-3.0.0-py3-none-any.whl
Collecting PTable (from pip-licenses)
Downloading https://files.pythonhosted.org/packages/ab/b3/b54301811173ca94119eb474634f120a49cd370f257d1aae5a4abaf12729/PTable-0.9.2.tar.gz
Building wheels for collected packages: PTable
Building wheel for PTable (setup.py): started
Building wheel for PTable (setup.py): finished with status 'done'
Created wheel for PTable: filename=PTable-0.9.2-cp37-none-any.whl size=22906 sha256=98facc4ac39cd0e7c89a7c87587cf9941e9aa75817f105b8e5e01b499d1efb2a
Stored in directory: /root/.cache/pip/wheels/22/cc/2e/55980bfe86393df3e9896146a01f6802978d09d7ebcba5ea56
Successfully built PTable
Installing collected packages: PTable, pip-licenses
Successfully installed PTable-0.9.2 pip-licenses-3.0.0
created path: ./licenses/license_info.csv
created path: ./licenses/license.txt
Build completed successfully
!docker run --name "advanced_cpp" -d --rm -p 9000:9000 seldonio/advanced-cpp:0.1aaa5795779f2e605f7ead2772e912c8dd7de04002457eb4b3966b2b2182c63f4
!curl -X POST -H 'Content-Type: application/json' \
-d '{"strData":"hello"}' \
http://localhost:9000/api/v1.0/predictions{"strData":"hello"}
!docker rm -f "advanced_cpp"advanced_cpp
%%bash
kubectl apply -f - << END
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: advanced-cpp
spec:
predictors:
- componentSpecs:
- spec:
containers:
- image: seldonio/advanced-cpp:0.1
name: classifier
engineResources: {}
graph:
name: classifier
type: MODEL
name: default
replicas: 1
ENDseldondeployment.machinelearning.seldon.io/advanced-cpp created
!curl -X POST -H 'Content-Type: application/json' \
-d '{"strData":"hello"}' \
http://localhost:80/seldon/default/advanced-cpp/api/v1.0/predictions{"strData":"hello"}
!kubectl delete sdep advanced-cppseldondeployment.machinelearning.seldon.io "advanced-cpp" deleted