推理服务‐ml

镜像

ccr.ccs.tencentyun.com/cube-studio/ml-server:20231001

配置config.json

[
    {
        "name": "模型英文名",
        "model_path": "模型地址",
        "algorithm": "decisiontree",
        "version": "20231001",
        "enable": true
    },
    {
        "name": "模型英文名",
        "model_path": "模型地址",
        "algorithm": "r",
        "version": "20231001",
        "enable": true
    }
]

1、地址支持http/https在线地址 2、xgb模型需要保存为.model格式,r语言的模型需要保存为.pmml,sklearn模型需要保存为.pkl

启动命令

python server.py --config_path xxx

标准化模型接口

Model status API：

GET http://host:port/v1/models/${MODEL_NAME}[/versions/${VERSION}|]
示例：
https://demo.service.kfserving.woa.com/v1/models/my_model1/versions/20210924

Model Metadata API

GET http://host:port/v1/models/${MODEL_NAME}[/versions/${VERSION}/metadata
示例：
https://demo.service.kfserving.woa.com/v1/models/my_model1/versions/20210924/metadata

Predict API

POST http://host:port/v1/models/${MODEL_NAME}[/versions/${VERSION}/predict

示例：

http://xx.xx.xx.xx/v1/models/my_model1/versions/20210924:predict