A flexible, high-performance serving system for machine learning models
-
Updated
Jul 19, 2024 - C++
A flexible, high-performance serving system for machine learning models
⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.
A flexible, high-performance carrier for machine learning models(『飞桨』服务化部署框架)
A scalable inference server for models optimized with OpenVINO™
A high-performance inference system for large language models, designed for production environments.
Serving Inside Pytorch With Multi-threads
TensorFlow Serving ARM - A project for cross-compiling TensorFlow Serving targeting popular ARM cores
【深度学习模型部署框架】支持tensorflow/torch/tensorrt/vllm以及更多nn框架,支持dynamic batching、streaming模式,可限制、可拓展、高性能。帮助用户快速地将模型部署到线上,并通过HTTP/RPC接口方式提供服务。
TensorFlow Serving based on encrypted model, protect model files from being stolen | 基于加密模型的 TensorFlow Serving ,保护模型文件免于被盗取
pytorch during training, libtorch during serving via gRPC
A simple tensorflow C++ REST API server
This project implements a common rest server which can serve tensorflow-serving & xgboost models.
tensorflow serving client using brpc
A flexible, high-performance serving system for machine learning models
A flexible, high-performance serving system for machine learning models
Add a description, image, and links to the serving topic page so that developers can more easily learn about it.
To associate your repository with the serving topic, visit your repo's landing page and select "manage topics."