A flexible, high-performance serving system for machine learning models
-
Updated
Mar 31, 2018 - C++
A flexible, high-performance serving system for machine learning models
This project implements a common rest server which can serve tensorflow-serving & xgboost models.
A simple tensorflow C++ REST API server
A flexible, high-performance serving system for machine learning models
pytorch during training, libtorch during serving via gRPC
tensorflow serving client using brpc
TensorFlow Serving ARM - A project for cross-compiling TensorFlow Serving targeting popular ARM cores
TensorFlow Serving based on encrypted model, protect model files from being stolen | 基于加密模型的 TensorFlow Serving ,保护模型文件免于被盗取
⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.
A flexible, high-performance carrier for machine learning models(『飞桨』服务化部署框架)
Boosting DL Service Throughput 1.5-4x by Ensemble Pipeline Serving with Concurrent CUDA Streams for PyTorch/LibTorch Frontend and TensorRT/CVCUDA, etc., Backends
A flexible, high-performance serving system for machine learning models
A high-performance inference system for large language models, designed for production environments.
A scalable inference server for models optimized with OpenVINO™
Add a description, image, and links to the serving topic page so that developers can more easily learn about it.
To associate your repository with the serving topic, visit your repo's landing page and select "manage topics."