#

serving

Here are 2 public repositories matching this topic...

kozistr / catboost-server-rs

CatBoost server in Rust + gRPC

rust machine-learning server grpc serving catboost

Updated Sep 9, 2022
Rust

LessUp / hetero-paged-infer

High-Performance LLM Inference Engine with PagedAttention & Continuous Batching | 高性能LLM推理引擎 - 内存浪费<5%, 吞吐率+50%

rust machine-learning high-performance transformer gpu-computing production-ready systems-programming inference-engine serving kv-cache llm vllm llm-inference paged-attention continuous-batching

Updated Apr 22, 2026
Rust

Improve this page

Add a description, image, and links to the serving topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the serving topic, visit your repo's landing page and select "manage topics."