diff --git a/README.rst b/README.rst index 1d8d99600e..b8a516266d 100644 --- a/README.rst +++ b/README.rst @@ -45,26 +45,25 @@ inference service via an HTTP or gRPC endpoint, allowing remote clients to request inferencing for any model being managed by the server. TRTIS provides the following features: -* Multiple model support. The server can manage any number and mix of +* `Multiple framework support `_. The server can manage any number and mix of models (limited by system disk and memory resources). Supports TensorRT, TensorFlow GraphDef, TensorFlow SavedModel and Caffe2 NetDef model formats. Also supports TensorFlow-TensorRT integrated models. * Multi-GPU support. The server can distribute inferencing across all system GPUs. -* Multi-tenancy support. Multiple models (or multiple instances of the +* `Concurrent model execution support `_. Multiple models (or multiple instances of the same model) can run simultaneously on the same GPU. * Batching support. For models that support batching, the server can accept requests for a batch of inputs and respond with the - corresponding batch of outputs. The server also supports *dynamic - batching* where individual inference requests are dynamically + corresponding batch of outputs. The server also supports `dynamic + batching `_ where individual inference requests are dynamically combined together to improve inference throughput. Dynamic batching is transparent to the client requesting inference. -* Model repositories may reside on a locally accessible file system or +* `Model repositories `_ may reside on a locally accessible file system (e.g. NFS) or in Google Cloud Storage. -* Readiness and liveness health endpoints suitable for - Kubernetes-style orchestration. -* Metrics indicating GPU utiliization, server throughput, and server +* Readiness and liveness `health endpoints `_ suitable for any orchestration or deployment framework, such as Kubernetes. +* `Metrics `_ indicating GPU utiliization, server throughput, and server latency. .. overview-end-marker-do-not-remove