- Model Server binary in tensorflow_serving/model_servers with a PredictionService API.
- Support SavedModel format and added ability to upconvert legacy SessionBundle exports. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/README.md
- NOTE: SessionBundle is now deprecated and we plan to end official support in the upcoming 1.0 release. Please move to use SavedModel.
- Multi-model batch scheduling: interleaveing batches for different models.
- Registry of servable types for Model Server to handle ones other than SessionBundle/SavedModel, including third-party ones not in TF-Serving codebase.
- Resource management: using model size estimates to avoid exceeding server memory capacity.
Concurrent model loading & fast initial load.
- Request logging:
- A protobuf based logging API.
- Provides ability for users to log a configurable sample, or all, of the queries served.
- Support querying named signatures.
- Assembled core manager setup code into a ServerCore object.
- Various bug fixes.
- Documentation updates.