diff --git a/README.md b/README.md index b719fd6094..54e8bfa51f 100644 --- a/README.md +++ b/README.md @@ -97,15 +97,17 @@ To look up arguments run: `deepsparse.server --help`. **⭐ Multiple Models ⭐** To serve multiple models in your deployment you can easily build a `config.yaml`. In the example below, we define two BERT models in our configuration for the question answering task: - models: +```yaml +models: - task: question_answering - model_path: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/base-none - batch_size: 1 - alias: question_answering/dense + model_path: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/base-none + batch_size: 1 + alias: question_answering/base - task: question_answering - model_path: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/pruned_quant-aggressive_95 - batch_size: 1 - alias: question_answering/sparse_quantized + model_path: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/pruned_quant-aggressive_95 + batch_size: 1 + alias: question_answering/pruned_quant +``` Finally, after your `config.yaml` file is built, run the server with the config file path as an argument: ```bash diff --git a/src/deepsparse/server/README.md b/src/deepsparse/server/README.md index cd7dae595a..5904eef558 100644 --- a/src/deepsparse/server/README.md +++ b/src/deepsparse/server/README.md @@ -89,16 +89,16 @@ __ __ To serve multiple models you can build a `config.yaml` file. In the sample YAML file below, we are defining two BERT models to be served by the `deepsparse.server` for the **question answering** task: -``` +```yaml models: - task: question_answering - model_path: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/base-none - batch_size: 1 - alias: question_answering/base + model_path: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/base-none + batch_size: 1 + alias: question_answering/base - task: question_answering - model_path: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/pruned_quant-aggressive_95 - batch_size: 1 - alias: question_answering/pruned_quant + model_path: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/pruned_quant-aggressive_95 + batch_size: 1 + alias: question_answering/pruned_quant ``` You can now run the server with the config file path passed in the `--config_file` argument: