Permalink
Browse files

Update README.md

  • Loading branch information...
cfregly committed Sep 4, 2018
1 parent fffe5ea commit 24f900425da329fea1792b17d8aa38de699d2cf6
Showing with 19 additions and 19 deletions.
  1. +19 −19 docs/quickstart/sagemaker/README.md
@@ -30,15 +30,15 @@ cd models
* Install [PipelineAI CLI](../README.md#install-pipelinecli)
### Build CPU and GPU Models (TensorFlow-based with TensorFlow Serving)
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
pipeline predict-server-build --model-name=mnist --model-tag=v3cpu --model-type=tensorflow --model-path=./tensorflow/mnist-v3/model --model-chip=cpu
pipeline predict-server-build --model-name=mnist --model-tag=v1cpu --model-type=tensorflow --model-path=./tensorflow/mnist-v1/model --model-chip=cpu
```
* Try different runtimes using `--model-runtime=tensorrt` or `--model-runtime=python`
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
pipeline predict-server-build --model-name=mnist --model-tag=v3gpu --model-type=tensorflow --model-path=./tensorflow/mnist-v3/model --model-chip=gpu
pipeline predict-server-build --model-name=mnist --model-tag=v1gpu --model-type=tensorflow --model-path=./tensorflow/mnist-v1/model --model-chip=gpu
```
* For GPU-based models, make sure you specify `--model-chip=gpu`
@@ -50,14 +50,14 @@ Defaults
* `--image-registry-url`: docker.io
* `--image-registry-repo`: pipelineai
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
pipeline predict-server-register --model-name=mnist --model-tag=v3cpu
pipeline predict-server-register --model-name=mnist --model-tag=v1cpu
```
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
pipeline predict-server-register --model-name=mnist --model-tag=v3gpu
pipeline predict-server-register --model-name=mnist --model-tag=v1gpu
```
### Start TensorFlow Models on AWS SageMaker
@@ -69,19 +69,19 @@ Notes
Examples
* `--aws-iam-arn`: arn:aws:iam::<account-number>:role/service-role/AmazonSageMaker-ExecutionRole...
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
pipeline predict-sage-start --model-name=mnist --model-tag=v3cpu --aws-iam-arn=<aws-iam-arn>
pipeline predict-sage-start --model-name=mnist --model-tag=v1cpu --aws-iam-arn=<aws-iam-arn>
```
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
pipeline predict-sage-start --model-name=mnist --model-tag=v3gpu --aws-iam-arn=<aws-iam-arn>
pipeline predict-sage-start --model-name=mnist --model-tag=v1gpu --aws-iam-arn=<aws-iam-arn>
```
### Split Traffic Between CPU Model (50%) and GPU Model (50%)
```
pipeline predict-sage-route --model-name=mnist --aws-instance-type-dict='{"v3cpu":"ml.p2.xlarge", "v3gpu":"ml.p2.xlarge"}' --model-split-tag-and-weight-dict='{"v3cpu":50, "v3gpu":50}'
pipeline predict-sage-route --model-name=mnist --aws-instance-type-dict='{"v1cpu":"ml.p2.xlarge", "v1gpu":"ml.p2.xlarge"}' --model-split-tag-and-weight-dict='{"v1cpu":50, "v1gpu":50}'
```
Notes:
* You may need to increase your AWS EC2 quotas for the special `ml.p2.xlarge` instance (note the `ml.` prefix).
@@ -119,9 +119,9 @@ Notes:
**Expected Output**
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
('{"variant": "mnist-v3cpu-tensorflow-tfserving-cpu", "outputs":{"outputs": '
('{"variant": "mnist-v1cpu-tensorflow-tfserving-cpu", "outputs":{"outputs": '
'[0.11128007620573044, 1.4478533557849005e-05, 0.43401220440864563, '
'0.06995827704668045, 0.0028081508353352547, 0.27867695689201355, '
'0.017851119861006737, 0.006651509087532759, 0.07679300010204315, '
@@ -130,9 +130,9 @@ Notes:
Request time: 240.805 milliseconds
```
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
('{"variant": "mnist-v3gpu-tensorflow-tfserving-gpu", "outputs":{"outputs": '
('{"variant": "mnist-v1gpu-tensorflow-tfserving-gpu", "outputs":{"outputs": '
'[0.11128010600805283, 1.4478532648354303e-05, 0.43401211500167847, '
'0.06995825469493866, 0.002808149205520749, 0.2786771059036255, '
'0.01785111241042614, 0.006651511415839195, 0.07679297775030136, '
@@ -150,7 +150,7 @@ Notes:
![AWS SageMaker CPU vs. GPU Latency](http://pipeline.ai/assets/img/sagemaker-cpu-gpu-latency.png)
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[CPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
2018-01-09 21:38:04.021915: I tensorflow_serving/model_servers/main.cc:147] Building single TensorFlow model file config: model_name: mnist model_base_path: /root/ml/model/pipeline_tfserving
...
@@ -162,7 +162,7 @@ E0109 21:38:04.210768165 53 ev_epoll1_linux.c:1051] grpc epoll fd: 5
2018-01-09 21:38:04.213992: I tensorflow_serving/model_servers/main.cc:288] Running ModelServer at 0.0.0.0:9000 ...
```
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v3)
[GPU](https://github.com/PipelineAI/models/tree/master/tensorflow/mnist-v1)
```
2018-01-09 21:40:47.842724: I tensorflow_serving/model_servers/main.cc:148] Building single TensorFlow model file config: model_name: mnist model_base_path: /root/ml/model/pipeline_tfserving
...

0 comments on commit 24f9004

Please sign in to comment.