From 27e9eea62436cd63d037bd14b99239b1854406ae Mon Sep 17 00:00:00 2001 From: Dan Choi Date: Wed, 30 Jan 2019 17:24:16 -0800 Subject: [PATCH 1/4] add section on ei context --- src/sagemaker/mxnet/README.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/sagemaker/mxnet/README.rst b/src/sagemaker/mxnet/README.rst index b6f09e75a3..ba74b61e37 100644 --- a/src/sagemaker/mxnet/README.rst +++ b/src/sagemaker/mxnet/README.rst @@ -487,7 +487,7 @@ After calling ``fit``, you can call ``deploy`` on an ``MXNet`` Estimator to crea You use the SageMaker MXNet model server to host your MXNet model when you call ``deploy`` on an ``MXNet`` Estimator. The model server runs inside a SageMaker Endpoint, which your call to ``deploy`` creates. You can access the name of the Endpoint by the ``name`` property on the returned ``Predictor``. -MXNet on SageMaker has support for `Elastic Inference `_, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to attach an Elastic Inference accelerator to your endpoint provide the accelerator type to ``accelerator_type`` to your ``deploy`` call. +MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to attach an Elastic Inference accelerator to your endpoint provide the accelerator type to ``accelerator_type`` to your ``deploy`` call. .. code:: python @@ -531,6 +531,12 @@ The following code-snippet shows an example custom ``model_fn`` implementation. net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) return net +MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module API within your ``model_fn`` needs to be set to ``eia``, as shown `here `__. + +The `default model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers. + +* For more information on how to enable the enhanced version of MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet `__. + Model serving ^^^^^^^^^^^^^ From c83d1fe984d383cf0e4b941301b4f02df6992d16 Mon Sep 17 00:00:00 2001 From: Dan Choi Date: Wed, 30 Jan 2019 17:31:48 -0800 Subject: [PATCH 2/4] add code example --- src/sagemaker/mxnet/README.rst | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/src/sagemaker/mxnet/README.rst b/src/sagemaker/mxnet/README.rst index ba74b61e37..f8082b37c4 100644 --- a/src/sagemaker/mxnet/README.rst +++ b/src/sagemaker/mxnet/README.rst @@ -531,11 +531,25 @@ The following code-snippet shows an example custom ``model_fn`` implementation. net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) return net -MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module API within your ``model_fn`` needs to be set to ``eia``, as shown `here `__. +MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module object within your ``model_fn`` needs to be set to ``eia``, as shown `here `__. -The `default model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers. +Based on the example above, the following code-snippet shows an example custom ``model_fn`` implementation, which enables loading and serving our MXNet model through Amazon Elastic Inference. -* For more information on how to enable the enhanced version of MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet `__. +.. code:: python + + def model_fn(model_dir): + """ + Load the gluon model in an Elastic Inference context. Called once when hosting service starts. + :param: model_dir The directory where model files are stored. + :return: a model (in this case a Gluon network) + """ + net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) + return net + +The `default_model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers. + +* For more information on how to enable MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet `__. Model serving ^^^^^^^^^^^^^ From b6fd452e62c1500f6d576551602a0fbf0b41973b Mon Sep 17 00:00:00 2001 From: Dan Choi Date: Thu, 31 Jan 2019 11:42:29 -0800 Subject: [PATCH 3/4] fix indent, remove bullet and add to sphinx section --- doc/using_mxnet.rst | 29 ++++++++++++++++++++++++++--- src/sagemaker/mxnet/README.rst | 10 +++++----- 2 files changed, 31 insertions(+), 8 deletions(-) diff --git a/doc/using_mxnet.rst b/doc/using_mxnet.rst index 092693ae08..29daf33e87 100644 --- a/doc/using_mxnet.rst +++ b/doc/using_mxnet.rst @@ -1,8 +1,11 @@ Using MXNet Estimators and Models --------------------------------- +----------------- +Table of Contents +----------------- .. contents:: - + :local: With MXNet estimators, you can train and host MXNet models on Amazon SageMaker. @@ -530,12 +533,32 @@ The following code-snippet shows an example custom ``model_fn`` implementation. """ Load the gluon model. Called once when hosting service starts. :param: model_dir The directory where model files are stored. - :return: a model (in this case a Gluon network) - """ + :return: a model (in this case a Gluon network) + """ net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) return net +MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module object within your ``model_fn`` needs to be set to ``eia``, as shown `here `__. + +Based on the example above, the following code-snippet shows an example custom ``model_fn`` implementation, which enables loading and serving our MXNet model through Amazon Elastic Inference. + +.. code:: python + + def model_fn(model_dir): + """ + Load the gluon model in an Elastic Inference context. Called once when hosting service starts. + :param: model_dir The directory where model files are stored. + :return: a model (in this case a Gluon network) + """ + net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) + return net + +The `default_model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers. + +For more information on how to enable MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet `__. + Model serving ^^^^^^^^^^^^^ diff --git a/src/sagemaker/mxnet/README.rst b/src/sagemaker/mxnet/README.rst index f8082b37c4..3a1cd644ce 100644 --- a/src/sagemaker/mxnet/README.rst +++ b/src/sagemaker/mxnet/README.rst @@ -525,8 +525,8 @@ The following code-snippet shows an example custom ``model_fn`` implementation. """ Load the gluon model. Called once when hosting service starts. :param: model_dir The directory where model files are stored. - :return: a model (in this case a Gluon network) - """ + :return: a model (in this case a Gluon network) + """ net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) return net @@ -541,15 +541,15 @@ Based on the example above, the following code-snippet shows an example custom ` """ Load the gluon model in an Elastic Inference context. Called once when hosting service starts. :param: model_dir The directory where model files are stored. - :return: a model (in this case a Gluon network) - """ + :return: a model (in this case a Gluon network) + """ net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) return net The `default_model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers. -* For more information on how to enable MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet `__. +For more information on how to enable MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet `__. Model serving ^^^^^^^^^^^^^ From ea8e84f6e9b837be788d7115cf19c76b57cb1db9 Mon Sep 17 00:00:00 2001 From: Dan Choi Date: Thu, 31 Jan 2019 11:56:31 -0800 Subject: [PATCH 4/4] fix overindent --- doc/using_mxnet.rst | 12 ++++++------ src/sagemaker/mxnet/README.rst | 12 ++++++------ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/doc/using_mxnet.rst b/doc/using_mxnet.rst index 29daf33e87..d6b06a8bd1 100644 --- a/doc/using_mxnet.rst +++ b/doc/using_mxnet.rst @@ -535,9 +535,9 @@ The following code-snippet shows an example custom ``model_fn`` implementation. :param: model_dir The directory where model files are stored. :return: a model (in this case a Gluon network) """ - net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) - net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) - return net + net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) + return net MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module object within your ``model_fn`` needs to be set to ``eia``, as shown `here `__. @@ -551,9 +551,9 @@ Based on the example above, the following code-snippet shows an example custom ` :param: model_dir The directory where model files are stored. :return: a model (in this case a Gluon network) """ - net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) - net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) - return net + net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) + return net The `default_model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers. diff --git a/src/sagemaker/mxnet/README.rst b/src/sagemaker/mxnet/README.rst index 3a1cd644ce..2e34878372 100644 --- a/src/sagemaker/mxnet/README.rst +++ b/src/sagemaker/mxnet/README.rst @@ -527,9 +527,9 @@ The following code-snippet shows an example custom ``model_fn`` implementation. :param: model_dir The directory where model files are stored. :return: a model (in this case a Gluon network) """ - net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) - net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) - return net + net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) + return net MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module object within your ``model_fn`` needs to be set to ``eia``, as shown `here `__. @@ -543,9 +543,9 @@ Based on the example above, the following code-snippet shows an example custom ` :param: model_dir The directory where model files are stored. :return: a model (in this case a Gluon network) """ - net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) - net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) - return net + net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) + return net The `default_model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers.