diff --git a/doc/using_mxnet.rst b/doc/using_mxnet.rst index 092693ae08..d6b06a8bd1 100644 --- a/doc/using_mxnet.rst +++ b/doc/using_mxnet.rst @@ -1,8 +1,11 @@ Using MXNet Estimators and Models --------------------------------- +----------------- +Table of Contents +----------------- .. contents:: - + :local: With MXNet estimators, you can train and host MXNet models on Amazon SageMaker. @@ -530,11 +533,31 @@ The following code-snippet shows an example custom ``model_fn`` implementation. """ Load the gluon model. Called once when hosting service starts. :param: model_dir The directory where model files are stored. - :return: a model (in this case a Gluon network) - """ - net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) - net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) - return net + :return: a model (in this case a Gluon network) + """ + net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) + return net + +MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module object within your ``model_fn`` needs to be set to ``eia``, as shown `here `__. + +Based on the example above, the following code-snippet shows an example custom ``model_fn`` implementation, which enables loading and serving our MXNet model through Amazon Elastic Inference. + +.. code:: python + + def model_fn(model_dir): + """ + Load the gluon model in an Elastic Inference context. Called once when hosting service starts. + :param: model_dir The directory where model files are stored. + :return: a model (in this case a Gluon network) + """ + net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) + return net + +The `default_model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers. + +For more information on how to enable MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet `__. Model serving ^^^^^^^^^^^^^ diff --git a/src/sagemaker/mxnet/README.rst b/src/sagemaker/mxnet/README.rst index b6f09e75a3..2e34878372 100644 --- a/src/sagemaker/mxnet/README.rst +++ b/src/sagemaker/mxnet/README.rst @@ -487,7 +487,7 @@ After calling ``fit``, you can call ``deploy`` on an ``MXNet`` Estimator to crea You use the SageMaker MXNet model server to host your MXNet model when you call ``deploy`` on an ``MXNet`` Estimator. The model server runs inside a SageMaker Endpoint, which your call to ``deploy`` creates. You can access the name of the Endpoint by the ``name`` property on the returned ``Predictor``. -MXNet on SageMaker has support for `Elastic Inference `_, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to attach an Elastic Inference accelerator to your endpoint provide the accelerator type to ``accelerator_type`` to your ``deploy`` call. +MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to attach an Elastic Inference accelerator to your endpoint provide the accelerator type to ``accelerator_type`` to your ``deploy`` call. .. code:: python @@ -525,11 +525,31 @@ The following code-snippet shows an example custom ``model_fn`` implementation. """ Load the gluon model. Called once when hosting service starts. :param: model_dir The directory where model files are stored. - :return: a model (in this case a Gluon network) - """ - net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) - net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) - return net + :return: a model (in this case a Gluon network) + """ + net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.cpu()) + return net + +MXNet on SageMaker has support for `Elastic Inference `__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module object within your ``model_fn`` needs to be set to ``eia``, as shown `here `__. + +Based on the example above, the following code-snippet shows an example custom ``model_fn`` implementation, which enables loading and serving our MXNet model through Amazon Elastic Inference. + +.. code:: python + + def model_fn(model_dir): + """ + Load the gluon model in an Elastic Inference context. Called once when hosting service starts. + :param: model_dir The directory where model files are stored. + :return: a model (in this case a Gluon network) + """ + net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10) + net.load_params('%s/model.params' % model_dir, ctx=mx.eia()) + return net + +The `default_model_fn `__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers. + +For more information on how to enable MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet `__. Model serving ^^^^^^^^^^^^^