Skip to content

Commit 5128ce1

Browse files
ChoiByungWooklaurenyu
authored andcommitted
fix merge
1 parent 658545c commit 5128ce1

File tree

1 file changed

+25
-5
lines changed

1 file changed

+25
-5
lines changed

doc/using_mxnet.rst

Lines changed: 25 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -504,11 +504,31 @@ The following code-snippet shows an example custom ``model_fn`` implementation.
504504
"""
505505
Load the gluon model. Called once when hosting service starts.
506506
:param: model_dir The directory where model files are stored.
507-
:return: a model (in this case a Gluon network)
508-
"""
509-
net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10)
510-
net.load_params('%s/model.params' % model_dir, ctx=mx.cpu())
511-
return net
507+
:return: a model (in this case a Gluon network)
508+
"""
509+
net = models.get_model('resnet34_v2', ctx=mx.cpu(), pretrained=False, classes=10)
510+
net.load_params('%s/model.params' % model_dir, ctx=mx.cpu())
511+
return net
512+
513+
MXNet on SageMaker has support for `Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html>`__, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to load and serve your MXNet model through Amazon Elastic Inference, the MXNet context passed to your MXNet Symbol or Module object within your ``model_fn`` needs to be set to ``eia``, as shown `here <https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-mxnet-elastic-inference.html#ei-mxnet>`__.
514+
515+
Based on the example above, the following code-snippet shows an example custom ``model_fn`` implementation, which enables loading and serving our MXNet model through Amazon Elastic Inference.
516+
517+
.. code:: python
518+
519+
def model_fn(model_dir):
520+
"""
521+
Load the gluon model in an Elastic Inference context. Called once when hosting service starts.
522+
:param: model_dir The directory where model files are stored.
523+
:return: a model (in this case a Gluon network)
524+
"""
525+
net = models.get_model('resnet34_v2', ctx=mx.eia(), pretrained=False, classes=10)
526+
net.load_params('%s/model.params' % model_dir, ctx=mx.eia())
527+
return net
528+
529+
The `default_model_fn <https://github.com/aws/sagemaker-mxnet-container/pull/55/files#diff-aabf018d906ed282a3c738377d19a8deR71>`__ will load and serve your model through Elastic Inference, if applicable, within the SageMaker MXNet containers.
530+
531+
For more information on how to enable MXNet to interact with Amazon Elastic Inference, see `Use Elastic Inference with MXNet <https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-mxnet-elastic-inference.html>`__.
512532

513533
Model serving
514534
^^^^^^^^^^^^^

0 commit comments

Comments
 (0)