-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feature: support MXNet 1.4 with MMS #812
Changes from 3 commits
760e1fe
1e896ec
2a9ab73
62c02fe
ddec477
1ac3a8c
92d12c7
b779403
4f6ef74
e26eb92
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,9 +14,11 @@ | |
|
||
import json | ||
import logging | ||
import os | ||
|
||
import sagemaker | ||
from sagemaker import fw_utils, local, session, utils | ||
from sagemaker.fw_utils import UploadedCode | ||
from sagemaker.transformer import Transformer | ||
|
||
LOGGER = logging.getLogger('sagemaker') | ||
|
@@ -408,6 +410,7 @@ def __init__(self, model_data, image, role, entry_point, source_dir=None, predic | |
else: | ||
self.bucket, self.key_prefix = None, None | ||
self.uploaded_code = None | ||
self.repacked_model_data = None | ||
|
||
def prepare_container_def(self, instance_type, accelerator_type=None): # pylint disable=unused-argument | ||
"""Return a container definition with framework configuration set in model environment variables. | ||
|
@@ -428,18 +431,28 @@ def prepare_container_def(self, instance_type, accelerator_type=None): # pylint | |
deploy_env.update(self._framework_env_vars()) | ||
return sagemaker.container_def(self.image, self.model_data, deploy_env) | ||
|
||
def _upload_code(self, key_prefix): | ||
def _upload_code(self, key_prefix, repack=False): | ||
local_code = utils.get_config_value('local.local_code', self.sagemaker_session.config) | ||
if self.sagemaker_session.local_mode and local_code: | ||
self.uploaded_code = None | ||
else: | ||
bucket = self.bucket or self.sagemaker_session.default_bucket() | ||
self.uploaded_code = fw_utils.tar_and_upload_dir(session=self.sagemaker_session.boto_session, | ||
bucket=bucket, | ||
s3_key_prefix=key_prefix, | ||
script=self.entry_point, | ||
directory=self.source_dir, | ||
dependencies=self.dependencies) | ||
if repack: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. seems like this part of the conditional should also include There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. out of scope There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (talked offline.) fair enough. let's track it internally as a possible enhancement though. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
self.repacked_model_data = utils.repack_model(inference_script=self.entry_point, | ||
source_directory=self.source_dir, | ||
model_uri=self.model_data, | ||
sagemaker_session=self.sagemaker_session) | ||
|
||
self.uploaded_code = UploadedCode(s3_prefix=self.repacked_model_data, | ||
script_name=os.path.basename(self.entry_point)) | ||
|
||
else: | ||
bucket = self.bucket or self.sagemaker_session.default_bucket() | ||
self.uploaded_code = fw_utils.tar_and_upload_dir(session=self.sagemaker_session.boto_session, | ||
bucket=bucket, | ||
s3_key_prefix=key_prefix, | ||
script=self.entry_point, | ||
directory=self.source_dir, | ||
dependencies=self.dependencies) | ||
|
||
def _framework_env_vars(self): | ||
if self.uploaded_code: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,9 +4,9 @@ Using MXNet with the SageMaker Python SDK | |
|
||
With the SageMaker Python SDK, you can train and host MXNet models on Amazon SageMaker. | ||
|
||
Supported versions of MXNet: ``1.3.0``, ``1.2.1``, ``1.1.0``, ``1.0.0``, ``0.12.1``. | ||
Supported versions of MXNet: ``1.4.0``, ``1.3.0``, ``1.2.1``, ``1.1.0``, ``1.0.0``, ``0.12.1``. | ||
|
||
Supported versions of MXNet for Elastic Inference: ``1.3.0``. | ||
Supported versions of MXNet for Elastic Inference: ``1.3.0``, ``1.4.0``. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd keep the order here consistent with the others (i.e. 1.4 first) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. oh wow nice catch |
||
|
||
For information about using MXNet with the SageMaker Python SDK, see https://sagemaker.readthedocs.io/en/stable/using_mxnet.html. | ||
|
||
|
@@ -15,29 +15,30 @@ SageMaker MXNet Containers | |
|
||
When training and deploying training scripts, SageMaker runs your Python script in a Docker container with several libraries installed. When creating the Estimator and calling deploy to create the SageMaker Endpoint, you can control the environment your script runs in. | ||
|
||
SageMaker runs MXNet Estimator scripts in either Python 2.7 or Python 3.5. You can select the Python version by passing a ``py_version`` keyword arg to the MXNet Estimator constructor. Setting this to ``py2`` (the default) will cause your training script to be run on Python 2.7. Setting this to ``py3`` will cause your training script to be run on Python 3.5. This Python version applies to both the Training Job, created by fit, and the Endpoint, created by deploy. | ||
SageMaker runs MXNet Estimator scripts in either Python 2.7 or Python 3.6. You can select the Python version by passing a ``py_version`` keyword arg to the MXNet Estimator constructor. Setting this to ``py2`` (the default) will cause your training script to be run on Python 2.7. Setting this to ``py3`` will cause your training script to be run on Python 3.5. This Python version applies to both the Training Job, created by fit, and the Endpoint, created by deploy. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. chaned |
||
|
||
Your MXNet training script will be run on version 1.2.1 by default. (See below for how to choose a different version, and currently supported versions.) The decision to use the GPU or CPU version of MXNet is made by the ``train_instance_type``, set on the MXNet constructor. If you choose a GPU instance type, your training job will be run on a GPU version of MXNet. If you choose a CPU instance type, your training job will be run on a CPU version of MXNet. Similarly, when you call deploy, specifying a GPU or CPU deploy_instance_type, will control which MXNet build your Endpoint runs. | ||
|
||
The Docker images have the following dependencies installed: | ||
|
||
+-------------------------+--------------+-------------+-------------+-------------+-------------+ | ||
| Dependencies | MXNet 0.12.1 | MXNet 1.0.0 | MXNet 1.1.0 | MXNet 1.2.1 | MXNet 1.3.0 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+ | ||
| Python | 2.7 or 3.5 | 2.7 or 3.5| 2.7 or 3.5| 2.7 or 3.5| 2.7 or 3.5| | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+ | ||
| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | 9.0 | 9.0 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+ | ||
| numpy | 1.13.3 | 1.13.3 | 1.13.3 | 1.14.5 | 1.14.6 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+ | ||
| onnx | N/A | N/A | N/A | 1.2.1 | 1.2.1 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+ | ||
| keras-mxnet | N/A | N/A | N/A | N/A | 2.2.2 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+ | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+ | ||
| Dependencies | MXNet 0.12.1 | MXNet 1.0.0 | MXNet 1.1.0 | MXNet 1.2.1 | MXNet 1.3.0 | MXNet 1.4.0 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+ | ||
| Python | 2.7 or 3.5 | 2.7 or 3.5| 2.7 or 3.5| 2.7 or 3.5| 2.7 or 3.5| 2.7 or 3.6| | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+ | ||
| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | 9.0 | 9.0 | 9.2 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+ | ||
| numpy | 1.13.3 | 1.13.3 | 1.13.3 | 1.14.5 | 1.14.6 | 1.16.3 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+ | ||
| onnx | N/A | N/A | N/A | 1.2.1 | 1.2.1 | 1.4.1 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+ | ||
| keras-mxnet | N/A | N/A | N/A | N/A | 2.2.2 | 2.2.4.1 | | ||
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+ | ||
|
||
The Docker images extend Ubuntu 16.04. | ||
|
||
You can select version of MXNet by passing a ``framework_version`` keyword arg to the MXNet Estimator constructor. Currently supported versions are listed in the above table. You can also set ``framework_version`` to only specify major and minor version, e.g ``1.2``, which will cause your training script to be run on the latest supported patch version of that minor version, which in this example would be 1.2.1. | ||
Alternatively, you can build your own image by following the instructions in the SageMaker MXNet containers repository, and passing ``image_name`` to the MXNet Estimator constructor. | ||
|
||
You can visit the SageMaker MXNet containers repository here: https://github.com/aws/sagemaker-mxnet-container | ||
You can visit the SageMaker MXNet training containers repository here: https://github.com/aws/sagemaker-mxnet-container | ||
You can visit the SageMaker MXNet serving containers repository here: https://github.com/aws/sagemaker-mxnet-serving-container |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is the default?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added