S3Operations is not supported by Step Functions error: Using TrainingPipeline with a TensorFlow model #15
Description
Error: The field \"S3Operations\" is not supported by Step Functions
{ "error": "States.Runtime", "cause": "An error occurred while executing the state 'Create Model' (entered at the event id #8). The Parameters '{\"ExecutionRoleArn\":\"arn:aws:iam::835319576252:role/service-role/AmazonSageMaker-ExecutionRole-20191006T135881\",\"PrimaryContainer\":{\"Image\":\"520713654638.dkr.ecr.us-east-1.amazonaws.com/sagemaker-tensorflow:2.0.0-gpu-py3\",\"Environment\":{\"SAGEMAKER_PROGRAM\":\"mnist_keras_tf2.py\",\"SAGEMAKER_SUBMIT_DIRECTORY\":\"s3://sagemaker-us-east-1-835319576252/training-pipeline-2019-12-27-19-39-29/estimator-source/source/sourcedir.tar.gz\",\"SAGEMAKER_ENABLE_CLOUDWATCH_METRICS\":\"false\",\"SAGEMAKER_CONTAINER_LOG_LEVEL\":\"20\",\"SAGEMAKER_REGION\":\"us-east-1\"},\"ModelDataUrl\":\"s3://sagemaker-us-east-1-835319576252/sagemaker/tensorflow-mnist/training-runs/training-pipeline-2019-12-27-19-39-29/models/estimator-training-pipeline-2019-12-27-19-39-35/output/model.tar.gz\"},\"ModelName\":\"training-pipeline-2019-12-27-19-39-35\",\"S3Operations\":{\"S3Upload\":[{\"Path\":\"mnist_keras_tf2.py\",\"Bucket\":\"sagemaker-us-east-1-835319576252\",\"Key\":\"training-pipeline-2019-12-27-19-39-29/estimator-source/source/sourcedir.tar.gz\",\"Tar\":true}]}}' could not be used to start the Task: [The field \"S3Operations\" is not supported by Step Functions]" }
--
The code is roughly like this...
from sagemaker.tensorflow import TensorFlow
mnist_estimator = TensorFlow(entry_point='mnist_keras_tf2.py',
source_dir='./src',
role=sagemaker_execution_role,
framework_version='2.0.0',
train_instance_count=1,
train_instance_type='ml.p3.2xlarge',
py_version='py3',
distributions={'parameter_server': {'enabled': True}})
model_output_s3_bucket = '{}/sagemaker/tensorflow-mnist/training-runs'.format(bucket)
pipeline = TrainingPipeline(
estimator=mnist_estimator,
role=workflow_execution_role,
inputs=training_data_uri,
s3_bucket=model_output_s3_bucket
)
pipeline.create()
pipeline.execute()
This happens during the Create Model step...
Any ideas?