Getting Pipeline Object from JSON

[andy-han]

Is there a way to create an object of type Pipeline when given a valid dictionary without having to reconstruct all pipeline parameters? For example:

Instead of:

pipeline = Pipeline(
    name="AbalonePipeline",
    parameters=[
        processing_instance_type, 
        processing_instance_count,
        training_instance_type,
        model_approval_status,
        input_data,
        batch_data,
    ],
    steps=[step_process, step_train, step_eval, step_cond],
)

Could we have something like:

pipeline = Pipeline(json.loads(<some valid json pipeline defintion>))

Also with this change, something like this would work:

pipeline_1 = Pipeline(
    name="AbalonePipeline1",
    parameters=[
        processing_instance_type, 
        processing_instance_count,
        training_instance_type,
        model_approval_status,
        input_data,
        batch_data,
    ],
    steps=[step_process, step_train, step_eval, step_cond],
)
pipeline_1_definition = json.loads(pipeline_1.definition())
pipeline_2 = Pipeline(
    name="AbalonePipeline2",
    parameters=pipeline_1_definition["Parameters"],
    steps=pipeline_1_definition["Steps"],
)

[metrizable]

Hello @andy-han

Thank you for using Amazon SageMaker. Currently, this is unsupported. However, we are always re-evaluating our backlog of features based on customer requests, so we appreciate the feedback on this feature.

[andy-han]

For anybody that stumbles upon this, I worked around this problem by using the boto3 client

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_pipeline

Hope to see the sagemaker client include this in the future!

[mouhannadali]

Hi @andy-han could you explain how did you solve this problem using boto3?

[andy-han]

@mouhannadali the .create_pipeline() function accepts a PipelineDefinition parameter which would be the json output as a string of your Pipeline

[ShwetaSingh801]

Hi @andy-han,

Thanks for using SageMaker and taking the time to suggest ways to improve SageMaker Python SDK. We have added your feature request to our backlog and may consider putting it into future SDK versions. I will go ahead and close the issue now. Please let me know if you have any more feedback. Let me know if you have any other questions.

Best,
Shweta