Allow to specify `name` argument in `sagmaker.model.Model` as a pipeline variable

[lorenzwalthert]

Describe the feature you'd like

Create an unversioned sagemaker model in a pipeline that has a name parametrised at runtime. I.e. I have in my pipeline definition code:

model = sagemaker.model.Model(
  # ...
  name=sagemaker.workflow.execution_variables.ExecutionVariables.PIPELINE_EXECUTION_ID,
)

sagemaker.workflow.model_step.ModelStep(
  name="create_model",
  step_args=model.create(),
    )

This code intends to create an unversioned model where the name of the model is derived from the current pipeline execution.
However, since name has to be a string, the pipeline creation fails with

TypeError: can only concatenate str (not "ExecutionVariable") to str

```python Traceback (most recent call last): File "/my/proj/pipelines/run_pipeline.py", line 100, in main pipeline = get_pipeline_driver( ^^^^^^^^^^^^^^^^^^^^ File "/my/proj/pipelines/_utils.py", line 34, in get_pipeline_driver return _imports.get_pipeline(role_arn=role_arn, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/proj/pipelines/inference/pipeline.py", line 84, in get_pipeline step_create_model = proj.pipelines.inference.steps.create_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/proj/pipelines/inference/steps/step_model.py", line 20, in create_model step_args=model.create(instance_type="ml.m5.large"), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/python/lib/site-packages/sagemaker/workflow/pipeline_context.py", line 304, in wrapper run_func(*args, **kwargs) File "/my/python/lib/site-packages/sagemaker/model.py", line 552, in create self._create_sagemaker_model( File "/my/python/lib/site-packages/sagemaker/model.py", line 771, in _create_sagemaker_model container_def = self.prepare_container_def( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/python/lib/site-packages/sagemaker/model.py", line 598, in prepare_container_def deploy_key_prefix = fw_utils.model_code_key_prefix( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/python/lib/site-packages/sagemaker/fw_utils.py", line 603, in model_code_key_prefix return s3_path_join(code_location_key_prefix, model_name or name_from_image) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/python/lib/site-packages/sagemaker/s3_utils.py", line 90, in s3_path_join merged_path += path TypeError: can only concatenate str (not "ExecutionVariable") to str Traceback (most recent call last): File "/my/proj/pipelines/run_pipeline.py", line 100, in main pipeline = get_pipeline_driver( ^^^^^^^^^^^^^^^^^^^^ File "/my/proj/pipelines/_utils.py", line 34, in get_pipeline_driver return _imports.get_pipeline(role_arn=role_arn, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/proj/pipelines/inference/pipeline.py", line 84, in get_pipeline step_create_model = proj.pipelines.inference.steps.create_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/proj/pipelines/inference/steps/step_model.py", line 20, in create_model step_args=model.create(instance_type="ml.m5.large"), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/python/lib/site-packages/sagemaker/workflow/pipeline_context.py", line 304, in wrapper run_func(*args, **kwargs) File "/my/python/lib/site-packages/sagemaker/model.py", line 552, in create self._create_sagemaker_model( File "/my/python/lib/site-packages/sagemaker/model.py", line 771, in _create_sagemaker_model container_def = self.prepare_container_def( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/python/lib/site-packages/sagemaker/model.py", line 598, in prepare_container_def deploy_key_prefix = fw_utils.model_code_key_prefix( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/python/lib/site-packages/sagemaker/fw_utils.py", line 603, in model_code_key_prefix return s3_path_join(code_location_key_prefix, model_name or name_from_image) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/my/python/lib/site-packages/sagemaker/s3_utils.py", line 90, in s3_path_join merged_path += path TypeError: can only concatenate str (not "ExecutionVariable") to str ```

How would this feature be used? Please describe.
Currently, pipelines create models with non-deterministic names. This makes it impossible to properly reference later. Given the pipeline execution id, we can list all models with boto and filter by name, but that's not guaranteed to be the match we want and cumbersome. Tags are not returned by that API, so we can't use tags to filter and need another call to describe model to get tags, which are typed Dict[str, Union[str, PipelineVariable]], so it should be possible to pass a PipelineVariable.

Describe alternatives you've considered

Using list all models and filter by NameContains the pipeline execution id.

[ShwetaSingh801]

Hi @lorenzwalthert,

Thanks for using SageMaker and taking the time to suggest ways to improve SageMaker Python SDK. We have added your feature request it to our backlog of feature requests and may consider putting it into future SDK versions. I will go ahead and close the issue now, please let me know if you have any more feedback. Let me know if you have any other questions.

Best,
Shweta

[Madabaru]

What is the status here?

[lorenzwalthert]

A workaround is to create a model in a processing, training or lambda step (the latter not supported by local mode). If you do that, you run into another problem: Pipeline Executions with Local Mode only supports models that were created as part of the local execution with the create model step. This means it's impossible to register a model with a method other than the create model step and use it in local mode.