Passing PipelineVariable as hyperparameters for Framework Estimator fails

[AndreiVoinovTR]

Describe the bug
Generating SageMaker Pipeline definition fails if hyperparameters defined as PipelineVariable are passed to a training step.

Expected behavior
Should PipelineVariable(s) be supported for hyperparametes?
In Estimator's constructor hyperparameters are defined as: Optional[Dict[str, Union[str, sagemaker.workflow.entities.PipelineVariable]]] = None.

Screenshots or logs

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-86301c565ea6> in <cell line: 1>()
----> 1 pipeline.build_and_deploy(experiment_config=experiment_config)

~/projects/src/ml-scale/labs-MLTools_CLI/src/trlabs_mltools_cli/pipeline.py in build_and_deploy(self, experiment_config, parameters_override)
     98             service responce : see https://sagemaker.readthedocs.io/en/stable/workflows/pipelines/sagemaker.workflow.pipelines.html#sagemaker.workflow.pipeline.Pipeline.upsert
     99         """  # noqa
--> 100         pipeline = self.build(experiment_config=experiment_config, parameters_override=parameters_override)
    101         logger.info(f"Deploying SageMaker Pipeline: {pipeline} ...")
    102         return pipeline.upsert(self.experiment.workspace.iam_role)

~/projects/src/ml-scale/labs-MLTools_CLI/src/trlabs_mltools_cli/pipeline.py in build(self, experiment_config, parameters_override)
     85             self.get_template(parameters_override), DeploymentContext.from_workspace(self.experiment.workspace)
     86         ).build_pipeline(self._base_ppl_name(), experiment_config=ppl_experiment_config)
---> 87         logger.debug(f"Built SageMaker Pipeline: {pipeline.definition()}")
     88         return pipeline
     89 

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/workflow/pipeline.py in definition(self)
    319     def definition(self) -> str:
    320         """Converts a request structure to string representation for workflow service calls."""
--> 321         request_dict = self.to_request()
    322         self._interpolate_step_collection_name_in_depends_on(request_dict["Steps"])
    323         request_dict["PipelineExperimentConfig"] = interpolate(

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/workflow/pipeline.py in to_request(self)
    103             if self.pipeline_experiment_config is not None
    104             else None,
--> 105             "Steps": list_to_request(self.steps),
    106         }
    107 

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/workflow/utilities.py in list_to_request(entities)
     51     for entity in entities:
     52         if isinstance(entity, Entity):
---> 53             request_dicts.append(entity.to_request())
     54         elif isinstance(entity, StepCollection):
     55             request_dicts.extend(entity.request_dicts())

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/workflow/steps.py in to_request(self)
    497     def to_request(self) -> RequestType:
    498         """Updates the request dictionary with cache configuration."""
--> 499         request_dict = super().to_request()
    500         if self.cache_config:
    501             request_dict.update(self.cache_config.config)

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/workflow/steps.py in to_request(self)
    349     def to_request(self) -> RequestType:
    350         """Gets the request structure for `ConfigurableRetryStep`."""
--> 351         step_dict = super().to_request()
    352         if self.retry_policies:
    353             step_dict["RetryPolicies"] = self._resolve_retry_policy(self.retry_policies)

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/workflow/steps.py in to_request(self)
    118             "Name": self.name,
    119             "Type": self.step_type.value,
--> 120             "Arguments": self.arguments,
    121         }
    122         if self.depends_on:

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/workflow/steps.py in arguments(self)
    476             request_dict = self.step_args
    477         else:
--> 478             self.estimator._prepare_for_training(self.job_name)
    479             train_args = _TrainingJob._get_train_args(
    480                 self.estimator, self.inputs, experiment_config=dict()

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/trlabs_mltools/sagemaker/factory.py in _prepare_for_training(self, job_name)
     91             )
     92 
---> 93         super()._prepare_for_training(job_name=job_name)
     94 
     95 

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/estimator.py in _prepare_for_training(self, job_name)
   2880                 constructor if applicable.
   2881         """
-> 2882         super(Framework, self)._prepare_for_training(job_name=job_name)
   2883 
   2884         self._validate_and_set_debugger_configs()

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/estimator.py in _prepare_for_training(self, job_name)
    705             # Modify hyperparameters in-place to point to the right code directory and
    706             # script URIs
--> 707             self._script_mode_hyperparam_update(code_dir, script)
    708 
    709         self._prepare_rules()

~/projects/src/ml-scale/labs-MLTools_CLI/.venv/lib/python3.8/site-packages/sagemaker/estimator.py in _script_mode_hyperparam_update(self, code_dir, script)
   2898         hyperparams[SAGEMAKER_REGION_PARAM_NAME] = self.sagemaker_session.boto_region_name
   2899 
-> 2900         self._hyperparameters.update(hyperparams)
   2901 
   2902     def _validate_and_set_debugger_configs(self):

AttributeError: 'ParameterString' object has no attribute 'update'

System information
A description of your system. Please provide:

• SageMaker Python SDK version: 2.107.0:
• PyTorch Estimator:

[qidewenwhen]

Hi @AndreiVoinovTR , thanks for using SageMaker Pipeline!
Can you also provide us the code snippet on how you defined hyperparameters as PipelineVariable so that we can reproduce the issue?

[qidewenwhen]

Expected behavior
Should PipelineVariable(s) be supported for hyperparametes?
In Estimator's constructor hyperparameters are defined as: Optional[Dict[str, Union[str, sagemaker.workflow.entities.PipelineVariable]]] = None.

Currently PipelineVariable only supports to replace Python primitive types, e.g. str, int, fload, bool. Given the expected behavior above, seems this issue is a feature request asking to make PipelineVariable support dict and list types.
Relabeling this issue to feature request and we have a backlog item already to keep track on this.

[ShwetaSingh801]

Hi @AndreiVoinovTR ,

Thanks for using SageMaker and taking the time to suggest ways to improve SageMaker Python SDK. We have added your feature request it to our backlog of feature requests and may consider putting it into future SDK versions. I will go ahead and close the issue now, please let me know if you have any more feedback. Let me know if you have any other questions.

Best,
Shweta