Pipeline paramaters validation for Local mode

[patrick-239]

Describe the bug
In local mode it is possible to pass empty SM pipeline parameter (ParameterString type) when calling the StartPipelineExecution operation, but the same code will rise an exception when launched on cloud: "botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the StartPipelineExecution operation: [string "" is too short (length: 0, required minimum: 1)]". It is hard to develop a code locally when same methods have a different behavior in a cloud.

To reproduce
Launch SM pipeline with an empty parameter in a cloud. Repeat same with local mode.

Expected behavior
Both mode (local and cloud) raise an error or allow to use empty parameters.

Screenshots or logs
If applicable, add screenshots or logs to help explain your problem.

System information
A description of your system. Please provide:

• SageMaker Python SDK version:
• Framework name (eg. PyTorch) or algorithm (eg. KMeans):
• Framework version:
• Python version:
• CPU or GPU:
• Custom Docker image (Y/N):

Additional context
Add any other context about the problem here.

[qidewenwhen]

Hi @patrick-239, thanks for reaching out!

Could you share us the complete error log trace for further investigation.
Also, can you share us some code snippet on what your pipeline looks like if possible? Thanks!

Asking because the error was thrown from StartPipelineExecution. We have explicit validation logics behind CreatePipeline rather than StartPipelineExecution so we need to understand where the ValidationException is coming from.

[qidewenwhen]

We have the following validation for json get input in local mode to replicate the behavior in API but seems we miss one for the ParameterString

sagemaker-python-sdk/src/sagemaker/workflow/steps.py

Line 220 in d355d5b

def _validate_json_get_function(

Next steps:
1. Check if the default string is empty and throw an error accordingly
2. Move such validation to an early stage when creating the ParameterString object, so that users can get notified as early as possible.
Tracked this quick fix in backlog with other issues

Update:
Reproduced the issue.
The error botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the StartPipelineExecution operation: [string "" is too short (length: 0, required minimum: 1)] is thrown by StartPipelineExecution API.

However, when defining the pipeline, the ParameterString's default value allows an empty string and it won't raise error if the parameter string value is not overridden when starting a pipeline execution.

Thus, to make local mode behavior consistent with the cloud, we should add the ParameterString's value validation in execution time (i.e. when starting an execution).