"error: argument *: invalid int value" for arguments that come from ParameterInteger or ParameterFloat

[jonsnowseven]

Describe the bug
Additionally opened an error on the aws-samples:
aws-samples/sagemaker-cross-validation-pipeline#1

To reproduce
Execute Example in AWS Samples - SageMaker Cross Validation

Expected behaviour
Argument is correctly parsed to an integer instead of "ParameterInteger(name='KFold', parameter_type=<ParameterTypeEnum.INTEGER: 'Integer'>, default_value=3)"

System information
A description of your system. Please provide:

• SageMaker Python SDK version: 2.77.1
• Framework name (eg. PyTorch) or algorithm (eg. KMeans): Custom
• Framework version: NA
• Python version: 3.8.12
• CPU or GPU: CPU
• Custom Docker image (Y/N): Y

[jonsnowseven]

I confirmed that with SageMaker Python SDK version 2.78.0 still happens.

[qidewenwhen]

Sorry for the inconvenience caused, we're currently working on improving the compatibility of Pipeline variables, which include the ParameterInteger, ParameterFloat, etc.

[qidewenwhen]

Hi @jonsnowseven, according to the your description, the issue is that k is an ParameterInteger and the str(k) in job_arguments=["-k", str(k), throws an error.

The issue was caused by the incompatibility between ParameterInteger and Python primitive int, as ParameterInteger does not support the cast to str operation. We've merged this PR to fix this issue, which is not yet available in public at this point.

Once this PR takes effect in public, you can do the following to walk around. It uses .to_string() to cast the ParameterInteger or ParameterFloat into a Join object, which will be further parsed and translated to a string type in runtime (Pipeline backend)

    job_arguments=["-k", k.to_string(),
                   "--image-uri", image_uri, 
                   "--train", s3_bucket_base_path_train, 
                   "--test", s3_bucket_base_path_test,
                   "--instance-type", training_instance_type,
                   "--instance-count", training_instance_count.to_string(),
                   "--output-path", s3_bucket_base_path_output,
                   "--max-jobs", max_jobs.to_string(),
                   "--max-parallel-jobs" , max_parallel_jobs.to_string(),
                   "--min-c", min_c.to_string(),
                   "--max-c", max_c.to_string(),
                   "--min-gamma", min_gamma.to_string(), 
                   "--max-gamma", max_gamma.to_string(),
                   "--gamma-scaling-type", gamma_scaling_type.to_string(),
                   "--region", str(region)],

I'll update this sagemaker-cross-validation-pipeline sample notebook accordingly later. Sorry again for the inconvenience.

[jonsnowseven]

Thanks for the explanation @qidewenwhen!

On which version will that change be included?

[qidewenwhen]

Hi @jonsnowseven, the latest Sagemaker v 2.83.0 (by written time) should include this change.
Furthermore, to unblock yourself and get the notebook sagemaker-cross-validation-pipeline passed, please follow these steps:

1. Upgrade the sagemaker. If the upgrade does not help (e.g. there're errors relating to the newly added .to_string() method), please try restarting the Kernel.
2. Update the following cell in the notebook:

# Variables / Constants used throughout the pipeline
model_package_group_name="IrisClassificationCrossValidatedModel"
framework_version = "0.23-1"
s3_bucket_base_path=f"s3://{sagemaker_session.default_bucket()}/cross_validation_iris_classification" # Update this assignment and replace the ParameterString with actual string as the ParameterString is only parsed during runtime (Pipeline backend)
s3_bucket_base_path_train = f"{s3_bucket_base_path}/train"
s3_bucket_base_path_test = f"{s3_bucket_base_path}/test"
s3_bucket_base_path_evaluation = f"{s3_bucket_base_path}/evaluation"
s3_bucket_base_path_jobinfo = f"{s3_bucket_base_path}/jobinfo"
s3_bucket_base_path_output = f"{s3_bucket_base_path}/output"

3. Update the job_arguments assignment to be:

    job_arguments=["-k", k.to_string(),
                   "--image-uri", image_uri, 
                   "--train", s3_bucket_base_path_train, 
                   "--test", s3_bucket_base_path_test,
                   "--instance-type", training_instance_type,
                   "--instance-count", training_instance_count.to_string(),
                   "--output-path", s3_bucket_base_path_output,
                   "--max-jobs", max_jobs.to_string(),
                   "--max-parallel-jobs" , max_parallel_jobs.to_string(),
                   "--min-c", min_c.to_string(),
                   "--max-c", max_c.to_string(),
                   "--min-gamma", min_gamma.to_string(), 
                   "--max-gamma", max_gamma.to_string(),
                   "--gamma-scaling-type", gamma_scaling_type.to_string(),
                   "--region", str(region)],

I'll open PR to update the notebook but it will take some time. Sorry for the inconvenience. Feel free to comment if you have other questions.

[qidewenwhen]

The notebook update will be tracked in issue - aws-samples/sagemaker-cross-validation-pipeline#1
I'll close this issue as it's duplicate.

@jonsnowseven feel free to reopen this issue and leave comments if you have any questions.