fix(sdk): Get short name of complex input/output types to ensure we can map to appropriate de|serializer

[alexlatchford]

Description of your changes:

The problem is that to turn an input value like "[]" into a python List type you need to use something like json.loads. To tell argparse.ArgumentParser to use json.loads kfp must be able to map the input type to this deserializer function, it does this in the get_deserializer_code_for_type_struct method (see code here). Unfortunately the type_name_to_deserializer cannot possibly contain all the List[<type>] combinations so if you have a type hint that uses List[str] instead of just List you need to run the type_annotation_utils.get_short_type_name to strip List[<type>] to just List so we can map to the appropriate serializer method. The same logic applies for Dict[<type>] to Dict too.

This is behavior that changed in Python v3.7+, KFP have been fixing various issues around this since then, see these PRs:

• feat(sdk): special handling for type annotation using typing.Optional feat(sdk): special handling for type annotation using typing.Optional #5716
• chore(sdk): allow serializing pipeline params with typing type hints chore(sdk): allow serializing pipeline params with typing type hints #5153

[chensun]

@alexlatchford Thank you for the contribution.

Can you please add an entry under https://github.com/kubeflow/pipelines/blob/master/sdk/RELEASE.md#current-version-still-in-development

[alexlatchford]

Hey @chensun thanks for the review, I think I've answered all feedback. Let me know if you need anything more from my end to get this merged 😄

[alexlatchford]

/retest

[alexlatchford]

Just fixed the conflicts on this one @chensun so should be ready once the tests pass!

[chensun]

/lgtm
/approve

Thanks!

[google-oss-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chensun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• sdk/OWNERS [chensun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment