Experimental support Kubeflow Pipelines v2 as runtime environment

[ptitzler]

Is your feature request related to a problem? Please describe.

This serves as an umbrella issue for the work items, which enable users to leverage KFP v2.x as runtime environment. The initial v2.0 release does not support all v1 features that Elyra leverages. Significant limitations may have therefore exist, rendering it impractical as a drop-in replacement for v1.

Describe the solution you'd like

• Add new pipeline runtime type (the existing v1-based runtime type will remain supported)

Work items:

• Metadata editor: allow for creation of runtime environment configurations for KFP 1.x and 2.x
• Pipeline editor: support KFP v2 parameters and artifacts
• Create new 'custom operator' for generic operations (which doesn't have any of the existing limitations in air-gapped environments)
• Create new KFP v2 runtime, which supports running and exporting of pipelines (this includes validation and custom component processing)
• Update documentation and tutorials
• Update/validate CLIs

Additional context
Resources:

• Evaluate use of Kubeflow Pipelines SDK v2 #1930
• Kubeflow Pipelines v2 SDK documentation