Add Kubeflow Pipeline components support in polyaxon-hub

[mmourafiq]

Use case

Hello，anyone knows how to integrate Polyaxon with Sagemaker? or integrate with Azure ML? I know Polyaxon hasn't released the feature yet, so if we dive into the opensource code, develop by ourself, any suggestion?

Kubeflow components are a simple use-case of Polyaxon components, and should be used with minimal changes.

Feature description

Fork Kubeflow Pipeline components and add support for several useful components directly in Polyaxon.

Alternatives

• We can add components that users request instead of adding all components.
• Some clients can be added directly to the connections module, e.g. sagemaker can be added to the aws module for better integration.

[github-actions]

This issue has not seen any recent activity.

[Ark-kun]

It would be great to just add generic support of Kubeflow Pipelines components.

The Kubeflow Pipelines SDK is factored in a way that the components module is completely independent from the rest of the SDK.

Are any Polyaxon users or developers interested in this?

[mmourafiq]

Hi @Ark-kun thank you for the comment. We’ve been following the progress of the IR and the new pipelines specification very closely, and we would like to support the components as well as the rest of the Kubeflow pipelines ecosystem in Polyaxon.

We are waiting for the IR to be finished before starting the work on this integration.

Are any Polyaxon users or developers interested in this?

Is there some reference you can share to see how this module can be used outside of KFP/Kubeflow cluster?