v1 visualizations backward compatible in v2 compatible

[Bobgy]

support v1 visualizations:
https://www.kubeflow.org/docs/components/pipelines/sdk/output-viewer/

Changes needed:

• Doc: update https://kubeflow.org/docs/components/pipelines/sdk/output-viewer/ according to https://www.kubeflow.org/docs/components/pipelines/sdk/pipelines-metrics/
• fix(sdk/compiler): v2 compat - fix mlpipeline-ui-metadata artifact. Part of #5666 #5832 SDK: remove hacks for mlpipeline-ui-metadata artifact in v2 compatible mode -- result: we should be able to see mlpipeline-ui-metadata as a usual artifact in MLMD.
• UI: also supports visualizing mlpipeline-ui-metadata artifact from MLMD

[Bobgy]

/assign @chensun
for SDK
/assign @zijianjoy
for UI

[Ark-kun]

I'd really like to see mlpipeline-ui-metadata gone in the future.
But first there needs to be an alternative to it.
Supporting "type-based visualizations" and "visualizations as components" will provide a better alternative.

[Bobgy]

@Ark-kun the goal of this issue is to ease migration, when switching to v2-compatible pipelines, people can gradually move to type-based visualization with their existing pipelines.

[Bobgy]

Moving to P1, because Chen does not have time to work on this before 6.4.

We may work on the doc update first.

[zijianjoy]

I have a few questions regarding writing sample pipelines and visualizing on UI:

Sample Pipeline

Is there a sample pipeline which writes v1 mlpipeline-ui-metadata artifact in V2 compatible mode using Python? It will be great to have a pipeline to create both V1 and V2 visualization, so I can test the scenario where both V1 and V2 artifacts exist in single execution. Furthermore:

mlpipeline-ui-metadata Artifact naming in python code

I tried to write the following component, but was not successful because python doesn't accept - in parameter name mlpipeline-ui-metadata:

@component
def visall(mlpipeline-ui-metadata: Output[Artifact]):
  import json
    
  metadata = {
    'outputs' : [
    # Markdown that is hardcoded inline
    {
      'storage': 'inline',
      'source': '# Inline Markdown\n[A link](https://www.kubeflow.org/)',
      'type': 'markdown',
    }]
  }
  with open(mlpipeline-ui-metadata.path, 'w') as metadata_file:
    json.dump(metadata, metadata_file)

I slightly changed the name to something else so I can preview the content in output artifact. Here is the content:

{"outputs": [{"storage": "inline", "source": "# Inline Markdown\n[A link](https://www.kubeflow.org/)", "type": "markdown"}]}

Source data sample for each visualization

There are 6 different V1 outputs in https://www.kubeflow.org/docs/components/pipelines/sdk/output-viewer/#available-output-viewers, but I am looking for some sample data in a csv file or other format to be able to store in the visualization content. See below:

• Confusion matrix
  • CONFUSION_MATRIX_CSV_FILE and vocab from the doc example.
• Markdown
  • A google cloud storage path for Markdown file (what kind of permission we need to provide?)
• ROC curve
  • roc_file file
• Table
  • prediction_results file
• TensorBoard
  • args.job_dir path
• Web app
  • A google cloud storage path for HTML file.

Read file permission and test cases

If we use a gcs path or minio path for storing the visualization data, what kind of permission do we need to give to KFP for accessing those files? Can they be added to the KFP repo?

[Bobgy]

I have a few questions regarding writing sample pipelines and visualizing on UI:

Sample Pipeline

Is there a sample pipeline which writes v1 mlpipeline-ui-metadata artifact in V2 compatible mode using Python? It will be great to have a pipeline to create both V1 and V2 visualization, so I can test the scenario where both V1 and V2 artifacts exist in single execution. Furthermore:

It's blocked by #5831. Ideally, all v1 samples should be able to run in v2 compatible mode.

mlpipeline-ui-metadata Artifact naming in python code

I tried to write the following component, but was not successful because python doesn't accept - in parameter name mlpipeline-ui-metadata:

@component
def visall(mlpipeline-ui-metadata: Output[Artifact]):
  import json
    
  metadata = {
    'outputs' : [
    # Markdown that is hardcoded inline
    {
      'storage': 'inline',
      'source': '# Inline Markdown\n[A link](https://www.kubeflow.org/)',
      'type': 'markdown',
    }]
  }
  with open(mlpipeline-ui-metadata.path, 'w') as metadata_file:
    json.dump(metadata, metadata_file)

I slightly changed the name to something else so I can preview the content in output artifact. Here is the content:

{"outputs": [{"storage": "inline", "source": "# Inline Markdown\n[A link](https://www.kubeflow.org/)", "type": "markdown"}]}

Traditionally, KFP v1 sanitizes artifact names by turning all letters to lower case and all non letter chars to -. Therefore, you don't need to match the exact name in python.

I think it's worth considering letting UI do the name sanitization for this specific artifact name if sdk v2 doesn't.

Source data sample for each visualization

There are 6 different V1 outputs in https://www.kubeflow.org/docs/components/pipelines/sdk/output-viewer/#available-output-viewers, but I am looking for some sample data in a csv file or other format to be able to store in the visualization content. See below:

• Confusion matrix
  • CONFUSION_MATRIX_CSV_FILE and vocab from the doc example.
• Markdown
  • A google cloud storage path for Markdown file (what kind of permission we need to provide?)
• ROC curve
  • roc_file file
• Table
  • prediction_results file
• TensorBoard
  • args.job_dir path
• Web app
  • A google cloud storage path for HTML file.

Not quite following, do you want samples for these cases?

Read file permission and test cases

If we use a gcs path or minio path for storing the visualization data, what kind of permission do we need to give to KFP for accessing those files? Can they be added to the KFP repo?

KFP UI server fetches these artifacts, by default we already mount MinIO secrets on it. On GCP it gets cluster default service account permission. So you don't need to worry about permission, users are responsible for setting them in their cluster.

[zijianjoy]

Thank you @Bobgy for the answers!

It's blocked by #5831. Ideally, all v1 samples should be able to run in v2 compatible mode.

So we cannot create python component with - parameter name, and cannot create yaml component with space yet. Can we use lightweight python component approach to create v1 mlpipeline_ui_metadata in v2 compatible mode? https://github.com/kubeflow/pipelines/blob/master/samples/core/lightweight_component/lightweight_component.ipynb

Or is there any other approach to create a pipeline while waiting for the blocking issue to resolve?

Traditionally, KFP v1 sanitizes artifact names by turning all letters to lower case and all non letter chars to -. Therefore, you don't need to match the exact name in python.

I think it's worth considering letting UI do the name sanitization for this specific artifact name if sdk v2 doesn't.

I would like to learn more about sanitization from UI approach: Does this logic already handle the sanitization logic on SDK side? https://github.com/kubeflow/pipelines/pull/5832/files#diff-6e6c99d682c8b3b8d586a8c50d381dee529c0c58622316532c6574c4f511b655

As a result, how does user define a component in v1 fashion but is runnable on v2 compatible mode? More specifically, how does user define a mlpipeline-ui-metadata parameter in python? Currently I tried this component definition def visall(mlpipeline_ui_metadata: Output[Artifact]) but SDK fails with TypeError: 'ContainerOp' object is not callable.

Not quite following, do you want samples for these cases?

Sorry for the confusion, yes I want to have a simplified sample for these output viewer case. For example: we can define a simple confusion matrix with only 2 labels, but with actual csv file and vocab living in KFP repo, or hardcoded confusion matrix values as an array in python code. so they are convenient to use. Because these placeholders are referenced in kubeflow doc: https://www.kubeflow.org/docs/components/pipelines/sdk/output-viewer/#available-output-viewers.

Is there any existing sample which can achieve this goal? I found this file https://github.com/kubeflow/pipelines/blob/master/components/local/confusion_matrix/component.yaml but was stuck at trying to create a GCSPath object for Predictions parameter.

KFP UI server fetches these artifacts, by default we already mount MinIO secrets on it. On GCP it gets cluster default service account permission. So you don't need to worry about permission, users are responsible for setting them in their cluster.

Sounds good, thank you for clarifying !

[Bobgy]

@zijianjoy https://www.github.com/kubeflow/pipelines/tree/f3e368410e2d29cf880cd6debc95572b636dd208/samples%2Fcore%2Fvisualization%2Ftensorboard_minio.py

You can adapt this example first. The only way to write a visualization component for both v1 and v2 compatible right now is to write a yaml component with artifact name mlpipeline-ui-metadata (it doesn't have spaces in the name).

Another way for lightweight components is to return the json metadata as NamedTuple. See example https://github.com/kubeflow/pipelines/blob/master/samples/core/lightweight_component/lightweight_component.ipynb.
I think it also works for v2 compatible, but you may need to confirm.

No, I don't have any existing examples. Existing examples are all around a specific use-case. It would be very helpful building an example with all the available visualizations simply hard coded.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.