Skip to content

Commit

Permalink
fix: update KFP SDK doc links throughout pipelines docs. fix #3569 (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
ryanjung-jm committed Sep 11, 2023
1 parent 9e19444 commit 83cd2d1
Show file tree
Hide file tree
Showing 18 changed files with 110 additions and 110 deletions.
Expand Up @@ -35,4 +35,4 @@ If there is a matching execution in Kubeflow Pipelines, the outputs of that exec

Cache is enabled by default with [Kubeflow Pipelines SDK v2](https://www.kubeflow.org/docs/components/pipelines/sdk-v2/) using `kfp.dsl.PipelineExecutionMode.V2_COMPATIBLE` mode.

You can turn off execution caching for pipeline runs that are created using Python. When you run a pipeline using [create_run_from_pipeline_func](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client.create_run_from_pipeline_func) or [create_run_from_pipeline_package](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client.create_run_from_pipeline_package) or [run_pipeline](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client.run_pipeline,) you can use the `enable_caching` argument to specify that this pipeline run does not use caching.
You can turn off execution caching for pipeline runs that are created using Python. When you run a pipeline using [create_run_from_pipeline_func](https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client.create_run_from_pipeline_func) or [create_run_from_pipeline_package](https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client.create_run_from_pipeline_package) or [run_pipeline](https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client.run_pipeline,) you can use the `enable_caching` argument to specify that this pipeline run does not use caching.
Expand Up @@ -80,9 +80,9 @@ print(client.list_runs(namespace=user_namespace))
```

{{% alert title="Tip" color="info" %}}
* To set a default namespace for Pipelines SDK commands, use the [`kfp.Client().set_user_namespace()`](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client.set_user_namespace) method,
* To set a default namespace for Pipelines SDK commands, use the [`kfp.Client().set_user_namespace()`](https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client.set_user_namespace) method,
this method stores your user namespace in a configuration file at `$HOME/.config/kfp/context.json`.
* Detailed documentation for `kfp.Client()` can be found in the [Kubeflow Pipelines SDK Reference](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html).
* Detailed documentation for `kfp.Client()` can be found in the [Kubeflow Pipelines SDK Reference](https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html).
{{% /alert %}}

## When using the REST API
Expand Down
Expand Up @@ -41,7 +41,7 @@ If you want to specify the `pipeline root` to S3, please choose one of the follo
`dsl.get_pipeline_conf().add_op_transformer(aws.use_aws_secret('xxx', ‘xxx’, ‘xxx’))`

**references**:
* [add-op-transformer](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.dsl.html#kfp.dsl.PipelineConf.add_op_transformer)
* [add-op-transformer](https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.PipelineConf.add_op_transformer)
* [use-aws-secret](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.extensions.html#kfp.aws.use_aws_secret)

## How to configure pipeline root
Expand All @@ -60,9 +60,9 @@ You can configure a pipeline root through the `kfp.dsl.pipeline` annotation when

#### Via Submitting a Pipeline through SDK
You can configure pipeline root via `pipeline_root` argument when you submit a Pipeline using one of the following:
* [create_run_from_pipeline_func](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client.create_run_from_pipeline_func)
* [create_run_from_pipeline_package](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client.create_run_from_pipeline_package)
* [run_pipeline](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client.run_pipeline).
* [create_run_from_pipeline_func](https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client.create_run_from_pipeline_func)
* [create_run_from_pipeline_package](https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client.create_run_from_pipeline_package)
* [run_pipeline](https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client.run_pipeline).

#### Via Submitting a Pipeline Run through UI
You can configure a pipeline root via the `pipeline_root` run parameters when you submit a pipeline run in the UI
Expand Down
Expand Up @@ -268,7 +268,7 @@
" public website. The function extracts the CSV files and then merges them\n",
" into a single file.\n",
"\n",
"[container-op]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.dsl.html#kfp.dsl.ContainerOp\n",
"[container-op]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/dsl.html#kfp.dsl.ContainerOp\n",
"[component-spec]: https://www.kubeflow.org/docs/components/pipelines/reference/component-spec/\n",
"[python-function-component]: https://www.kubeflow.org/docs/components/pipelines/sdk-v2/python-function-components/\n",
"[component-dev]: https://www.kubeflow.org/docs/components/pipelines/sdk-v2/component-development/\n",
Expand Down Expand Up @@ -454,7 +454,7 @@
" to load the component specification YAML for any components that you are\n",
" reusing in this pipeline.\n",
"\n",
"[load_component_from_url]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html?highlight=load_component_from_url#kfp.components.load_component_from_url"
"[load_component_from_url]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html?highlight=load_component_from_url#kfp.components.load_component_from_url"
]
},
{
Expand Down Expand Up @@ -570,7 +570,7 @@
"\n",
"1. Create an instance of the [`kfp.Client` class][kfp-client] following steps in [connecting to Kubeflow Pipelines using the SDK client][connect-api].\n",
"\n",
"[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.client.html#kfp.Client\n",
"[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/client.html#kfp.Client\n",
"[connect-api]: https://www.kubeflow.org/docs/components/pipelines/sdk/connect-api"
]
},
Expand Down
Expand Up @@ -264,7 +264,7 @@ when designing a pipeline.
public website. The function extracts the CSV files and then merges them
into a single file.

[container-op]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.dsl.html#kfp.dsl.ContainerOp
[container-op]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.ContainerOp
[component-spec]: https://www.kubeflow.org/docs/components/pipelines/reference/component-spec/
[python-function-component]: https://www.kubeflow.org/docs/components/pipelines/sdk-v2/python-function-components/
[component-dev]: https://www.kubeflow.org/docs/components/pipelines/sdk-v2/component-development/
Expand Down Expand Up @@ -403,7 +403,7 @@ def merge_csv(tar_data: Input[Artifact], output_csv: Output[Dataset]):
to load the component specification YAML for any components that you are
reusing in this pipeline.

[load_component_from_url]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html?highlight=load_component_from_url#kfp.components.load_component_from_url
[load_component_from_url]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html?highlight=load_component_from_url#kfp.components.load_component_from_url


```python
Expand Down Expand Up @@ -476,7 +476,7 @@ See the guide to [getting started with the UI][quickstart].

1. Create an instance of the [`kfp.Client` class][kfp-client] following steps in [connecting to Kubeflow Pipelines using the SDK client][connect-api].

[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client
[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client
[connect-api]: https://www.kubeflow.org/docs/components/pipelines/sdk/connect-api


Expand Down
Expand Up @@ -559,10 +559,10 @@ client.create_run_from_pipeline_func(my_pipeline, arguments={},
mode=kfp.dsl.PipelineExecutionMode.V2_COMPATIBLE)
```

[kfp-load-comp-file]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.load_component_from_file
[kfp-load-comp-url]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.load_component_from_url
[kfp-load-comp-text]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.load_component_from_text
[kfp-containerop]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.dsl.html#kfp.dsl.ContainerOp
[kfp-load-comp-file]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.load_component_from_file
[kfp-load-comp-url]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.load_component_from_url
[kfp-load-comp-text]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.load_component_from_text
[kfp-containerop]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.ContainerOp

## Organizing the component files

Expand Down
Expand Up @@ -78,7 +78,7 @@
"source": [
"3. Create an instance of the [`kfp.Client` class][kfp-client] following steps in [connecting to Kubeflow Pipelines using the SDK client][connect-api].\n",
"\n",
"[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.client.html#kfp.Client\n",
"[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/client.html#kfp.Client\n",
"[connect-api]: https://www.kubeflow.org/docs/components/pipelines/sdk/connect-api"
]
},
Expand Down Expand Up @@ -110,7 +110,7 @@
" into a factory function that you can use to create\n",
" [`kfp.dsl.ContainerOp`][container-op] class instances to use as steps in your pipeline.\n",
"\n",
"[container-op]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.dsl.html#kfp.dsl.ContainerOp"
"[container-op]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.ContainerOp"
]
},
{
Expand Down Expand Up @@ -329,7 +329,7 @@
"The following example demonstrates how to return multiple outputs by value. \n",
"\n",
"[python37]: https://hub.docker.com/layers/python/library/python/3.7/images/sha256-7eef781ed825f3b95c99f03f4189a8e30e718726e8490651fa1b941c6c815ad1?context=explore\n",
"[create-component-from-func]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.create_component_from_func\n",
"[create-component-from-func]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.create_component_from_func\n",
"[subprocess]: https://docs.python.org/3/library/subprocess.html\n",
"[tf-docker]: https://www.tensorflow.org/install/docker\n",
"[pytorch-docker]: https://hub.docker.com/r/pytorch/pytorch/tags\n",
Expand All @@ -338,8 +338,8 @@
"[named-tuple]: https://docs.python.org/3/library/collections.html#collections.namedtuple\n",
"[kfp-visualize]: https://www.kubeflow.org/docs/components/pipelines/sdk/output-viewer/\n",
"[kfp-metrics]: https://www.kubeflow.org/docs/components/pipelines/sdk/pipelines-metrics/\n",
"[input-path]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.InputPath\n",
"[output-path]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.OutputPath\n",
"[input-path]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.InputPath\n",
"[output-path]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.OutputPath\n",
"[vs-dsl-component]: https://github.com/kubeflow/pipelines/blob/sdk/release-1.8/sdk/python/kfp/v2/components/component_decorator.py"
]
},
Expand Down Expand Up @@ -456,12 +456,12 @@
" that your function expects an argument to be an\n",
" [`io.TextIOWrapper`][textiowrapper] that this function can write to.\n",
"\n",
"[input-binary]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.InputBinaryFile\n",
"[input-path]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.InputPath\n",
"[input-text]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.InputTextFile\n",
"[output-binary]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.OutputBinaryFile\n",
"[output-path]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.OutputPath\n",
"[output-text]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.OutputTextFile\n",
"[input-binary]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.InputBinaryFile\n",
"[input-path]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.InputPath\n",
"[input-text]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.InputTextFile\n",
"[output-binary]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.OutputBinaryFile\n",
"[output-path]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.OutputPath\n",
"[output-text]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/components.html#kfp.components.OutputTextFile\n",
"[bytesio]: https://docs.python.org/3/library/io.html#io.BytesIO\n",
"[textiowrapper]: https://docs.python.org/3/library/io.html#io.TextIOWrapper\n",
"\n",
Expand Down
Expand Up @@ -84,7 +84,7 @@ from kfp.v2.dsl import (

3. Create an instance of the [`kfp.Client` class][kfp-client] following steps in [connecting to Kubeflow Pipelines using the SDK client][connect-api].

[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client
[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client
[connect-api]: https://www.kubeflow.org/docs/components/pipelines/sdk/connect-api


Expand All @@ -107,7 +107,7 @@ through the process of creating a simple component.
into a factory function that you can use to create
[`kfp.dsl.ContainerOp`][container-op] class instances to use as steps in your pipeline.

[container-op]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.dsl.html#kfp.dsl.ContainerOp
[container-op]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.ContainerOp


```python
Expand Down Expand Up @@ -299,7 +299,7 @@ outputs as a new subclass of `tuple`.
The following example demonstrates how to return multiple outputs by value.

[python37]: https://hub.docker.com/layers/python/library/python/3.7/images/sha256-7eef781ed825f3b95c99f03f4189a8e30e718726e8490651fa1b941c6c815ad1?context=explore
[create-component-from-func]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.create_component_from_func
[create-component-from-func]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.create_component_from_func
[subprocess]: https://docs.python.org/3/library/subprocess.html
[tf-docker]: https://www.tensorflow.org/install/docker
[pytorch-docker]: https://hub.docker.com/r/pytorch/pytorch/tags
Expand All @@ -308,8 +308,8 @@ The following example demonstrates how to return multiple outputs by value.
[named-tuple]: https://docs.python.org/3/library/collections.html#collections.namedtuple
[kfp-visualize]: https://www.kubeflow.org/docs/components/pipelines/sdk/output-viewer/
[kfp-metrics]: https://www.kubeflow.org/docs/components/pipelines/sdk/pipelines-metrics/
[input-path]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.InputPath
[output-path]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.OutputPath
[input-path]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.InputPath
[output-path]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.OutputPath
[vs-dsl-component]: https://github.com/kubeflow/pipelines/blob/sdk/release-1.8/sdk/python/kfp/v2/components/component_decorator.py


Expand Down Expand Up @@ -408,12 +408,12 @@ To return a file as an output, use one of the following type annotations:
that your function expects an argument to be an
[`io.TextIOWrapper`][textiowrapper] that this function can write to.

[input-binary]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.InputBinaryFile
[input-path]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.InputPath
[input-text]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.InputTextFile
[output-binary]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.OutputBinaryFile
[output-path]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.OutputPath
[output-text]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.OutputTextFile
[input-binary]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.InputBinaryFile
[input-path]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.InputPath
[input-text]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.InputTextFile
[output-binary]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.OutputBinaryFile
[output-path]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.OutputPath
[output-text]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.OutputTextFile
[bytesio]: https://docs.python.org/3/library/io.html#io.BytesIO
[textiowrapper]: https://docs.python.org/3/library/io.html#io.TextIOWrapper

Expand Down
Expand Up @@ -157,5 +157,5 @@ Kubeflow Pipelines v2 compatible mode is currently in Beta stage. It is under ac
[build-pipeline]: /docs/components/pipelines/sdk-v2/build-pipeline/
[build-component]: /docs/components/pipelines/sdk-v2/component-development/
[python-component]: /docs/components/pipelines/sdk-v2/python-function-components/
[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.client.html#kfp.Client
[kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.Client
[connect-api]: /docs/components/pipelines/sdk/connect-api/

0 comments on commit 83cd2d1

Please sign in to comment.