Skip to content

Commit

Permalink
update KFP website for KFP SDK v2 GA (#3526)
Browse files Browse the repository at this point in the history
* update v2 section of docs

* update v1 section of docs

* capitalize F for consistency

* small simplification
  • Loading branch information
connor-mccarthy committed Jun 20, 2023
1 parent c62421e commit 21b9c33
Show file tree
Hide file tree
Showing 22 changed files with 45 additions and 57 deletions.
Expand Up @@ -48,13 +48,13 @@ building and running pipelines that are compatible with the Pipelines SDK v2.

## Before you begin

1. Run the following command to install the Kubeflow Pipelines SDK v1.6.2 or higher.
1. Run the following command to install the Kubeflow Pipelines SDK v1.8.
If you run this command in a Jupyter notebook, restart the kernel after
installing the SDK.


```python
$ pip install --upgrade kfp
$ pip install kfp==1.8
```

2. Import the `kfp` packages.
Expand Down
Expand Up @@ -21,10 +21,10 @@ building and running pipelines that are compatible with the Pipelines SDK v2.

## Before you begin

Run the following command to install the Kubeflow Pipelines SDK v1.6.1 or higher.
Run the following command to install the Kubeflow Pipelines SDK v1.8.

```bash
pip install kfp --upgrade
pip install kfp==1.8
```

For more information about the Kubeflow Pipelines SDK, see the [SDK reference guide][sdk-ref].
Expand Down
Expand Up @@ -59,12 +59,12 @@ building and running pipelines that are compatible with the Pipelines SDK v2.

## Before you begin

1. Run the following command to install the Kubeflow Pipelines SDK v1.6.2 or higher. If you run this command in a Jupyter
1. Run the following command to install the Kubeflow Pipelines SDK v1.8. If you run this command in a Jupyter
notebook, restart the kernel after installing the SDK.


```python
$ pip install --upgrade kfp
$ pip install kfp==1.8
```

2. Import the `kfp`, `kfp.dsl`, and `kfp.v2.dsl` packages.
Expand Down
Expand Up @@ -21,19 +21,19 @@ mode](https://bit.ly/kfp-v2-compatible), or join the Kubeflow Pipelines communit

1. Install [Kubeflow Pipelines Standalone](/docs/components/pipelines/installation/standalone-deployment) 1.7.0 or higher. Note, support for other distributions is under development, see [Current Caveats section](#current-caveats).

1. Run the following command to install Kubeflow Pipelines SDK v1.7.2 or higher. If you run this command in a Jupyter notebook, restart the kernel after installing the SDK.
2. Run the following command to install Kubeflow Pipelines SDK v1.8. If you run this command in a Jupyter notebook, restart the kernel after installing the SDK.

```bash
pip install kfp --upgrade
pip install kfp==1.8
```

1. Import the kfp and kfp.components packages.
3. Import the kfp and kfp.components packages.

```python
import kfp
```

1. Create an instance of the kfp.Client class. To find your Kubeflow Pipelines cluster’s hostname and URL scheme, open the Kubeflow Pipelines user interface in your browser. The URL of the Kubeflow Pipelines user interface is something like https://my-cluster.my-organization.com/pipelines. In this case, the host name and URL scheme are https://my-cluster.my-organization.com.
4. Create an instance of the kfp.Client class. To find your Kubeflow Pipelines cluster’s hostname and URL scheme, open the Kubeflow Pipelines user interface in your browser. The URL of the Kubeflow Pipelines user interface is something like https://my-cluster.my-organization.com/pipelines. In this case, the host name and URL scheme are https://my-cluster.my-organization.com.

```python
# If you run this command on a Jupyter notebook running on Kubeflow, you can
Expand Down
Expand Up @@ -13,7 +13,7 @@ components, and demonstrates how to get started building components.
Run the following command to install the Kubeflow Pipelines SDK.

```python
$ pip3 install kfp --upgrade
$ pip install kfp==1.8
```

For more information about the Kubeflow Pipelines SDK, see the [SDK reference guide][sdk-ref].
Expand Down
4 changes: 2 additions & 2 deletions content/en/docs/components/pipelines/v1/sdk/install-sdk.md
Expand Up @@ -78,7 +78,7 @@ up Python using [Miniconda](https://conda.io/miniconda.html):
Run the following command to install the Kubeflow Pipelines SDK:

```bash
pip3 install kfp --upgrade
pip install kfp==1.8
```

**Note:** If you are not using a virtual environment, such as `conda`, when installing the Kubeflow Pipelines SDK, you may receive the following error:
Expand All @@ -91,7 +91,7 @@ Consider using the `--user` option or check the permissions.
If you get this error, install `kfp` with the `--user` option:

```bash
pip3 install kfp --upgrade --user
pip install kfp==1.8
```

This command installs the `dsl-compile` and `kfp` binaries under `~/.local/bin`, which is not part of the PATH in some Linux distributions, such as Ubuntu. You can add `~/.local/bin` to your PATH by appending the following to a new line at the end of your `.bashrc` file:
Expand Down
Expand Up @@ -56,7 +56,7 @@ This document describes how to build Python function-based components and use th


```python
$ pip3 install kfp --upgrade
$ pip install kfp==1.8
```

2. Import the `kfp` package.
Expand Down
2 changes: 0 additions & 2 deletions content/en/docs/components/pipelines/v2/_index.md
Expand Up @@ -6,8 +6,6 @@ weight = 16

{{% kfp-v2-keywords %}}

**Note:** Kubeflow Pipelines v2 is in pre-release stage and is not yet stable. The v2 docs being are continually developed and links to v2 documentation are also not yet stable.

Please see [v1 documentation][v1-documentation] for documentation on the latest stable KFP release.

[v1-documentation]: /docs/components/pipelines/v1
11 changes: 4 additions & 7 deletions content/en/docs/components/pipelines/v2/cli.md
Expand Up @@ -9,7 +9,6 @@ weight = 9
<!-- TODO: Improve or standardize rendering of variables and placeholders -->
<!-- TODO: Standardize inline references to KFP CLI SDK -->

<!-- TODO: use /latest instead of /master when SDK goes GA -->
This section provides a summary of the available commands in the KFP CLI. For more comprehensive documentation about all the available commands in the KFP CLI, see [Command Line Interface][cli-reference-docs] in the [KFP SDK reference documentation][kfp-sdk-api-ref].

## Usage
Expand Down Expand Up @@ -129,18 +128,16 @@ You can use the `kfp dsl compile` command to compile pipelines or components def
```

### Build containerized Python components
<!-- TODO: Revisit the links after the refactoring is completed -->

You can author [Containerized Python Components][containerized-python-components] in the KFP SDK. This lets you use handle more source code with better code organization than the simpler [Lightweight Python Component][lightweight-python-component] authoring experience.

<!-- TODO(GA): remove --pre -->

#### Before you begin

Run the following command to install the KFP SDK with the additional Docker dependency:

```shell
pip install --pre kfp[all]
pip install kfp[all]
```

#### Build the component
Expand All @@ -157,9 +154,9 @@ For example:
kfp component build src/ --component-filepattern my_component --push-image
```

For more information about the arguments and flags supported by the `kfp component build` command, see [build](https://kubeflow-pipelines.readthedocs.io/en/master/source/cli.html#kfp-component-build) in the [KFP SDK API reference][kfp-sdk-api-ref]. For more information about creating containerized Python components, see [Authoring Python Containerized Components][containerized-python-components].
For more information about the arguments and flags supported by the `kfp component build` command, see [build](https://kubeflow-pipelines.readthedocs.io/en/stable/source/cli.html#kfp-component-build) in the [KFP SDK API reference][kfp-sdk-api-ref]. For more information about creating containerized Python components, see [Authoring Python Containerized Components][containerized-python-components].

[cli-reference-docs]: https://kubeflow-pipelines.readthedocs.io/en/master/source/cli.html
[kfp-sdk-api-ref]: https://kubeflow-pipelines.readthedocs.io/en/master/index.html
[cli-reference-docs]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/cli.html
[kfp-sdk-api-ref]: https://kubeflow-pipelines.readthedocs.io/en/stable/index.html
[lightweight-python-component]: /docs/components/pipelines/v2/components/lightweight-python-components
[containerized-python-components]: /docs/components/pipelines/v2/components/containerized-python-components
Expand Up @@ -64,7 +64,7 @@ Note that if you provide a `description` argument to the [`@dsl.pipeline`][dsl-p

[ir-yaml]: /docs/components/pipelines/v2/compile-a-pipeline#ir-yaml
[google-docstring-style]: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html
[dsl-pipeline]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.pipeline
[dsl-pipeline]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.pipeline
[output-artifacts]: /docs/components/pipelines/v2/data-types/artifacts#declaring-inputoutput-artifacts
[dsl-outputpath]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/dsl.html#kfp.dsl.OutputPath
[output-type-marker]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/dsl.html#kfp.dsl.Output
Expand Up @@ -100,7 +100,7 @@ compiler.Compiler().compile(addition_pipeline, 'pipeline.yaml')
Since `add`'s `target_image` uses [Google Cloud Artifact Registry][artifact-registry] (indicated by the `gcr.io` URI), the pipeline shown here assumes you have pushed your image to Google Cloud Artifact Registry, you are running your pipeline on [Google Cloud Vertex AI Pipelines][vertex-pipelines], and you have configured [IAM permissions][iam] so that Vertex AI Pipelines can pull images from Artifact Registry.


[kfp-component-build]: https://kubeflow-pipelines.readthedocs.io/en/master/source/cli.html#kfp-component-build
[kfp-component-build]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/cli.html#kfp-component-build
[lightweight-python-components]: /docs/components/pipelines/v2/components/lightweight-python-components
[image-tag]: https://docs.docker.com/engine/reference/commandline/tag/
[docker-from]: https://docs.docker.com/engine/reference/builder/#from
Expand Down
Expand Up @@ -126,7 +126,7 @@ Note that setting `install_kfp_package` to `False` is rarely necessary and is di

[hello-world-pipeline]: /docs/components/pipelines/v2/hello-world
[containerized-python-components]: /docs/components/pipelines/v2/components/containerized-python-components
[dsl-component]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.component
[dsl-component]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.component
[data-types]: /docs/components/pipelines/v2/data-types
[parameters]: /docs/components/pipelines/v2/data-types/parameters
[artifacts]: /docs/components/pipelines/v2/data-types/artifacts
Expand Down
Expand Up @@ -171,7 +171,7 @@ def my_pipeline() -> NamedTuple('pipeline_outputs', c=int, d=str):
[containerized-python-components]: /docs/components/pipelines/v2/components/containerized-python-components
[container-component]: /docs/components/pipelines/v2/components/container-components
[container-component-outputs]: /docs/components/pipelines/v2/components/container-components#create-component-outputs
[pipelinetask]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.PipelineTask
[pipelinetask]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.PipelineTask
[dsl-outputpath]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/dsl.html#kfp.dsl.OutputPath
[ml-metadata]: https://github.com/google/ml-metadata
[typing-namedtuple]: https://docs.python.org/3/library/typing.html#typing.NamedTuple
10 changes: 5 additions & 5 deletions content/en/docs/components/pipelines/v2/hello-world.md
Expand Up @@ -9,7 +9,7 @@ weight = 3
To get started with the tutorials, pip install `kfp` v2:

```sh
pip install kfp --pre
pip install kfp
```

Here is a simple pipeline that prints a greeting:
Expand Down Expand Up @@ -58,9 +58,9 @@ The client will print a link to view the pipeline execution graph and logs in th
In the next few sections, you'll learn more about the core concepts of authoring pipelines and how to create more expressive, useful pipelines.

[installation]: /docs/components/pipelines/v2/installation/
[client]: https://kubeflow-pipelines.readthedocs.io/en/master/source/client.html#kfp.client.Client
[compiler]: https://kubeflow-pipelines.readthedocs.io/en/master/source/compiler.html#kfp.compiler.Compiler
[client]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html#kfp.client.Client
[compiler]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/compiler.html#kfp.compiler.Compiler
[ir-yaml]: /docs/components/pipelines/v2/compile-a-pipeline#ir-yaml
[compile-a-pipeline]: /docs/components/pipelines/v2/compile-a-pipeline/
[dsl-pipeline]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.pipeline
[dsl-component]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.component
[dsl-pipeline]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.pipeline
[dsl-component]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.component
Expand Up @@ -26,11 +26,9 @@ Before you begin, you need the following prerequisites:

* Run the following script to install the KFP SDK:
```shell
pip install kfp --pre
pip install kfp
```

**Note:** This command installs KFP v2, which is in pre-release stage and is not yet stable. The v2 documentation is being developed continually and some of the links to the v2 documentation might be unavailable.

After you complete the prerequisites, click each step to view the instructions:

<details>
Expand Down
3 changes: 1 addition & 2 deletions content/en/docs/components/pipelines/v2/introduction.md
Expand Up @@ -41,7 +41,6 @@ A [pipeline][pipelines] is a definition of a workflow that composes one or more
[pipelines]: /docs/components/pipelines/v2/pipelines
[installation]: /docs/components/pipelines/v2/installation
[ir-yaml]: /docs/components/pipelines/v2/compile-a-pipeline#ir-yaml
<!-- GA TODO: drop /#history tag -->
[pypi]: https://pypi.org/project/kfp/#history
[pypi]: https://pypi.org/project/kfp/
[hello-world-pipeline]: /docs/components/pipelines/v2/hello-world
[control-flow]: /docs/components/pipelines/v2/pipelines/control-flow
16 changes: 7 additions & 9 deletions content/en/docs/components/pipelines/v2/migration.md
Expand Up @@ -310,9 +310,7 @@ def flip_coin(rand: int, result: dsl.OutputPath(str)):

`VolumeOp` and `ResourceOp` expose direct access to Kubernetes resources within a pipeline definition. There is no support for these features on a non-Kubernetes platforms.

These features will be supported via platform-specific functionality, which includes Kubernetes-specific functionality. This functionality is a work in progress and will be supported in the KFP v2 GA release.

<!-- TODO: link to docs and add change instructions once released -->
KFP v2 enables support for [platform-specific features](/docs/components/pipelines/v2/platform-specific-features/) via KFP SDK extension libraries. Kubernetes-specific features are supported in KFP v2 via the [`kfp-kubernetes`](https://kfp-kubernetes.readthedocs.io/) extension library.

#### v1 component YAML support

Expand Down Expand Up @@ -481,15 +479,15 @@ If you believe we missed a breaking change or an important migration step, pleas
[artifacts]: /docs/components/pipelines/v2/data-types/artifacts
[cli]: /docs/components/pipelines/v2/cli/
[compile]: /docs/components/pipelines/v2/compile-a-pipeline
[compiler-compile]: https://kubeflow-pipelines.readthedocs.io/en/master/source/compiler.html#kfp.compiler.Compiler.compile
[components-load-component-from-file]: https://kubeflow-pipelines.readthedocs.io/en/master/source/components.html#kfp.components.load_component_from_file
[compiler-compile]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/compiler.html#kfp.compiler.Compiler.compile
[components-load-component-from-file]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/components.html#kfp.components.load_component_from_file
[container-components]: https://www.kubeflow.org/docs/components/pipelines/v2/components/containerized-python-components/
[containerized-python-components]: /docs/components/pipelines/v2/components/containerized-python-components/
[create-custom-training-job-from-component]: https://cloud.google.com/vertex-ai/docs/pipelines/customjob-component
[dsl-collected]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.Collected
[dsl-component]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.component
[dsl-container-component]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.container_component
[dsl-parallelfor]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.ParallelFor
[dsl-collected]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.Collected
[dsl-component]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.component
[dsl-container-component]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.container_component
[dsl-parallelfor]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.ParallelFor
[gcpc]: https://cloud.google.com/vertex-ai/docs/pipelines/components-introduction
[ir-yaml]: /docs/components/pipelines/v2/compile-a-pipeline/#ir-yaml
[lightweight-python-components]: /docs/components/pipelines/v2/components/lightweight-python-components/
Expand Down
Expand Up @@ -188,16 +188,16 @@ def pythagorean(a: float = 1.2, b: float = 1.2) -> float:
```

<!-- TODO: make this reference more precise throughout -->
[dsl-reference-docs]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html
[dsl-pipeline]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.pipeline
[dsl-reference-docs]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html
[dsl-pipeline]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.pipeline
[control-flow]: /docs/components/pipelines/v2/pipelines/control-flow
[components]: /docs/components/pipelines/v2/components
[pipelinetask]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.PipelineTask
[pipelinetask]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.PipelineTask
[vertex-pipelines]: https://cloud.google.com/vertex-ai/docs/pipelines/introduction
[oss-be]: /docs/components/pipelines/v2/installation/
[data-types]: /docs/components/pipelines/v2/data-types
[output-artifacts]: /docs/components/pipelines/v2/data-types/artifacts#using-output-artifacts
[container-component-outputs]: /docs/components/pipelines/v2/components/container-components#create-component-outputs
[parameters-namedtuple]: /docs/components/pipelines/v2/data-types/parameters#multiple-output-parameters
[dsl-pipeline-job-name-placeholder]: https://kubeflow-pipelines.readthedocs.io/en/master/source/dsl.html#kfp.dsl.PIPELINE_JOB_NAME_PLACEHOLDER
[dsl-pipeline-job-name-placeholder]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/dsl.html#kfp.dsl.PIPELINE_JOB_NAME_PLACEHOLDER
[component-docstring-format]: /docs/components/pipelines/v2/components/additional-functionality#component-docstring-format
@@ -1,5 +1,5 @@
+++
title = "Platform-specific features"
title = "Platform-specific Features"
description = "Author tasks with platform-specific functionality"
weight = 10
+++
Expand All @@ -25,7 +25,7 @@ The following assumes basic familiarity with [PersistentVolume and PersistentVol
### Step 1: Install the platform-specific plugin library with the KFP SDK

```sh
pip install kfp[kubernetes] --pre
pip install kfp[kubernetes]
```

### Step 2: Create components that read/write to the mount path
Expand Down
4 changes: 1 addition & 3 deletions content/en/docs/components/pipelines/v2/reference/sdk.md
Expand Up @@ -5,6 +5,4 @@ weight = 20

+++

See the [generated reference docs for the Python
SDK V2](https://kubeflow-pipelines.readthedocs.io/en/master/) (hosted on
*Read the Docs*).
See the [KFP SDK v2 reference documentation](https://kubeflow-pipelines.readthedocs.io/en/stable/).
4 changes: 2 additions & 2 deletions content/en/docs/components/pipelines/v2/run-a-pipeline.md
Expand Up @@ -77,6 +77,6 @@ kfp run create --experiment-name my-experiment --package-file path/to/pipeline.y
For more information about the `kfp run create` command, see the [KFP Command Line Interface reference documentation][kfp-run-create-reference-docs]. For more information on the KFP CLI generally see [Command Line Interface user docs][kfp-cli].

[compile-a-pipeline]: /docs/components/pipelines/v2/compile-a-pipeline
[kfp-sdk-api-ref-client]: https://kubeflow-pipelines.readthedocs.io/en/master/source/client.html
[kfp-sdk-api-ref-client]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/client.html
[kfp-cli]: /docs/components/pipelines/v2/cli/
[kfp-run-create-reference-docs]: https://kubeflow-pipelines.readthedocs.io/en/master/source/cli.html#kfp-run-create
[kfp-run-create-reference-docs]: https://kubeflow-pipelines.readthedocs.io/en/stable/source/cli.html#kfp-run-create
Empty file.

0 comments on commit 21b9c33

Please sign in to comment.