refactor Components / Kubeflow Pipelines section (#3063)

* refactor pipelines section

* fix links & move "introduction" to root

* revert rename of "TFX Compatibility Matrix"

Author: thesuperzapper

content/en/_redirects

@@ -167,6 +167,16 @@ docs/started/requirements/                    /docs/started/getting-started/
 /docs/components/notebooks/troubleshoot                   /docs/components/notebooks/troubleshooting
 /docs/components/notebooks/why-use-jupyter-notebook       /docs/components/notebooks/overview
 
+# Refactor Pipelines section
+/docs/components/pipelines/caching                            /docs/components/pipelines/overview/caching
+/docs/components/pipelines/caching-v2                         /docs/components/pipelines/overview/caching-v2
+/docs/components/pipelines/multi-user                         /docs/components/pipelines/overview/multi-user
+/docs/components/pipelines/pipeline-root                      /docs/components/pipelines/overview/pipeline-root
+/docs/components/pipelines/pipelines-overview                 /docs/components/pipelines/introduction
+/docs/components/pipelines/pipelines-quickstart               /docs/components/pipelines/overview/quickstart
+/docs/components/pipelines/overview/concepts/*                /docs/components/pipelines/concepts/:splat
+/docs/components/pipelines/sdk/v2/*                           /docs/components/pipelines/sdk-v2/:splat
+
 # ===============
 # IMPORTANT NOTE:
 # Catch-all redirects should be added at the end of this file as redirects happen from top to bottom

content/en/docs/components/pipelines/concepts/_index.md

@@ -0,0 +1,5 @@
++++
+title = "Concepts"
+description = "Concepts used in Kubeflow Pipelines"
+weight = 30
++++

content/en/docs/components/pipelines/overview/concepts/component.md content/en/docs/components/pipelines/concepts/component.md

@@ -57,8 +57,8 @@ deserialize the data for use in the downstream component.
 
 ## Next steps
 
-* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/pipelines-overview/).
-* Follow the [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart/) 
+* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/introduction/).
+* Follow the [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart/) 
   to deploy Kubeflow and run a sample pipeline directly from the Kubeflow 
   Pipelines UI.
 * Build your own 

content/en/docs/components/pipelines/overview/concepts/experiment.md content/en/docs/components/pipelines/concepts/experiment.md

@@ -8,11 +8,11 @@ weight = 40
 An *experiment* is a workspace where you can try different configurations of
 your pipelines. You can use experiments to organize your runs into logical
 groups. Experiments can contain arbitrary runs, including 
-[recurring runs](/docs/components/pipelines/overview/concepts/run/).
+[recurring runs](/docs/components/pipelines/concepts/run/).
 
 ## Next steps
 
-* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/overview/pipelines-overview/).
-* Follow the [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart/) 
+* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/introduction/).
+* Follow the [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart/) 
   to deploy Kubeflow and run a sample pipeline directly from the Kubeflow 
   Pipelines UI.

content/en/docs/components/pipelines/overview/concepts/graph.md content/en/docs/components/pipelines/concepts/graph.md

@@ -24,7 +24,7 @@ parent contains a conditional clause.)
 
 ## Next steps
 
-* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/pipelines-overview/).
-* Follow the [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart/) 
+* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/introduction/).
+* Follow the [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart/) 
   to deploy Kubeflow and run a sample pipeline directly from the Kubeflow 
   Pipelines UI.

content/en/docs/components/pipelines/overview/concepts/output-artifact.md content/en/docs/components/pipelines/concepts/output-artifact.md

@@ -15,8 +15,8 @@ data to rich interactive visualizations.
 
 ## Next steps
 
-* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/overview/pipelines-overview/).
-* Follow the [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart/) 
+* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/introduction/).
+* Follow the [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart/) 
   to deploy Kubeflow and run a sample pipeline directly from the Kubeflow 
   Pipelines UI.
 * Read more about the available 

content/en/docs/components/pipelines/overview/concepts/pipeline.md content/en/docs/components/pipelines/concepts/pipeline.md

@@ -6,19 +6,19 @@ weight = 10
 +++
 
 A *pipeline* is a description of a machine learning (ML) workflow, including all
-of the [components](/docs/components/pipelines/overview/concepts/component/) in the workflow and how the components relate to each other in
-the form of a [graph](/docs/components/pipelines/overview/concepts/graph/). The pipeline
+of the [components](/docs/components/pipelines/concepts/component/) in the workflow and how the components relate to each other in
+the form of a [graph](/docs/components/pipelines/concepts/graph/). The pipeline
 configuration includes the definition of the inputs (parameters) required to run
 the pipeline and the inputs and outputs of each component.
 
 When you run a pipeline, the system launches one or more Kubernetes Pods
-corresponding to the [steps](/docs/components/pipelines/overview/concepts/step/) (components) in your workflow (pipeline). The Pods
+corresponding to the [steps](/docs/components/pipelines/concepts/step/) (components) in your workflow (pipeline). The Pods
 start Docker containers, and the containers in turn start your programs.
 
 After developing your pipeline, you can upload your pipeline using the Kubeflow Pipelines UI or the Kubeflow Pipelines SDK.
 
 ## Next steps
-* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/overview/pipelines-overview/).
-* Follow the [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart/) 
+* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/introduction/).
+* Follow the [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart/) 
   to deploy Kubeflow and run a sample pipeline directly from the Kubeflow 
   Pipelines UI.

content/en/docs/components/pipelines/overview/concepts/run-trigger.md content/en/docs/components/pipelines/concepts/run-trigger.md

@@ -15,7 +15,7 @@ available:
 
 ## Next steps
 
-* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/overview/pipelines-overview/).
-* Follow the [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart/) 
+* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/introduction/).
+* Follow the [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart/) 
   to deploy Kubeflow and run a sample pipeline directly from the Kubeflow 
   Pipelines UI.

content/en/docs/components/pipelines/overview/concepts/run.md content/en/docs/components/pipelines/concepts/run.md

@@ -15,7 +15,7 @@ output artifacts, and logs for each step in the run.
 A *recurring run*, or job in the Kubeflow Pipelines [backend APIs](https://github.com/kubeflow/pipelines/tree/06e4dc660498ce10793d566ca50b8d0425b39981/backend/api/go_http_client/job_client), is a repeatable run of
 a pipeline. The configuration for a recurring run includes a copy of a pipeline
 with all parameter values specified and a 
-[run trigger](/docs/components/pipelines/overview/concepts/run-trigger/).
+[run trigger](/docs/components/pipelines/concepts/run-trigger/).
 You can start a recurring run inside any experiment, and it will periodically
 start a new copy of the run configuration. You can enable/disable the recurring
 run from the Kubeflow Pipelines UI. You can also specify the maximum number of
@@ -25,7 +25,7 @@ triggered to run frequently.
 
 ## Next steps
 
-* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/pipelines-overview/).
-* Follow the [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart/) 
+* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/introduction/).
+* Follow the [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart/) 
   to deploy Kubeflow and run a sample pipeline directly from the Kubeflow 
   Pipelines UI.

content/en/docs/components/pipelines/overview/concepts/step.md content/en/docs/components/pipelines/concepts/step.md

@@ -13,7 +13,7 @@ an if/else like clause in the pipeline code.
 
 ## Next steps
 
-* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/pipelines-overview/).
-* Follow the [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart/) 
+* Read an [overview of Kubeflow Pipelines](/docs/components/pipelines/introduction/).
+* Follow the [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart/) 
   to deploy Kubeflow and run a sample pipeline directly from the Kubeflow 
   Pipelines UI.

content/en/docs/components/pipelines/installation/_index.md

@@ -1,5 +1,5 @@
 +++
-title = "Installing Pipelines"
+title = "Installation"
 description = "Options for installing Kubeflow Pipelines"
-weight = 15
+weight = 35
 +++

content/en/docs/components/pipelines/installation/choose-executor.md

@@ -1,7 +1,7 @@
 +++
-title = "Argo Workflow Executors"
-description = "How to choose and configure the Argo Workflow Executor?"
-weight = 40
+title = "Choosing an Argo Workflows Executor"
+description = "How to choose an Argo Workflows Executor"
+weight = 80
 +++
 
 An Argo workflow executor is a process that conforms to a specific interface that allows Argo to perform certain actions like monitoring pod logs, collecting artifacts, managing container lifecycles, etc.
@@ -81,7 +81,7 @@ Pipelines test infrastructure has been running stably with the emissary executor
   * Cannot escape the privileges of the pod's service account.
 * Migration: `command` must be specified in [Kubeflow Pipelines component specification](https://www.kubeflow.org/docs/components/pipelines/reference/component-spec/).
 
-  Note, the same migration requirement is required by [Kubeflow Pipelines v2 compatible mode](https://www.kubeflow.org/docs/components/pipelines/sdk/v2/v2-compatibility/), refer to
+  Note, the same migration requirement is required by [Kubeflow Pipelines v2 compatible mode](https://www.kubeflow.org/docs/components/pipelines/sdk-v2/v2-compatibility/), refer to
   [known caveats & breaking changes](https://github.com/kubeflow/pipelines/issues/6133).
 
 #### Migrate to Emissary Executor

content/en/docs/components/pipelines/installation/compatibility-matrix.md

@@ -1,7 +1,7 @@
 +++
 title = "Compatibility Matrix"
 description = "Kubeflow Pipelines compatibility matrix with TensorFlow Extended (TFX)"
-weight = 50
+weight = 100
 +++
 
 ## Kubeflow Pipelines Backend and TFX compatibility

content/en/docs/components/pipelines/installation/localcluster-deployment.md

@@ -1,7 +1,7 @@
 +++
-title = "Deploying Kubeflow Pipelines on a local cluster"
-description = "How to deploy Kubeflow Pipelines locally with kind, K3s, and K3ai for testing purposes"
-weight = 30
+title = "Local Deployment"
+description = "Information about local Deployment of Kubeflow Pipelines (kind, K3s, K3ai)"
+weight = 20
 +++
 
 This guide shows how to deploy Kubeflow Pipelines standalone on a local

content/en/docs/components/pipelines/installation/overview.md

@@ -1,5 +1,5 @@
 +++
-title = "Installation Options for Kubeflow Pipelines"
+title = "Installation Options"
 description = "Overview of the ways to deploy Kubeflow Pipelines"
 weight = 10
 

content/en/docs/components/pipelines/installation/standalone-deployment.md

@@ -1,7 +1,7 @@
 +++
-title = "Kubeflow Pipelines Standalone Deployment"
-description = "Instructions to deploy Kubeflow Pipelines standalone to a cluster"
-weight = 20
+title = "Standalone Deployment"
+description = "Information about Standalone Deployment of Kubeflow Pipelines"
+weight = 30
 +++
 
 As an alternative to deploying Kubeflow Pipelines (KFP) as part of the

content/en/docs/components/pipelines/installation/upgrade.md

@@ -1,7 +1,7 @@
 +++
 title = "Upgrade Notes"
 description = "Notices and breaking changes when you upgrade Kubeflow Pipelines Backend"
-weight = 35
+weight = 90
 +++
 
 This page introduces notices and breaking changes you need to know when upgrading Kubeflow Pipelines Backend.
@@ -24,4 +24,4 @@ For upgrade instructions, refer to distribution specific documentations:
 
     For detailed configuration and migration instructions for both options, refer to [Argo Workflow Executors](https://www.kubeflow.org/docs/components/pipelines/installation/choose-executor/).
 
-* **Notice**: [Kubeflow Pipelines SDK v2 compatibility mode](/docs/components/pipelines/sdk/v2/v2-compatibility/) (Beta) was recently released. The new mode adds support for tracking pipeline runs and artifacts using ML Metadata. In v1.7 backend, complete UI support and caching capabilities for v2 compatibility mode are newly added. We welcome any [feedback](https://github.com/kubeflow/pipelines/issues/6451) on positive experiences or issues you encounter.
+* **Notice**: [Kubeflow Pipelines SDK v2 compatibility mode](/docs/components/pipelines/sdk-v2/v2-compatibility/) (Beta) was recently released. The new mode adds support for tracking pipeline runs and artifacts using ML Metadata. In v1.7 backend, complete UI support and caching capabilities for v2 compatibility mode are newly added. We welcome any [feedback](https://github.com/kubeflow/pipelines/issues/6451) on positive experiences or issues you encounter.

content/en/docs/components/pipelines/overview/pipelines-overview.md content/en/docs/components/pipelines/introduction.md

@@ -1,6 +1,6 @@
 +++
-title = "Overview of Kubeflow Pipelines"
-description = "Understanding the goals and main concepts of Kubeflow Pipelines"
+title = "Introduction"
+description = "An introduction to the goals and main concepts of Kubeflow Pipelines"
 weight = 10
 
 +++
@@ -13,7 +13,7 @@ scalable machine learning (ML) workflows based on Docker containers.
 ## Quickstart
 
 Run your first pipeline by following the 
-[pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart).
+[pipelines quickstart guide](/docs/components/pipelines/overview/quickstart).
 
 ## What is Kubeflow Pipelines?
 
@@ -56,7 +56,7 @@ A _pipeline component_ is a self-contained set of user code, packaged as a
 performs one step in the pipeline. For example, a component can be responsible
 for data preprocessing, data transformation, model training, and so on.
 
-See the conceptual guides to [pipelines](/docs/components/pipelines/overview/concepts/pipeline/)
+See the conceptual guides to [pipelines](/docs/components/pipelines/concepts/pipeline/)
 and [components](/docs/components/pipelines/concepts/component/).
 
 ## Example of a pipeline
@@ -275,7 +275,7 @@ At a high level, the execution of a pipeline proceeds as follows:
 ## Next steps
 
 * Follow the 
-  [pipelines quickstart guide](/docs/components/pipelines/pipelines-quickstart) to 
+  [pipelines quickstart guide](/docs/components/pipelines/overview/quickstart) to 
   deploy Kubeflow and run a sample pipeline directly from the 
   Kubeflow Pipelines UI.
 * Build machine-learning pipelines with the [Kubeflow Pipelines 

content/en/docs/components/pipelines/overview/_index.md

@@ -1,5 +1,5 @@
 +++
-title = "Understanding Pipelines"
-description = "Overview and concepts in Kubeflow Pipelines"
+title = "Overview"
+description = "Overview of Kubeflow Pipelines"
 weight = 20
 +++

content/en/docs/components/pipelines/caching-v2.md content/en/docs/components/pipelines/overview/caching-v2.md

@@ -1,13 +1,13 @@
 +++
 title = "Caching v2"
 description = "Getting started with Kubeflow Pipelines caching v2"
-weight = 50
+weight = 41
 
 +++
 {{% beta-status
 feedbacklink="https://github.com/kubeflow/pipelines/issues" %}}
 
-Starting from [Kubeflow Pipelines SDK v2](https://www.kubeflow.org/docs/components/pipelines/sdk/v2/) and Kubeflow Pipelines 1.7.0, Kubeflow Pipelines supports step caching capabilities in both [standalone deployment](https://www.kubeflow.org/docs/components/pipelines/installation/standalone-deployment/) and [AI platform Pipelines](https://cloud.google.com/ai-platform/pipelines/docs).
+Starting from [Kubeflow Pipelines SDK v2](https://www.kubeflow.org/docs/components/pipelines/sdk-v2/) and Kubeflow Pipelines 1.7.0, Kubeflow Pipelines supports step caching capabilities in both [standalone deployment](https://www.kubeflow.org/docs/components/pipelines/installation/standalone-deployment/) and [AI platform Pipelines](https://cloud.google.com/ai-platform/pipelines/docs).
 
 ## Before you start
 This guide tells you the basic concepts of Kubeflow Pipelines caching and how to use it. 
@@ -17,7 +17,7 @@ guide](/docs/components/pipelines/installation/) to deploy Kubeflow Pipelines.
 ## What is step caching?
 
 Kubeflow Pipelines caching provides step-level output caching, a process that helps to reduce costs by skipping computations that were completed in a previous pipeline run.
-Caching is enabled by default for all tasks of pipelines built with [Kubeflow Pipelines SDK v2](https://www.kubeflow.org/docs/components/pipelines/sdk/v2/) using `kfp.dsl.PipelineExecutionMode.V2_COMPATIBLE` mode.
+Caching is enabled by default for all tasks of pipelines built with [Kubeflow Pipelines SDK v2](https://www.kubeflow.org/docs/components/pipelines/sdk-v2/) using `kfp.dsl.PipelineExecutionMode.V2_COMPATIBLE` mode.
 When Kubeflow Pipeline runs a pipeline, it checks to see whether 
 an execution exists in Kubeflow Pipeline with the interface of each pipeline task.
 The task's interface is defined as the combination of the pipeline task specification (base image, command, args), the pipeline task's inputs (the name and id of artifacts, the name and value of parameters),
@@ -33,6 +33,6 @@ If there is a matching execution in Kubeflow Pipelines, the outputs of that exec
 
 ## Disabling/enabling caching
 
-Cache is enabled by default with [Kubeflow Pipelines SDK v2](https://www.kubeflow.org/docs/components/pipelines/sdk/v2/) using `kfp.dsl.PipelineExecutionMode.V2_COMPATIBLE` mode.
+Cache is enabled by default with [Kubeflow Pipelines SDK v2](https://www.kubeflow.org/docs/components/pipelines/sdk-v2/) using `kfp.dsl.PipelineExecutionMode.V2_COMPATIBLE` mode.
 
 You can turn off execution caching for pipeline runs that are created using Python. When you run a pipeline using [create_run_from_pipeline_func](https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.client.html#kfp.Client.create_run_from_pipeline_func) or [create_run_from_pipeline_package](https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.client.html#kfp.Client.create_run_from_pipeline_package) or [run_pipeline](https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.client.html#kfp.Client.run_pipeline,) you can use the `enable_caching` argument to specify that this pipeline run does not use caching.

content/en/docs/components/pipelines/caching.md content/en/docs/components/pipelines/overview/caching.md

@@ -1,7 +1,7 @@
 +++
 title = "Caching"
 description = "Getting started with Kubeflow Pipelines step caching"
-weight = 50
+weight = 40
 
 +++
 {{% alpha-status

content/en/docs/components/pipelines/overview/concepts/_index.md

@@ -1,5 +1,5 @@
 +++
-title = "Introduction to the Pipelines Interfaces"
+title = "Pipelines Interfaces"
 description = "The ways you can interact with the Kubeflow Pipelines system"
 weight = 20
 
@@ -25,15 +25,15 @@ From the Kubeflow Pipelines UI you can perform the following tasks:
   that someone has shared with you.
 * Create an *experiment* to group one or more of your pipeline runs.
   See the [definition of an
-  experiment](/docs/components/pipelines/overview/concepts/experiment/).
+  experiment](/docs/components/pipelines/concepts/experiment/).
 * Create and start a *run* within the experiment. A run is a single execution
   of a pipeline. See the [definition of a
-  run](/docs/components/pipelines/overview/concepts/run/).
+  run](/docs/components/pipelines/concepts/run/).
 * Explore the configuration, graph, and output of your pipeline run.
 * Compare the results of one or more runs within an experiment.
 * Schedule runs by creating a recurring run.
 
-See the [quickstart guide](/docs/components/pipelines/pipelines-quickstart/) for more
+See the [quickstart guide](/docs/components/pipelines/overview/quickstart/) for more
 information about accessing the Kubeflow Pipelines UI and running the samples.
 
 When building a pipeline component, you can write out information for display

content/en/docs/components/pipelines/overview/interfaces.md

@@ -1,7 +1,7 @@
 +++
 title = "Multi-user Isolation for Pipelines"
 description = "Getting started with Kubeflow Pipelines multi-user isolation"
-weight = 35
+weight = 30
 +++
 
 Multi-user isolation for Kubeflow Pipelines is an integration to [Kubeflow multi-user isolation](/docs/components/multi-tenancy/).