Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/book/.gitbook/assets/create-snapshot-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/book/.gitbook/assets/create-snapshot-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/book/.gitbook/assets/run-snapshot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Binary file removed docs/book/.gitbook/assets/run-templates-run-1.png
Binary file not shown.
Binary file removed docs/book/.gitbook/assets/run-templates.gif
Binary file not shown.
Binary file added docs/book/.gitbook/assets/snapshot-run-dag.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
31 changes: 15 additions & 16 deletions docs/book/api-docs/pro-api/pro-api/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Whether you're using the [SaaS version](https://zenml.io/pro) or a self-hosted Z

ZenML Pro includes two distinct APIs:

1. **Workspace API**: This is similar to the [OSS API](../../oss-api/oss-api/) but includes additional endpoints like Run Templates. Each workspace in ZenML Pro has its own API.
1. **Workspace API**: This is similar to the [OSS API](../../oss-api/oss-api/) but includes additional endpoints (for example to run snapshots). Each workspace in ZenML Pro has its own API.
2. **ZenML Pro API**: This is the management API for ZenML Pro and handles organization-level resources like workspaces, users, and roles.

### Server URL
Expand Down Expand Up @@ -113,20 +113,20 @@ See the high‑level overview: [Connect to a server](https://docs.zenml.io/how-t

## Workspace API Authentication

The **Workspace API** is different from the ZenML Pro API and supports different authentication methods. This is the API you'll use for triggering run templates and other pipeline operations.
The **Workspace API** is different from the ZenML Pro API and supports different authentication methods. This is the API you'll use for running snapshots and other pipeline operations.

### When to Use Which API

| Task | Use This API | Authentication Method |
|------|-------------|----------------------|
| Managing organizations, workspaces, users | **ZenML Pro API** (cloudapi.zenml.io) | Short-lived API tokens from organization settings |
| Accessing the ZenML Pro API programmatically from external systems | **ZenML Pro API** (cloudapi.zenml.io) | Service accounts + API keys (recommended for automation) |
| Triggering run templates, pipeline operations | **Workspace API** (your workspace URL) | Service accounts + API keys (recommended for automation) |
| Running snapshots, pipeline operations | **Workspace API** (your workspace URL) | Service accounts + API keys (recommended for automation) |
| Pipeline development, artifact management | **Workspace API** (your workspace URL) | Temporary API tokens or service accounts |

### Workspace API Authentication Methods

Programmatic access to the ZenML Pro workspace API is achieved mostly the same way as the ZenML OSS server API. This is because the Workspace API in ZenML Pro is an extension of the OSS API with some additional endpoints like Run Templates. The only exception is that workspace level service accounts and API keys are disabled and organization level service accounts and API keys are used instead (see [Service Accounts](https://docs.zenml.io/pro/core-concepts/service-accounts)).
Programmatic access to the ZenML Pro workspace API is achieved mostly the same way as the ZenML OSS server API. This is because the Workspace API in ZenML Pro is an extension of the OSS API with some additional endpoints. The only exception is that workspace level service accounts and API keys are disabled and organization level service accounts and API keys are used instead (see [Service Accounts](https://docs.zenml.io/pro/core-concepts/service-accounts)).

You can use one of these two methods to authenticate with your workspace API:

Expand Down Expand Up @@ -177,20 +177,19 @@ Remember to refer to the complete API documentation available at [https://clouda

The Workspace API includes all OSS API endpoints plus some additional Pro-specific endpoints. These are available at your workspace URL at the `/docs` path (e.g., https://1bfe8d94-zenml.cloudinfra.zenml.io/docs):

#### Run Templates (Pro-specific)
#### Running snapshots (Pro-specific)

Run Templates allow you to trigger pipeline runs from external systems using HTTP requests.
Snapshots allow you to start pipeline runs from external systems using HTTP requests.

* List run templates: `GET /run_templates`
* Create a run template: `POST /run_templates`
* Get run template details: `GET /run_templates/{template_id}`
* Update a run template: `PATCH /run_templates/{template_id}`
* **Trigger a run template: `POST /run_templates/{template_id}/runs`**
* List snapshots: `GET /pipeline_snapshots`
* Get snapshot details: `GET /pipeline_snapshots/<SNAPSHOT-ID>`
* Update a snapshot: `PATCH /pipeline_snapshots/<SNAPSHOT-ID>`
* **Run a snapshot: `POST /pipeline_snapshots/<SNAPSHOT-ID>/runs`**

{% hint style="success" %}
**For Run Template Automation**
**For Snapshot Automation**

When triggering run templates from external systems or CI/CD pipelines, we strongly recommend using **service accounts with API keys** rather than temporary tokens. Service accounts provide:
When running snapshots from external systems or CI/CD pipelines, we strongly recommend using **service accounts with API keys** rather than temporary tokens. Service accounts provide:

* Long-term credentials that don't expire hourly
* Dedicated authentication for automation (not tied to individual users)
Expand All @@ -217,7 +216,7 @@ Use this decision tree to determine the right approach:

| **What do you want to do?** | **Which API?** | **Authentication Method** | **Why?** |
|------------------------------|----------------|---------------------------|----------|
| Trigger run templates from CI/CD or external systems | **Workspace API** (your workspace URL) | Service account + API key | Long-lived, automation-friendly, not tied to users |
| Run snapshots from CI/CD or external systems | **Workspace API** (your workspace URL) | Service account + API key | Long-lived, automation-friendly, not tied to users |
| Manage organizations, create workspaces, invite users | **ZenML Pro API** (cloudapi.zenml.io) | Short-lived API tokens from org settings | Administrative operations need user-level permissions |
| Access the ZenML Pro API programmatically from external systems | **ZenML Pro API** (cloudapi.zenml.io) | Service accounts + API keys (recommended for automation) | Long-lived, automation-friendly, not tied to users |
| Develop pipelines, explore artifacts | **Workspace API** (your workspace URL) | Temporary API tokens or service accounts | Flexible for development needs |
Expand All @@ -234,6 +233,6 @@ Use this decision tree to determine the right approach:
- ✅ Switch from temporary API tokens to service account API keys
- ✅ Service account tokens don't expire automatically like user tokens

**"I can't find the run template endpoints"**
- ✅ Run templates are only available on the **Workspace API**, not the ZenML Pro API
**"I can't find the endpoint to run a snapshot"**
- ✅ Running snapshots is only possible using the **Workspace API**, not the ZenML Pro API
- ✅ Use your workspace URL + `/docs` to see the full API documentation
4 changes: 4 additions & 0 deletions docs/book/getting-started/core-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,10 @@ ZenML comes with a _Docker deployer_ that can run deployments on your local mach

A **Pipeline Run** is a record of a pipeline execution. When you run a pipeline using an orchestrator, a pipeline run is created tracking information about the execution such as the status, the artifacts and metadata produced by the pipeline and all its steps. When a pipeline is deployed for online mode execution, a pipeline run is similarly created for every HTTP request made to it.

#### Pipeline Snapshot

A **Pipeline Snapshot** is an immutable snapshot of your pipeline that includes the pipeline DAG, code, configuration, and container images. Snapshots can be run from the server or dashboard, and can also be [deployed](#deployment).

#### Artifact Store

An **Artifact Store** is a component that houses all data that passes through the pipeline as inputs and outputs. Each artifact that gets stored in the artifact store is tracked and versioned and this allows for extremely useful features like data caching, which speeds up your workflows.
Expand Down
2 changes: 1 addition & 1 deletion docs/book/getting-started/hello-world.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,6 @@ To continue your ZenML journey, explore these key topics:
* **Containerization**: Understand how ZenML [handles containerization](../how-to/containerization/containerization.md) for reproducible execution
* **Stacks & Infrastructure**: Explore the concepts behind [stacks](../how-to/stack-components/stack_components.md) and [service connectors](../how-to/stack-components/service_connectors.md) for authentication
* **Secrets Management**: Learn how to [handle sensitive information](../how-to/secrets/secrets.md) securely
* **Templates**: Create [reusable pipeline templates](../how-to/templates/templates.md) for standardized workflows
* **Snapshots**: Create [reusable snapshots](../how-to/snapshots/snapshots.md) for standardized workflows
{% endstep %}
{% endstepper %}
2 changes: 1 addition & 1 deletion docs/book/getting-started/zenml-pro/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Deploy multiple ZenML servers through [workspaces](workspaces.md) with enterpris
### Advanced MLOps Capabilities
- Enhanced [Model Control Plane](https://docs.zenml.io/user-guides/starter-guide/track-ml-models) for ML asset tracking
- [Artifact Control Plane](https://docs.zenml.io/user-guides/starter-guide/manage-artifacts) for improved asset management
- [Run templates](https://docs.zenml.io/how-to/trigger-pipelines#run-templates) for quick pipeline iteration
- [Run Snapshots](https://docs.zenml.io/concepts/snapshots) for quick pipeline iteration
- Early access to pro-specific features like triggers, filters, and usage reports

{% hint style="info" %}
Expand Down
1 change: 0 additions & 1 deletion docs/book/getting-started/zenml-pro/hierarchy.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ The image above shows the hierarchy of concepts in ZenML Pro.
* [**Teams**](teams.md) are groups of users within an organization. They help in organizing users and managing access to resources at organization, workspace, and project levels.
* **Users** are single individual accounts on a ZenML Pro instance.
* [**Roles**](roles.md) exist at organization, workspace, and project levels to control what actions users can perform.
* [**Templates**](https://docs.zenml.io/how-to/trigger-pipelines) are pipeline runs that can be re-run with a different configuration.

More details about each of these concepts are available in their linked pages below:

Expand Down
2 changes: 1 addition & 1 deletion docs/book/getting-started/zenml-pro/projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Projects provide isolation for various MLOps resources:

* Pipelines created within a project are only visible to project members
* Pipeline runs and their artifacts are scoped to the project
* Pipeline configurations and templates are project-specific
* Pipeline configurations and snapshots are project-specific

### Artifacts and Models

Expand Down
24 changes: 12 additions & 12 deletions docs/book/getting-started/zenml-pro/self-hosted.md
Original file line number Diff line number Diff line change
Expand Up @@ -1497,19 +1497,19 @@ export ZENML_PRO_API_URL=https://zenml-pro.staging.cloudinfra.zenml.io/api/v1
zenml login
```

## Enabling Run Templates Support
## Enabling Snapshot Support

The ZenML Pro workspace server can be configured to optionally support Run Templates - the ability to run pipelines straight from the dashboard. This feature is not enabled by default and needs a few additional steps to be set up.
The ZenML Pro workspace server can be configured to optionally support running pipeline snapshots straight from the dashboard. This feature is not enabled by default and needs a few additional steps to be set up.

{% hint style="warning" %}
The Run Templates feature is only available from ZenML workspace server version 0.81.0 onwards.
Snapshots are only available from ZenML workspace server version 0.90.0 onwards.
{% endhint %}

The Run Templates feature comes with some optional sub-features that can be turned on or off to customize the behavior of the feature:
Snapshots come with some optional sub-features that can be turned on or off to customize the behavior of the feature:

* **Building runner container images**: Running pipelines from the dashboard relies on Kubernetes jobs (aka "runner" jobs) that are triggered by the ZenML workspace server. These jobs need to use container images that have the correct Python software packages installed on them to be able to launch the pipelines.

The good news is that run templates are based on pipeline runs that have already run in the past and already have container images built and associated with them. The same container images can be reused by the ZenML workspace server for the "runner jobs". However, for this to work, the Kubernetes cluster itself has to be able to access the container registries where these images are stored. This can be achieved in several ways:
The good news is that snapshots are based on pipeline runs that have already run in the past and already have container images built and associated with them. The same container images can be reused by the ZenML workspace server for the "runner jobs". However, for this to work, the Kubernetes cluster itself has to be able to access the container registries where these images are stored. This can be achieved in several ways:

* use implicit workload identity access to the container registry - available in most cloud providers by granting the Kubernetes service account access to the container registry
* configure a service account with implicit access to the container registry - associating some cloud service identity (e.g. a GCP service account, an AWS IAM role, etc.) with the Kubernetes service account used by the "runner" jobs
Expand All @@ -1523,19 +1523,19 @@ The Run Templates feature comes with some optional sub-features that can be turn

To avoid this, you can configure the ZenML workspace server to store the logs in an external location, like an S3 bucket. This can be achieved by setting the `ZENML_KUBERNETES_WORKLOAD_MANAGER_ENABLE_EXTERNAL_LOGS` environment variable to `true`.

This option is only currently available with the AWS implementation of the Run Templates feature and also requires the `ZENML_AWS_KUBERNETES_WORKLOAD_MANAGER_BUCKET` environment variable to be set to point to the S3 bucket where the logs will be stored.
This option is only currently available with the AWS implementation of the snapshots feature and also requires the `ZENML_AWS_KUBERNETES_WORKLOAD_MANAGER_BUCKET` environment variable to be set to point to the S3 bucket where the logs will be stored.

1. Decide on an implementation.

There are currently three different implementations of the Run Templates feature:
There are currently three different implementations of the snapshots feature:

* **Kubernetes**: runs pipelines in the same Kubernetes cluster as the ZenML Pro workspace server.
* **AWS**: extends the Kubernetes implementation to be able to build and push container images to AWS ECR and to store run the template logs in AWS S3.
* **GCP**: currently, this is the same as the Kubernetes implementation, but we plan to extend it to be able to push container images to GCP GCR and to store run template logs in GCP GCS.

If you're going for a fast, minimalistic setup, you should go for the Kubernetes implementation. If you want a complete cloud provider solution with all features enabled, you should go for the AWS implementation.

2. Prepare Run Templates configuration.
2. Prepare Snapshots configuration.

You'll need to prepare a list of environment variables that will be added to the Helm chart values used to deploy the ZenML workspace server.

Expand All @@ -1545,8 +1545,8 @@ The Run Templates feature comes with some optional sub-features that can be turn
* `zenml_cloud_plugins.kubernetes_workload_manager.KubernetesWorkloadManager`
* `zenml_cloud_plugins.aws_kubernetes_workload_manager.AWSKubernetesWorkloadManager`
* `zenml_cloud_plugins.gcp_kubernetes_workload_manager.GCPKubernetesWorkloadManager`
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_NAMESPACE` (mandatory): the Kubernetes namespace where the "runner" jobs will be launched. It must exist before the run templates are enabled.
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_SERVICE_ACCOUNT` (mandatory): the Kubernetes service account to use for the "runner" jobs. It must exist before the run templates are enabled.
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_NAMESPACE` (mandatory): the Kubernetes namespace where the "runner" jobs will be launched. It must exist before the snapshots are enabled.
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_SERVICE_ACCOUNT` (mandatory): the Kubernetes service account to use for the "runner" jobs. It must exist before the snapshots are enabled.
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_BUILD_RUNNER_IMAGE` (optional): whether to build the "runner" container images or not. Defaults to `false`.
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_DOCKER_REGISTRY` (optional): the container registry where the "runner" images will be pushed. Mandatory if `ZENML_KUBERNETES_WORKLOAD_MANAGER_BUILD_RUNNER_IMAGE` is set to `true`, ignored otherwise.
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_RUNNER_IMAGE` (optional): the "runner" container image to use. Only used if `ZENML_KUBERNETES_WORKLOAD_MANAGER_BUILD_RUNNER_IMAGE` is set to `false`, ignored otherwise.
Expand All @@ -1557,7 +1557,7 @@ The Run Templates feature comes with some optional sub-features that can be turn
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_TOLERATIONS` (optional): the Kubernetes tolerations to use for the "runner" jobs, in JSON format. Example: `[{"key": "node-pool", "operator": "Equal", "value": "zenml-pool", "effect": "NoSchedule"}]`.
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_JOB_BACKOFF_LIMIT` (optional): the Kubernetes backoff limit to use for the builder and runner jobs.
* `ZENML_KUBERNETES_WORKLOAD_MANAGER_POD_FAILURE_POLICY` (optional): the Kubernetes pod failure policy to use for the builder and runner jobs.
* `ZENML_SERVER_MAX_CONCURRENT_TEMPLATE_RUNS` (optional): the maximum number of concurrent run templates that can be started at the same time by each server container or pod. Defaults to 2. If a client exceeds this number, the request will be rejected with a 429 Too Many Requests HTTP error. Note that this only limits the number of parallel run templates that can be *started* at the same time, not the number of parallel pipeline runs.
* `ZENML_SERVER_MAX_CONCURRENT_TEMPLATE_RUNS` (optional): the maximum number of concurrent snapshot runs that can be started at the same time by each server container or pod. Defaults to 2. If a client exceeds this number, the request will be rejected with a 429 Too Many Requests HTTP error. Note that this only limits the number of parallel snapshots that can be *started* at the same time, not the number of parallel pipeline runs.

For the AWS implementation, the following additional variables are supported:

Expand Down Expand Up @@ -1613,7 +1613,7 @@ The Run Templates feature comes with some optional sub-features that can be turn
ZENML_KUBERNETES_WORKLOAD_MANAGER_NAMESPACE: zenml-workspace-namespace
ZENML_KUBERNETES_WORKLOAD_MANAGER_SERVICE_ACCOUNT: zenml-workspace-service-account
ZENML_KUBERNETES_WORKLOAD_MANAGER_BUILD_RUNNER_IMAGE: "true"
ZENML_KUBERNETES_WORKLOAD_MANAGER_DOCKER_REGISTRY: europe-west3-docker.pkg.dev/zenml-project/zenml-run-templates/zenml
ZENML_KUBERNETES_WORKLOAD_MANAGER_DOCKER_REGISTRY: europe-west3-docker.pkg.dev/zenml-project/zenml-snapshots/zenml
ZENML_KUBERNETES_WORKLOAD_MANAGER_POD_RESOURCES: '{"requests": {"cpu": "100m", "memory": "400Mi"}, "limits": {"memory": "700Mi"}}'
ZENML_KUBERNETES_WORKLOAD_MANAGER_NODE_SELECTOR: '{"node-pool": "zenml-pool"}'
ZENML_KUBERNETES_WORKLOAD_MANAGER_TOLERATIONS: '[{"key": "node-pool", "operator": "Equal", "value": "zenml-pool", "effect": "NoSchedule"}]'
Expand Down
4 changes: 2 additions & 2 deletions docs/book/getting-started/zenml-pro/workspaces.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,8 +168,8 @@ Some Pro-only features that you can leverage in your workspace are as follows:
* [Projects for Resource Organization](projects.md)
* [Model Control Plane](https://docs.zenml.io/how-to/model-management-metrics/model-control-plane/register-a-model)
* [Artifact Control Plane](https://docs.zenml.io/how-to/data-artifact-management/handle-data-artifacts)
* [Ability to run pipelines from the Dashboard](https://docs.zenml.io/how-to/trigger-pipelines/use-templates-rest-api),
* [Create templates out of your pipeline runs](https://docs.zenml.io/how-to/trigger-pipelines/use-templates-rest-api)
* [Create snapshots out of your pipeline runs](https://docs.zenml.io/concepts/snapshots#using-the-dashboard)
* [Run snapshots from the Dashboard](https://docs.zenml.io/concepts/snapshots#using-the-dashboard-1)

and [more](https://zenml.io/pro)!

Expand Down
Loading
Loading