Deployments

Prefector registers Prefect deployments from YAML spec files, resolving Docker image names from a separate manifest. It validates flow parameters before registering and supports selective deployment by name.

Commands

# List deployment specs found in the directory
prefector deployments list --deployments-dir path/to/specs

# Deploy all deployments
prefector deployments deploy \
  --deployments-dir path/to/specs \
  --images-manifest path/to/images.yaml \
  --image-prefix ghcr.io/example \
  --api-url "$PREFECT_API_URL"

# Preview without making changes
prefector deployments deploy \
  --deployments-dir path/to/specs \
  --images-manifest path/to/images.yaml \
  --image-prefix ghcr.io/example \
  --api-url "$PREFECT_API_URL" \
  --dry-run

# Deploy specific deployments only
prefector deployments deploy ... --target my-deployment --target other-deployment

Deployment spec files

Each deployment is a YAML file in --deployments-dir. The three required fields are name, flow, and image_key. All other fields are optional.

name: my-deployment
flow: flows.my_module:my_flow    # <module>:<function> format
image_key: flow_runtime          # key from the images manifest

cron: "0 6 * * *"               # standard cron expression
tags:
  - project_name
  - bronze
parameters:
  retries: 3
env:
  ENVIRONMENT: ${ENVIRONMENT}    # resolved from the environment at deploy time
  LOG_LEVEL: INFO

flow

The module path and function name in <module>:<function> format. The module must be importable from the current Python environment and the function must exist in that module.

flow: flows.ingestion.redcap:run_ingestion

image_key

Refers to an entry in the images manifest (see below). Prefector looks up the image name from the manifest and combines it with --image-prefix and --image-tag to form the full Docker image reference used for the deployment.

parameters

Passed to Prefect as deployment default parameters. Prefector validates them against the flow's parameter schema before registering, and exits with a clear error if any are invalid.

parameters:
  retries: 3
  notify_on_failure: true

env

Key/value pairs set as job_variables.env on the deployment. Values support ${VAR_NAME} substitution (see below). These are stored in Prefect and visible in the UI — avoid storing secrets here; use Prefect blocks instead.

cron

A schedule on which to run the flow, in the standard cron syntax.

cron: "0 8 * * 1-5"   # 08:00 Monday–Friday

Environment variable substitution

Values in the form ${VAR_NAME} in the deployment YAML are replaced with the corresponding environment variable when the spec is loaded. This happens at deploy time (e.g. in CI), not at flow run time.

env:
  COMMIT_SHA: ${CI_COMMIT_SHORT_SHA}
  ENVIRONMENT: ${ENVIRONMENT}

Only ${VAR} brace syntax is supported. Bare $VAR is left unchanged. Substitution occurs on raw text before YAML parsing — if a variable value contains YAML special characters (:, {, }, #) quote the value:

env:
  LABEL: "${MY_LABEL}"

Substitution applies only to deployments that are actually being deployed. Untargeted deployments (filtered with --target) and the list command do not require variables to be set.

Images manifest

The images manifest is a YAML file that maps short image keys to fully qualified image names. This decouples deployment specs from the registry and image naming conventions used by the CI pipeline that builds images.

# images.yaml
- key: flow_runtime
  dockerfile: Dockerfile
  name: my-registry/my-project-flows
- key: dbt
  name: my-registry/my-project-dbt

The key must match the image_key in each deployment spec. The name is combined with --image-prefix and --image-tag:

--image-prefix	name from manifest	--image-tag	Resulting image
ghcr.io/acme	my-flows	2024.06.01	ghcr.io/acme/my-flows:2024.06.01
(empty)	my-flows	latest	my-flows:latest
(empty)	my-registry/my-flows:fixed	(any)	my-registry/my-flows:fixed

If the image name already contains a tag (: after the last /), the --image-tag is ignored.

Deploy options

Option	Default	Description
--deployments-dir	.	Directory containing deployment YAML files
--images-manifest	(required)	Path to the images manifest YAML
--image-prefix	(required)	Registry prefix prepended to image names
--image-tag	latest	Image tag applied unless the manifest name already contains one
--work-pool	default	Prefect work pool name
--work-queue	default	Prefect work queue name
--target	(all)	Deploy only this deployment; repeatable
--dry-run	false	Validate and print without registering