RFC: Conda Package Workflow and Ownership for AYON on Deadline Cloud

[leon-li-inspire]

RFC: Conda Package Workflow and Ownership for AYON on Deadline Cloud

---

Problem

The PUBLISH step on Deadline Cloud Service-Managed Fleet (SMF) workers requires AYON libraries (ayon-core, ayon-python-api, pyblish-base, ayon-deadline-cloud addon) to run. Without them, SMF workers fail with ayon: command not found.

Two distribution options were considered (analysis):

1. Ship the full AYON Launcher — handles version resolution automatically, but is heavyweight (~hundreds of MB), self-updates at runtime (determinism risk across workers), and conflicts with conda's immutability model.
2. Ship a subset of AYON libraries as a conda package — lightweight, version-pinned at build time, deterministic, conda-native.

We propose option 2: a headless conda package containing only the libraries needed for publishing.

---

Proposal

What ships to workers

A single noarch: python conda package (~10 MB) containing:

Component	Why it's needed
ayon-core	Core pipeline logic — provides the publish pipeline, addon discovery, settings resolution, and the ayon CLI. Only the headless client code is included (no GUI/Qt).
ayon-python-api	Python client for the AYON server REST API — used to register versions, query project structure, and authenticate workers.
pyblish-base	The publish framework that orchestrates Collect → Validate → Extract → Integrate phases. All AYON publishing is built on pyblish plugins.
clique, nxtools, and other deps	Utilities for file sequence detection (e.g., grouping frame.0001.exr ... frame.0100.exr), path handling, and media metadata.
ayon-deadline-cloud addon	The Deadline Cloud-specific publish plugins — defines what happens during the PUBLISH step (discovers outputs, applies path mapping, registers with AYON server).

No launcher GUI, no self-update, no cx_Freeze bundle. Version pinned at build time.

Recipe location

The recipe lives in ynput/ayon-core. This provides:

• Natural version coupling — the recipe versions alongside the primary library it packages
• YNPUT has full ownership of the recipe, build process, and release cadence
• No external PR review dependency for urgent fixes

Channel hosting

YNPUT owns and operates the public conda channel. This is an S3-backed channel (e.g., s3://ynput-conda-channel/ayon/) that customers add to their queue's CondaChannels parameter.

Rationale:

• YNPUT controls release cadence, versioning, and compatibility testing
• Decouples AYON releases from AWS Deadline Cloud release cycles
• YNPUT already owns the AYON server, addons, and client code — the channel is the natural extension
• Customers configure access via standard S3 bucket policies (public read, or scoped IAM)

Not in the deadline-cloud channel — that channel is AWS-managed and tied to AWS release processes. AYON's release cadence is independent.

---

Workflow

Build & publish (YNPUT responsibility)

1. YNPUT releases a new version of ayon-core, ayon-python-api, or ayon-deadline-cloud
2. YNPUT builds a new ayon-publish package version (rattler-build)
3. YNPUT uploads to their S3 conda channel
4. YNPUT reindexes the channel (rattler-index)

The channel accumulates all published versions — older versions remain available:

noarch/
  ayon-publish-1.9.0-pyh4616a5c_0.conda
  ayon-publish-1.9.5-pyh4616a5c_0.conda
  ayon-publish-2.0.0-pyh4616a5c_0.conda

Version pinning

Each ayon-publish package version bundles a fixed set of AYON libraries. Customers choose which version to install at submission time:

• ayon-publish → latest available
• ayon-publish=1.9.0 → exact version matching their AYON server/bundle

The AYON addon automatically pins the version at submission based on the studio's active bundle configuration. This ensures every worker in a job runs the same code, regardless of when it starts.

Customer setup (one-time)

1. Add YNPUT's conda channel URL to queue's CondaChannels parameter
   (e.g., "conda-forge s3://ynput-conda-channel/ayon deadline-cloud")
2. Ensure queue IAM policy allows s3:GetObject on the channel bucket
3. Submit jobs — the AYON addon automatically adds the correct ayon-publish version

Job execution

Artist submits job via AYON
  → AYON addon adds publish step with ayon-publish=X.Y.Z in CondaPackages
  → Render step runs on SMF worker (attr.role: render)
  → Publish step runs on SMF worker (attr.role: publish)
    → conda installs ayon-publish=X.Y.Z from YNPUT channel
    → ayon addon deadline_cloud publish runs headless
    → Registers version metadata with AYON server

---

Key Design Decisions

noarch: python — cross-platform by default

The package is noarch: python (platform-independent, pure Python). A single package works on both Linux and Windows workers — conda handles platform-specific file placement (entry points go to bin/ on Linux, Scripts/ on Windows). No separate builds needed per platform.

Requires conda-forge in the queue's channel list to provide a standalone python package for the solver and linker — matching the pattern used by all official Deadline Cloud sample recipes.

Addon pinning strategy

Three approaches for how addons and libraries ship:

	A. Single fat package (MVP)	B. Separate conda packages per component	C. Plugin sync from S3
When addon version is decided	At package build time	At job submission (via CondaPackages spec)	At upload time (studio uploads to S3)
What triggers an update	YNPUT rebuilds and publishes a new package	YNPUT publishes updated individual package	Studio uploads new addon files to S3 bucket
Package rebuild needed for addon change?	Yes — full package	Only the changed component	No — addon lives in S3, not in any conda package
Worker needs AYON server at install time?	No — everything from conda channel	No — everything from conda channel	No — downloads from S3 job attachments bucket
Determinism	Fully deterministic — package is immutable	Deterministic — versions pinned via CondaPackages	Deterministic per S3 path — studio controls what's uploaded
Studio flexibility	Must wait for new package publish	Mix and match versions freely (e.g., ayon-core=1.9.0 ayon-deadline-cloud=0.2.0)	Full control — studio uploads any addon version to S3 at any time
Complexity	Simple — one artifact	Medium — each component needs its own recipe, CI, and compatibility testing	Medium — base conda package + activation script + S3 upload convention
Testing confidence	What's in the package is what was tested together	Studio takes responsibility for compatible combinations	Studio takes responsibility for compatible combinations
Example CondaPackages	ayon-publish=1.9.0	ayon-core=1.9.0 ayon-python-api=1.2.19 ayon-deadline-cloud=0.2.0	ayon-publish-core=1.9.0 (addons synced from S3)

Option C detail: Uses the same plugin sync pattern that Deadline Cloud DCC packages use for Maya/Blender plugins. The conda package includes a base (ayon-core + ayon-python-api + deps) plus an activation script that downloads addons from the queue's job attachments S3 bucket at session start:

s3://<job-attachments-bucket>/<root-prefix>/plugins/<os>/ayon/<version>/
    ayon_deadline_cloud/
    other_addons/

The activation script (runs when conda env activates) downloads these to the session directory and adds them to PYTHONPATH. Studios manage addon versions by uploading to S3 — no conda rebuild, no AYON server dependency.

MVP: Option A (single fat package). Simple, deterministic, no coordination overhead.

Option B becomes viable when: YNPUT has stable CI for each component and studios need to independently version addons without server-side resolution.

Option C becomes viable when: Studios want full control over addon versions without waiting for any package publish, and want to use the same plugin distribution pattern they already use for DCC plugins.

Version compatibility

The ayon-publish package version must be compatible with the studio's AYON server version. A mismatch causes failures:

Mismatch	Risk
ayon-python-api vs server	API contract breaks — REST calls fail or return unexpected data
ayon-core vs server	Settings schema drift — publish pipeline uses wrong defaults or missing config
ayon-deadline-cloud vs ayon-core	Internal API changes — import errors or changed behavior

AYON manages compatibility through bundles — a bundle declares which versions of server + addons are tested together. Each ayon-publish package version corresponds to a tested bundle combination.

Mitigation: The AYON addon at submission time automatically selects the package version matching the studio's active bundle. If a studio upgrades their AYON server, they must also ensure a compatible ayon-publish version is available in the channel. YNPUT is responsible for publishing a package version for each supported bundle.

---

Open Questions

1. Public channel access model — Should the YNPUT channel be fully public (anonymous read) or require customer-specific IAM credentials? Public is simpler; credentialed allows YNPUT to track adoption.

2. Version compatibility matrix — How do we document which ayon-publish version works with which AYON server version? A compatibility table in the repo README, or enforced via the AYON server's bundle configuration?

[antirotor]

Thanks for the analysis. definitely something to think about. It goes a little bit against some core concepts of AYON, but maybe we can distill something from it.

The biggest issue is that AYON is designed to handle versions and dependencies of its components in very flexible manner. This "packaging" works like this:

Bundles are the core concept - they group together AYON addons versions, AYON launcher and dependency package.

---

Launcher

Launcher is important in this case not just for bootstrapping but it also provides basic dependencies - like specific version of ayon-python-api, Qt and python, etc.

Addon versions

Settings in AYON are bound to addon versions (and variants production, staging and dev). What this implies is that bundles with different addon versions can have different settings. This is important for feature called Project Bundles where you associate specific bundle with specific project (therefore "freezing" the settings for that project)

Dependency package

Is a collection of all python dependencies specified by individual addon versions with bundle minus the ones defined by AYON launcher.

---

This allows you to run AYON with various configurations and environments but also complicates versioning externally.

How this works with farm schedulers:

1. during the job submission, the bundle name is passed as a part of the the job.
2. on the worker machine, AYON launcher uses the bundle name to get specific addon versions, their settings and python dependencies to re-create the exact environment used to submit the job.

Another point to consider are Variants - bundle variants can be used to switch between addon settings so you can test various addon versions and their settings before you run them in production. This is an important aspect - it allows you to submit the jobs in staging environment.

All of it is difficult to convert to a single conda package version thanks to the mentioned package immutability. You would need to either completely replace the logic of how versioning works in AYON or seriously limit these features.

What could work:
Instead of packaging the whole launcher, ayon-core and dependencies, having single, very simple python-only conda package that would:

• connect to the specified AYON server
• resolve all that is needed - addons / launcher version / dependency package
• download and cache all that is needed on s3 bucket that would be available for all workers
• execute AYON Launcher to process the job

This will solve consistent environment, bootstrapping of workers, and it will minimize redownloading of larger data.

Now I am not that versed in AWS ecosystem so there might be flaws I am not seeing - the cache persistance or maybe associated cost, or some other difficulities. What do you think @leon-li-inspire and @oehmends

[antirotor]

just to add to it - this proposed package would be owned by YNPUT and be available in conda channel. It would be specified during the job submission along with the additional conda channels.

[oehmends]

Outcome of the 9 June dev deep dive — where we landed

Following up on this RFC and @antirotor's reply. On the call (@leon-li-inspire, @antirotor and me) we aligned on an approach to take forward as a PoC and validate for viability. Two parts, and they're independent of each other.

1. Getting AYON + the bundle onto the worker

• AYON Launcher → Conda package (@leon-li-inspire). It's an immutable, rarely-customized release, so it packages cleanly. ~800 MB–1 GB by platform.
• Per-studio bundle via job attachments (@antirotor building the hook here). The bundle (add-ons + deps, a few hundred MB) ships as job-attachment inputs from a pre-hook that reuses the same AYON functions the Launcher already uses to resolve and fetch a bundle. First submit uploads; later submits hit the content-hash cache, so they're fast. Env vars point the Launcher at the cached files wherever they live (S3, fileshare, mount).
• Conda as the standard environment for both SMF and CMF, so we don't maintain two setups.

This lines up with @antirotor's proposal — a lightweight package that resolves the bundle, caches what workers need, and executes the Launcher. The open part is to build it and confirm it holds up in practice.

2. Reaching the AYON server

Separate from distributing the attachments, and it varies by where the AYON server lives (on-prem, AYON-managed cloud, or self-hosted EC2 → VPN / VPC Lattice / SSM). I'll map the connection scenarios per studio setup as a mermaid diagram for the next call, building on the distribution analysis doc: launcher-distribution.md. One constraint to keep in mind: hybrid SMF/CMF works, but only with synchronized storage — otherwise on-prem workers pull the whole payload from cloud S3.

Next steps

• @leon-li-inspire — Launcher Conda package.
• @antirotor — job-attachment hook (PR here).
• Me — connectivity scenarios as a mermaid diagram for Monday.

Next call is Monday. Shout on Discord in the meantime if anything's unclear.

[oehmends]

Design doc updated ahead of Monday — the connectivity scenarios are in as a mermaid diagram, and I reconciled the runtime/distribution section with Leon's Launcher conda recipe.

In deadline-cloud-integration/design.md on feature/mvp-integration-design:

• AYON Server Connectivity (new section) — decision flowchart + scenario table: on-prem → VPN / VPC Lattice, AYON-managed cloud → public HTTPS (+ on-prem fileshare three-way), self-hosted EC2 → SSM, on-prem CMF → direct. Plus a sequence diagram of the SMF→on-prem PoC (VPC Lattice resource endpoint → reverse-proxy EC2 → outbound reverse SSH tunnel; no account IDs/IPs).
• Runtime and Bundle Distribution (new section) — full AYON Launcher conda package run headless on SMF + CMF (matching @leon-li-inspire's deadline-cloud-samples#237), per-studio bundle via the content-hash-cached job-attachment hook, conda standard on both fleet types.
• Publish I/O — RENDER step outputs flow to the dependent PUBLISH step as session inputs via the job template, so no deadline queue sync-output cron / automatic-download setup is needed.

Feedback welcome before the call.