feat: add CRE workflow deploy changeset and operation#1
Conversation
In order to deploy with the CRE CLI Runner, we need to ensure we have a `project.yaml`, `workflow.yaml`, and `context.yaml` files available in the working dir. All of the information to build these files is already available either via the `domain.yaml` or the durable pipelines inputs for the deploy changeset. This PR adds some helpers to easily generate these files at runtime so that the CRE CLI has everything needed to deploy a workflow succesfully. * Adds `workflow.yaml` go types to unmarshal to yaml. * Similarly, adds types and helper functions to generate the `project.yaml` and `context.yaml` * Adds a function to the CLIRunner to return the domain defined registries. This is something we need to expose at the changeset level to allow users to select the correct registry when deploying. I considered putting it directly on the `Environment` object instead of adding a new function to the CLI runner interface, but I decided to keep it on the CLIRunner because this is only needed for the CLI implementation and will probably be removed once we have the go lib implementation available. * Refactor the CLIRunner moving it into the `cre/cli` pkg so we can encapsulate most of the cli related logic. It will be easier to remove once we start adding the go based runner. A usage example can be found in this WIP PR, where we add the deploy workflow operation using all of these helpers: smartcontractkit/cld-changesets#1 ## AI Summary This pull request introduces a wrapper object for the Chainlink Resource Engine (CRE) to support both CLI and Go API implementations, and adds helpers for generating configuration files for the CRE CLI runner. It also refactors and extends the configuration and artifact types, removes legacy defaulting logic, and introduces comprehensive tests for new helpers. **Major enhancements for CRE CLI support:** * Added a wrapper object for CRE to allow CLI and Go API implementations to coexist. (.changeset/large-ads-clean.md) * Introduced helpers for generating `workflow.yaml`, `project.yaml`, and `context.yaml` for the CRE CLI runner, including their corresponding Go types and YAML serialization logic in `cre/cli/config.go`, with full test coverage. (.changeset/nasty-symbols-crash.md, cre/cli/config.go, cre/cli/config_test.go) [[1]](diffhunk://#diff-f341d61fe8536a1fb1c83f1e0c63f03f6d9cf0f8bc86729ecf20081907d0b34bR1-R5) [[2]](diffhunk://#diff-39d17f1dab7080012886f3c7c1f10da5bdc255bb277791d42ef1383f065c9bb6R1-R78) [[3]](diffhunk://#diff-dd896cc82b2be9ac093e78b8e3bfb66d4e897551647f180cf8c75508c51daf76R1-R101) **Configuration and artifact type improvements:** * Updated `WorkflowBundle` in `cre/artifacts/types.go` to include a `DeploymentRegistry` field and improved validation logic. Also removed the `ApplyDeployDefaults` method and its tests, simplifying defaulting behavior. (cre/artifacts/types.go, cre/artifacts/types_test.go) [[1]](diffhunk://#diff-b86bb22debe274e65ec385e65aeb597a64aaa5435f06b9f867ae08c8f65995b9L8-R16) [[2]](diffhunk://#diff-b86bb22debe274e65ec385e65aeb597a64aaa5435f06b9f867ae08c8f65995b9R25) [[3]](diffhunk://#diff-b86bb22debe274e65ec385e65aeb597a64aaa5435f06b9f867ae08c8f65995b9L39-L55) [[4]](diffhunk://#diff-d6f477315cd2e4e0c8e1aafe42c61379ae97d611755eae25df9545533bb7bcc3L7-L8) [[5]](diffhunk://#diff-d6f477315cd2e4e0c8e1aafe42c61379ae97d611755eae25df9545533bb7bcc3L453-L526) **Context and environment file helpers:** * Added helpers to build and write CRE CLI `.env` files and context YAML files, including logic to merge domain defaults and user overrides, and utility functions for registry handling in `cre/cli/context.go`, `cre/cli/env.go`, with comprehensive tests. (cre/cli/context.go, cre/cli/context_test.go, cre/cli/env.go, cre/cli/env_test.go) [[1]](diffhunk://#diff-8434e1bb9b502271888175d3ad0a08587c065bbdbfdb2ce2ab3558fbef0eb2ebR1-R76) [[2]](diffhunk://#diff-25aa911dde41faa825b8eef4f745b1a23b29ab0bf16c62e07fc9b6bda554ecbcR1-R174) [[3]](diffhunk://#diff-01b0bc93fb21d7f880165f0b4f3c61ea226c5f8e7c1d62de849050bacc7e5db2R1-R48) [[4]](diffhunk://#diff-32d9eb94ab70e98405b64ea2d05251fb78481cf17120beee1277c694296daabdR1-R85) **Mockery configuration update:** * Updated `.mockery.yml` to generate mocks for the `CLIRunner` interface in the new CLI package location. (.mockery.yml) **Test file organization:** * Renamed and updated the exit error test to reside in the CLI package, ensuring correct package usage and import paths. (cre/cli/result_test.go, previously cre/exit_test.go) --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a new cre package providing an example changeset + operation for deploying a CRE workflow via the CRE CLI, including artifact resolution and operation execution through the deployments framework.
Changes:
- Introduces
CREWorkflowDeployOpto assemble workflow artifacts/config files and invokecre workflow deploy. - Adds
CREWorkflowDeployChangesetto validate inputs/environment and execute the operation. - Updates module dependencies and README documentation for the new
crepackage.
Reviewed changes
Copilot reviewed 6 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
README.md |
Documents the new cre package, its main changeset/operation, and layout. |
go.mod |
Adds direct + indirect dependencies required for CRE deploy integration and tests. |
go.sum |
Adds checksums for newly introduced dependencies. |
cre/operations/workflow_deploy.go |
Implements the CRE workflow deploy operation, including context/workflow config generation and CLI invocation. |
cre/operations/workflow_deploy_test.go |
Adds unit tests for the operation and arg construction. |
cre/changesets/workflow_deploy.go |
Implements the changeset wrapper around the operation with precondition checks + env loading. |
cre/changesets/workflow_deploy_test.go |
Adds unit tests for changeset precondition validation. |
.gitignore |
Ignores .idea/ directory. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 8 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 8 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
I think you should add |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 10 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ring instead of key value pairs.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 10 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Thanks @gustavogama-cll, I've addressed most of the comments now |
graham-chainlink
left a comment
There was a problem hiding this comment.
Not sure if it is worth extending the test engine to support testing with CRE
https://docs.cld.cldev.sh/guides/changesets/testing-changesets#run-a-single-changeset
Currently if users want to test a changeset that uses cre with test engine, it wont work, we might have to supply like a mock cre object. Something to think about too
Yeah it's a good point, it might be something we'll need to add. Although I'd prefer to add these utils for the graph ql go client so we are working on top of something more stable longterm. |
🤖 I have created a release *beep* *boop* --- ## [0.1.0](cld-changesets-v0.0.1...cld-changesets-v0.1.0) (2026-05-06) ### Features * add "catalog create chain metadata" changeset ([#36](#36)) ([c12bb51](c12bb51)) * add catalog create contract metadata changeset ([#27](#27)) ([8ec1cbc](8ec1cbc)) * add CRE workflow deploy changeset and operation ([#1](#1)) ([960fde1](960fde1)) * add target name param to cre deploy changeset ([#3](#3)) ([8b25662](8b25662)) * fund mcms pdas ([#22](#22)) ([821e542](821e542)) * **jobspec:** port jobspec changesets from chainlink ([#21](#21)) ([0ce3f74](0ce3f74)) * link token ([#30](#30)) ([07345c1](07345c1)) * **pkg:** add contract constants and Solana MCMS state loading ([#5](#5)) ([6a2bbee](6a2bbee)) * port BuildProposalFromBatchesV2 ([#24](#24)) ([28d53d7](28d53d7)) * port run changeset from chainlink ([#32](#32)) ([693922f](693922f)) * port solana grant role ([#33](#33)) ([7eca5be](7eca5be)) * port token approve ([#40](#40)) ([04130de](04130de)) * **port:** firedrill mcms with operations api refactor ([#25](#25)) ([6d9010c](6d9010c)) * **solana:** add SOL funding helpers for deployer transfers ([#19](#19)) ([63bebf8](63bebf8)) * **solana:** port over mcms pda loader ([#11](#11)) ([7170ddc](7170ddc)) ### Bug Fixes * **aptos:** move state load ([#8](#8)) ([de58102](de58102)) * **evm:** port state load evm funcs ([#9](#9)) ([99279f1](99279f1)) * **state:** restore addressbook usage ([#17](#17)) ([017cd5e](017cd5e)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: app-token-issuer-ops-platform[bot] <275822481+app-token-issuer-ops-platform[bot]@users.noreply.github.com> Co-authored-by: James Kong <61834+jkongie@users.noreply.github.com>
Adds 1 operation and 1 changeset to deploy a CRE workflow. These serve as examples for basic workflow deployment using the CRE CLI. Operation can be composed to create more complex changesets or sequences. The operation will:
project.yamllocationcontext.yamlfile with the provided workflow registries. (It's possible we can drop this when we have the new version of the CLI with api key support, for now we are leaving it as it does not affect the deployment process)CLIRunnerprovided by CLDF.AI Summary
This pull request introduces a new CRE workflow deploy changeset and operation, enabling automated deployment of CRE workflows via the CLI. It includes robust input validation, integration with the deployment framework, and comprehensive unit tests for both the changeset and operation. The documentation is also updated to describe the new package and its structure.
CRE Workflow Deploy Feature
CREWorkflowDeployChangesetincre/changesets/workflow_deploy.goto resolve workflow artifacts, validate inputs, and trigger the CRE CLI workflow deploy operation. This includes precondition checks for environment and input validity.CREWorkflowDeployOpincre/operations/workflow_deploy.go, which prepares workflow artifacts, builds required config files, and invokes the CRE CLI with appropriate arguments and environment. Handles result reporting and error propagation.Testing
cre/changesets/workflow_deploy_test.go, covering various precondition scenarios and input validation.cre/operations/workflow_deploy_test.go, including CLI invocation, argument construction, error handling, and output propagation.Documentation
README.mdto document the newcrepackage, its purpose, main changeset, operation, and package structure.