[CRE] Confidential workflow execution in Chainlink nodes#21603
Open
[CRE] Confidential workflow execution in Chainlink nodes#21603
Conversation
…way wiring Adds end-to-end support for confidential CRE workflows: ConfidentialModule: delegates workflow execution to the confidential-workflows capability, packing owner/executionID into the WorkflowExecution proto. Relay handler: forwards enclave GetSecrets requests to VaultDON with proper namespace defaulting, owner normalization, transmission config, multi-PCR attestation validation, and local node metadata. Gateway handler: routes confidential.secrets.get and confidential.capability.execute JSON-RPC methods between enclaves and relay DON nodes. Supporting changes: capability launcher discovers remote capabilities for capability DONs, workflow syncer handles confidential attributes and HTTP binary URLs, migration adds workflow_attributes column.
chainlink-common: v0.10.1-0.20260319085950-259a5a1c1768 (Owner rename, v2/actions/confidentialrelay, WorkflowExecution proto fields) chainlink-protos/cre/go: cre/go/v1alpha.23 (owner + execution_id on WorkflowExecution) CC plugins: ab85142 (framework dep bump for ComputeRequest.Metadata)
product-security-plaid-production
bot
requested review from
MStreet3 and
jmank88
Contributor
|
👋 nadahalli, thanks for creating this pull request! To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team. Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks! |
Contributor
CORA - Pending Reviewers
Legend: ✅ Approved | ❌ Changes Requested | 💬 Commented | 🚫 Dismissed | ⏳ Pending | ❓ Unknown For more details, see the full review summary. |
Contributor
|
✅ No conflicts with other open PRs targeting |
Contributor
There was a problem hiding this comment.
Pull request overview
Risk Rating: HIGH (introduces a new confidential execution path, new gateway handler, and new DB persistence for workflow attributes; also touches security-sensitive routing/attestation and workflow engine creation paths)
This PR enables “confidential” CRE workflow execution by plumbing workflow attributes through registration/storage/syncer, delegating execution to a confidential-workflows capability, and adding a confidential relay pathway (gateway handler + CRE subservice) to proxy enclave requests for secrets and capability execution.
Changes:
- Add workflow
attributesplumbing end-to-end (registration → DB persistence → syncer → engine routing) and aConfidentialModulefor delegating execution toconfidential-workflows. - Introduce a gateway confidential relay handler (fan-out + quorum) and wire a CRE confidential relay subservice with Nitro attestation validation.
- Update capability launcher discovery behavior for single-DON and capability-type DON topologies; add plugin wiring for
confidential-workflows.
Reviewed changes
Copilot reviewed 33 out of 35 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| system-tests/tests/test-helpers/t_helpers.go | Add workflow registration attributes and a helper to deploy confidential workflows in system tests. |
| system-tests/lib/cre/workflow/workflow.go | Extend workflow registration to pass attributes into v2 registry upsert. |
| system-tests/lib/cre/types.go | Add capability flag for confidential relay. |
| system-tests/lib/cre/features/confidential_relay/confidential_relay.go | Configure nodes/topology for confidential relay in test environments. |
| plugins/plugins.private.yaml | Add confidential-workflows plugin; update confidential-http git ref. |
| go.mod | Dependency updates and additions to support confidential relay / attestation. |
| go.sum | Corresponding dependency checksum updates. |
| deployment/cre/jobs/pkg/gateway_job.go | Add gateway handler type/service name and default config for confidential relay handler. |
| core/store/migrate/migrations/0295_add_workflow_attributes_column.sql | Add attributes column to workflow_specs_v2. |
| core/services/workflows/v2/confidential_module.go | New module delegating execution to confidential-workflows capability; parse attributes + hash binary. |
| core/services/workflows/v2/confidential_module_test.go | Unit tests for attribute parsing + confidential module execution request/response wiring. |
| core/services/workflows/syncer/v2/handler.go | Persist attributes; route confidential workflows to a confidential engine creation path; refactor engine start/register. |
| core/services/workflows/syncer/v2/handler_test.go | Add tests validating confidential routing bypasses engine factory; malformed attribute handling. |
| core/services/workflows/syncer/v2/fetcher.go | Allow file fetcher to accept HTTP(S) URLs (for enclave-fetched WASM) by mapping to local FS. |
| core/services/workflows/syncer/fetcher.go | Same as above for non-v2 syncer fetcher. |
| core/services/workflows/artifacts/v2/orm.go | Include attributes in workflow spec upsert/update SQL. |
| core/services/standardcapabilities/conversions/conversions.go | Map mock command ↔ capability ID. |
| core/services/job/models.go | Add Attributes []byte to job.WorkflowSpec DB model. |
| core/services/gateway/handlers/confidentialrelay/handler.go | New gateway handler that fans out enclave relay JSON-RPC requests and aggregates responses. |
| core/services/gateway/handlers/confidentialrelay/handler_test.go | Tests for gateway confidential relay handler fan-out/quorum/timeout/rate limiting paths. |
| core/services/gateway/handlers/confidentialrelay/aggregator.go | Quorum aggregator (F+1 matching digests) for relay node responses. |
| core/services/gateway/handler_factory.go | Wire the new confidential relay gateway handler type. |
| core/services/cre/cre.go | Start confidential relay subservice when enabled in CRE config. |
| core/services/chainlink/config_cre.go | Implement CRE.ConfidentialRelay() adapter over TOML config. |
| core/config/toml/types.go | Add TOML config block for CRE confidential relay settings and merge behavior. |
| core/config/cre_config.go | Add CREConfidentialRelay interface and plumb into CRE config interface. |
| core/capabilities/launcher.go | Fix capability serving in single-DON topologies; ensure capability DONs discover remote capabilities. |
| core/capabilities/launcher_test.go | Regression test for single-DON capability serving. |
| core/capabilities/confidentialrelay/service.go | Lifecycle wrapper that defers relay handler creation until gateway connector is available. |
| core/capabilities/confidentialrelay/handler.go | New relay handler validating attestation and proxying secrets.get / capability.execute. |
| core/capabilities/confidentialrelay/handler_test.go | Unit tests for the relay handler request handling and lifecycle. |
| core/scripts/go.mod | Align script module dependencies with updated workspace versions. |
| core/scripts/cre/environment/environment/workflow.go | Update workflow registration call signature to pass attributes. |
| .changeset/fix-single-don-capability-serving.md | Changelog entry for single-DON capability serving fix. |
| .changeset/confidential-relay-wiring.md | Changelog entry for confidential relay wiring (marked wip). |
| .changeset/confidential-module-plumbing.md | Changelog entry for confidential module + attributes plumbing. |
Comments suppressed due to low confidence (2)
core/services/workflows/syncer/v2/fetcher.go:231
- Using strings.HasPrefix(fullPath, basePath) is not a safe containment check: a path can resolve outside basePath while still sharing the same prefix (e.g. "/tmp/baseX" has prefix "/tmp/base"). Use filepath.Rel with a ".." check, or a path-segment boundary safe comparison after cleaning.
var fullPath string
if u.Scheme == "http" || u.Scheme == "https" {
// For HTTP(S) URLs, extract just the filename and resolve against basePath.
// This supports confidential workflows where the on-chain URL must be HTTP
// (so the enclave can fetch the binary), but the syncer reads from the local filesystem.
fullPath = filepath.Join(basePath, filepath.Base(u.Path))
} else {
fullPath = filepath.Clean(u.Path)
// ensure that the incoming request URL is either relative or absolute but within the basePath
if !filepath.IsAbs(fullPath) {
// If it's not absolute, we assume it's relative to the basePath
fullPath = filepath.Join(basePath, fullPath)
}
}
if !strings.HasPrefix(fullPath, basePath) {
return nil, fmt.Errorf("request URL %s is not within the basePath %s", fullPath, basePath)
}
core/services/workflows/syncer/fetcher.go:197
- Using strings.HasPrefix(fullPath, basePath) is not a safe containment check: a path like basePath+"../basePathEvil/file" can resolve outside basePath (e.g. "/tmp/baseX") while still sharing the same prefix ("/tmp/base"). Use filepath.Rel (and reject paths starting with ".."), or ensure the cleaned fullPath has the basePath as a path segment boundary (e.g. compare after filepath.Clean and append a path separator).
var fullPath string
if u.Scheme == "http" || u.Scheme == "https" {
// For HTTP(S) URLs, extract just the filename and resolve against basePath.
// This supports confidential workflows where the on-chain URL must be HTTP
// (so the enclave can fetch the binary), but the syncer reads from the local filesystem.
fullPath = filepath.Join(basePath, filepath.Base(u.Path))
} else {
fullPath = filepath.Clean(u.Path)
// ensure that the incoming request URL is either relative or absolute but within the basePath
if !filepath.IsAbs(fullPath) {
// If it's not absolute, we assume it's relative to the basePath
fullPath = filepath.Join(basePath, fullPath)
}
}
if !strings.HasPrefix(fullPath, basePath) {
return nil, fmt.Errorf("request URL %s is not within the basePath %s", fullPath, basePath)
}
Comment on lines
+718
to
+721
| ChainID: registryChainSelector, | ||
| DonID: testEnv.Dons.List()[0].ID, | ||
| ContainerTargetDir: creworkflow.DefaultWorkflowTargetDir, | ||
| Blockchains: testEnv.CreEnvironment.Blockchains, |
Contributor
Author
There was a problem hiding this comment.
Fixed. Using MustWorkflowDON().ID now.
Comment on lines
+63
to
+69
| if v, exists := capConfig.Values["trustedPCRs"]; exists { | ||
| s := v.(string) | ||
| relayConf.TrustedPCRs = &s | ||
| } | ||
| if v, exists := capConfig.Values["caRootsPEM"]; exists { | ||
| s := v.(string) | ||
| relayConf.CARootsPEM = &s |
Contributor
Author
There was a problem hiding this comment.
Fixed. Checked type assertions with error return.
|
|
||
| for _, er := range expiredRequests { | ||
| var nodeResponses string | ||
| for nodeKey, nodeResponse := range er.responses { |
Contributor
Author
There was a problem hiding this comment.
Fixed. Using copiedResponses() which holds the lock.
Comment on lines
+223
to
+227
| var nodeResponses string | ||
| for nodeKey, nodeResponse := range er.responses { | ||
| nodeResponses += fmt.Sprintf("%s ---::: %v ", nodeKey, nodeResponse) | ||
| } | ||
| err := h.sendResponse(ctx, er, h.errorResponse(er.req, api.RequestTimeoutError, errors.New("request expired without getting quorum of responses from nodes. Available responses: "+nodeResponses), []byte(nodeResponses))) |
Contributor
Author
There was a problem hiding this comment.
Fixed. Timeout error now only includes response count. Details go to debug log.
- Use MustWorkflowDON() instead of List()[0] for DON ID - Add checked type assertions for trustedPCRs and caRootsPEM - Fix data race in removeExpiredRequests: use copiedResponses() - Remove sensitive node responses from timeout error messages
chainlink-ccip was behind develop, missing gobindings packages that other code imports. Also ran go mod tidy on core/scripts, system-tests/lib, and system-tests/tests.
|
|
- Run goimports on all changed files - Rename cap -> capability/executable to avoid redefining builtin - Fix err shadow in handler.go (use = instead of :=) - Add fmt import to confidential_relay.go - Use require.NoError instead of assert.NoError for error checks - Tidy deployment go.mod
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chainlink nodes can now run CRE workflows inside Nitro enclaves. This PR adds the three pieces that make that possible:
ConfidentialModule (
core/services/workflows/v2/confidential_module.go): When a workflow is markedconfidential: truein its on-chain attributes, the engine delegates execution to theconfidential-workflowscapability instead of running WASM locally. The module plumbs workflow owner, execution ID, and VaultDON secret identifiers through to the enclave.Confidential relay handler (
core/capabilities/confidentialrelay/): Sits on the relay DON and forwards two types of requests from enclaves:secrets.get(routed to VaultDON for TDH2-encrypted secret shares) andcapability.execute(routed to CRE DON nodes for capability execution). Validates Nitro attestation documents against configured PCR measurements before forwarding.Gateway handler (
core/services/gateway/handlers/confidentialrelay/): Routes JSON-RPC messages between enclaves and relay DON nodes over the gateway transport layer. Aggregates responses from multiple relay nodes.Supporting changes: capability launcher now discovers remote capabilities for capability-type DONs, workflow syncer handles the
confidentialattribute and HTTP binary URLs for enclave-fetched WASM, migration addsworkflow_attributescolumn.