Make SAM artifacts and scheduled-target previews identity-stable

[alexeygrigorev]

Make SAM artifacts and scheduled-target previews identity-stable

Status: blocked
Tags: bug, backend, infra, testing, P0
Depends on: None
Blocks: #146
Next owner: HUMAN — authorized sandbox operator
Resume condition: Sanitized public issue evidence confirms preview PASS from exact commit 676a4f66cb4aafa2eef13287bc9c06e6c5f6e115 within the issue allowlist, never executed, and deleted after review.

Scope

Ship an independent architecture-preparation release that removes the two causes of the blocked #146 preview while preserving the normal application deployment path and all existing runtime identities.

The preparation release has two inseparable parts:

1. Make the shared SAM Lambda payload reproducible. Pin the build Node toolchain to exact Node 20.20.2 for local preview builds and both jobs in .github/workflows/deploy-dataops-v1.yml. Keep host/toolchain/cache provenance (including Node version, platform, architecture, and cache fingerprint) outside every deployed Lambda CodeUri payload. Preserve integrity and runtime-boundary verification using deterministic payload metadata or an external build sidecar under the owned .tmp/ cache boundary. A wrong Node version must fail before an artifact is accepted.
2. Replace only the six inline SAM Schedule events attached to the three Label deployed sandbox runtime and portable exports as sandbox #146 consumer functions with explicit AWS::Events::Rule and AWS::Lambda::Permission resources. Reuse the exact SAM-generated logical IDs so CloudFormation adopts the existing resources in place. Feed targets from three required, stable function-name parameters whose values are read from the existing stack outputs after OIDC credential setup; construct target ARNs from those names and the current partition/region/account. This removes rule/permission dependencies on mutable function resources while retaining the existing physical function, rule, and permission identities.

The six schedule pairs are:

• ConversationalExecutionWorkerFunctionExecutionRecovery and ...Permission
• ConversationalExecutionWorkerFunctionExecutionHealthPulse and ...Permission
• ConversationalResultDispatcherFunctionResultDelivery and ...Permission
• BackendFunctionDailyBackendCron and ...Permission
• BackendFunctionDailyBackendExport and ...Permission
• BackendFunctionDailyMailingExport and ...Permission

For each pair, preserve the currently transformed schedule expression, description, enabled/condition state, input, target ID, invocation principal/action, and source-rule relationship. Do not assign new physical names. The deployment workflow must resolve the existing ConversationalExecutionWorkerFunctionName, ConversationalResultDispatcherFunctionName, and BackendFunctionName outputs, validate that each is one non-empty Lambda name from the expected stack/account/region, and pass them as the three exact target-name parameters to the ordinary sam deploy command.

This issue must be implemented from the canonical deployed baseline that excludes commit 3491f65acad1986bd002cd1dabbb3e6b8eca0cfa. The current known deployed baseline is 6236865e509c0e142d364e6c56f7856d8f932076; if canonical main advances first, the orchestrator must prove the selected baseline still excludes #146. The local main ancestry is not authoritative while it contains unpushed descendants of 3491f65.

Acceptance Criteria

• One canonical file declares exact Node 20.20.2; local SAM builds and both workflow jobs use it, and the artifact build fails clearly on a different Node version before publishing or reusing an artifact.
• Host/toolchain/cache provenance is outside all six deployed Lambda payloads. No deployed file contains process.version, host platform/architecture, or a host-derived cache fingerprint.
• Two clean builds of the same source with the pinned toolchain produce byte-identical payload inventories and hashes for all six function artifacts; warm-cache reuse produces the same result.
• All six functions retain their current logical IDs, handlers, CodeUri: sam-build, runtime, configuration, roles/policies, and physical-name behavior. No alias, version, replacement function, live S3 code pointer, or packaged-template edit is introduced.
• Exactly the six listed SAM schedules become twelve explicit rule/permission resources under their existing transformed logical IDs. Their resolved behavior matches the pre-change transformed template except for the target reference changing to a stable function-name parameter.
• The three target-name parameters are required and have no environment-specific or physical-name default. After OIDC setup, the normal deploy job reads only the existing stack outputs, validates them, and passes exactly those values to sam deploy.
• A source-template diff is limited to adding the twelve explicit resources, removing their six inline event declarations, and adding the three target-name parameters. A processed-template diff proves zero logical-resource additions/removals/renames and no changes to physical names, retention/deletion policies, functions, IAM, queues, tables, buckets, alarms, Function URL, DynamoDB stream events, Sponsor send recovery, or the explicit Sponsor SES EventBridge rule beyond the approved one-time artifact and six schedule-reference migration.
• The existing schedule expressions, state conditions, event inputs, target IDs, and invocation permissions are covered by focused tests, including default-off/on transforms for the conversational schedules.
• The normal main GitHub Actions -> OIDC -> make sam-build -> artifact gates -> sam deploy --config-env full-sandbox path remains the only executing app deployment path. No manual app deployment, alternate stack, long-lived credential, or bypass is added.
• The candidate diff contains none of Label deployed sandbox runtime and portable exports as sandbox #146's provenance semantics: DataOpsEnvironment remains the deployed baseline shape (Default: prod with its existing allowed pattern), no explicit DataOpsEnvironment=sandbox deploy override is present, export/archive expectations and docs are unchanged, and none of Label deployed sandbox runtime and portable exports as sandbox #146's focused provenance tests are introduced. Shared files may change only for Make SAM artifacts and scheduled-target previews identity-stable #217's packaging/schedule contracts.
• The branch contains no changes from unrelated local work or ../aws-infra, ../dtc-operations, ../datatasks, or ../podcast-assistant.
• [HUMAN] From the immutable accepted Make SAM artifacts and scheduled-target previews identity-stable #217 commit, an authorized sandbox operator uses exact Node 20.20.2, the ordinary make sam-build/SAM packaging path, and --no-execute-changeset to create one preview. The preview is never executed and is deleted after review. Sanitized evidence must show zero adds/removes, no Replacement=True, and modifications limited to the reviewed one-time six-function payload normalization, the six listed rules, the six listed permissions (where Conditional is acceptable only for this reference migration after resolved old/new function names are proven equal), plus ordinary deployment-ID drift if present. Any other resource is BLOCK/ABORT.
• After preview PASS, the accepted commit is merged and pushed through normal main; On-Call records the terminal GitHub Actions/OIDC deployment result. There is no manual deployment or execution of the preview change set.
• [HUMAN] After the normal deployment, an authorized operator provides sanitized read-only CloudFormation/Lambda/EventBridge evidence that the same six functions, six rules, and six permissions retain their prior physical identities, all schedules retain their prior state/targets/input, and the stack is terminal-successful. No invocation, data write, export, provider message, or destructive operation is needed.
• Make SAM artifacts and scheduled-target previews identity-stable #217 is closed only after the normal deployment and identity evidence pass. Label deployed sandbox runtime and portable exports as sandbox #146 remains unmerged/unshipped, is reconstructed from the deployed Make SAM artifacts and scheduled-target previews identity-stable #217 baseline, and receives fresh Architecture, Security, Tester, PM, immutable preview, and release evidence.

Test Scenarios

Scenario: Host metadata cannot change deployed bytes

Given: Two clean caches and the exact same source tree under Node 20.20.2
When: All six SAM function artifacts are built twice and once through warm-cache reuse
Then: Canonical file inventories and SHA-256 hashes are identical, no payload contains host/cache provenance, and all artifact/runtime-boundary checks pass

Scenario: Wrong preview toolchain fails closed

Given: A local or CI SAM build running any Node version other than 20.20.2
When: The shared artifact builder starts
Then: It exits before accepting/reusing/publishing an artifact and names the required and actual versions

Scenario: Six schedules preserve behavior but lose function dependencies

Given: SAM-processed templates immediately before and after #217 with equivalent three target-name parameter values
When: Their resource graphs and resolved properties are compared
Then: The same six rule and six permission logical IDs remain, schedule behavior is unchanged, and none references Ref/GetAtt of the three mutable function resources

Scenario: Deployment resolves stable targets safely

Given: The existing full-sandbox stack and OIDC credentials in the normal deploy job
When: The workflow resolves the three existing function-name outputs
Then: It validates and passes the three names as required SAM parameters without writing configuration, editing a packaged template, or using live S3 code pointers

Scenario: Preparation candidate excludes #146

Given: The selected deployed baseline and the complete #217 candidate
When: The source, tests, docs, workflow, and processed-template diff are audited
Then: Only #217 packaging/schedule work is present and all sandbox provenance/export changes from 6236865...3491f65 are absent

Scenario: Independent release unblocks reconstruction

Given: Agent verification, PM acceptance, the HUMAN no-execute preview, normal OIDC deployment, and HUMAN read-only identity evidence all pass
When: #217 is closed and #146 is resumed
Then: #146 starts from the deployed #217 SHA and its fresh preview no longer changes unchanged function Code identities or the six schedule/permission resources

Proportional Verification

Agent-verifiable before commit (no AWS/provider access):

• Focused Node tests for artifact fingerprints/cache isolation, payload inventory, wrong-version failure, runtime boundary, and warm/cold reproducibility.
• Focused tests/infra/test_template.py tests for all twelve explicit resources, exact transformed behavior, stable target parameters, zero mutable-function references, baseline identity inventory, OIDC ordering, and Label deployed sandbox runtime and portable exports as sandbox #146-exclusion.
• Full uv run pytest tests/infra/test_template.py.
• Full backend tests, backend typecheck, and backend build because the shared packaged payload and its verifier change.
• make sam-validate, two clean make sam-build runs under Node 20.20.2, standalone SAM artifact/runtime/frontend gates, and a recorded six-artifact hash comparison under .tmp/.
• git diff/ancestry checks against the selected baseline and against 6236865...3491f65; no provider calls, preview, deploy, commit, push, or runtime invocation during implementation/testing/PM acceptance.

External evidence is deliberately separate:

• [HUMAN] One immutable non-executing CloudFormation preview, reviewed and deleted; never execute it.
• Normal merge/push and GitHub Actions OIDC deployment owned by the orchestrator/On-Call after preview PASS.
• [HUMAN] Sanitized read-only post-deployment identity evidence only; no functional invocation or data mutation is required.

Exact Sequencing

1. Orchestrator selects a clean canonical/deployed baseline that excludes 3491f65 and records its SHA.
2. Software Engineer implements only Make SAM artifacts and scheduled-target previews identity-stable #217 in an isolated worktree; no commit, push, preview, provider access, or deployment.
3. Architecture reviews deterministic payload boundaries, the three stable target inputs, exact twelve-resource identity migration, and the preview allowlist.
4. Tester runs all agent-verifiable checks above and reports exact commands/hashes; PM accepts or rejects the immutable code slice.
5. After Tester PASS and PM acceptance, Software Engineer commits exactly the accepted paths with Refs #217; do not merge or push yet.
6. [HUMAN] Authorized operator creates/reviews/deletes the exact non-executing preview from that commit. Any result outside the allowlist is BLOCK/ABORT and returns to Architecture/Engineering.
7. After preview PASS, orchestrator merges and pushes main. The existing workflow performs the sole deployment through OIDC; On-Call monitors to terminal success.
8. [HUMAN] Authorized operator supplies sanitized read-only post-deployment identity evidence. PM closes Make SAM artifacts and scheduled-target previews identity-stable #217 after all criteria pass.
9. Only then reconstruct/rebase Label deployed sandbox runtime and portable exports as sandbox #146 from the deployed Make SAM artifacts and scheduled-target previews identity-stable #217 SHA and repeat its complete lifecycle with a fresh immutable preview. Do not reuse Label deployed sandbox runtime and portable exports as sandbox #146 commit 3491f65, its aborted preview, or its fingerprints as release evidence.

Safety Boundaries

• Preserve application stack name/config environment, logical and physical resource identities, existing OIDC role/path, parameter ordering/guards, and all data retention/deletion policies.
• Fail closed if a stack output is missing/ambiguous, a target differs from the existing function identity, the Node version differs, or a preview contains an unapproved resource/action.
• Do not execute a preparation preview, manually deploy the app, edit packaged templates, substitute deployed/live S3 pointers, create alternate resources, or introduce dual paths/compatibility fallbacks.
• Do not invoke Lambdas or schedules; write/export/import/restore/migrate data; access secrets/providers; send Telegram/sponsor/client messages; or publish private identifiers, ARNs, account IDs, paths, or credentials in this public issue.

Out of Scope

• Any Label deployed sandbox runtime and portable exports as sandbox #146 sandbox provenance, archive-key/manifest, DataOpsEnvironment, export test, or documentation change.
• Reconstructing, rebasing, accepting, previewing, merging, pushing, deploying, or closing Label deployed sandbox runtime and portable exports as sandbox #146.
• Changing function names, aliases, versions, handlers, runtimes, IAM, queues, tables, buckets, alarms, Sponsor schedules, the SES event rule, DynamoDB stream sources, or application behavior.
• Editing ../aws-infra or any source repository, changing OIDC/IAM/provider configuration, or adding a manual deployment route.
• Production, new environments, fresh-stack bootstrap design, runtime smoke/invocation, data mutation, export/import/restore, or private operational evidence.

Dependencies and Blocks

• Architecture diagnosis and evidence: Label deployed sandbox runtime and portable exports as sandbox #146 (comment)
• Make SAM artifacts and scheduled-target previews identity-stable #217 has no implementation dependency and is intentionally releasable before Label deployed sandbox runtime and portable exports as sandbox #146.
• Label deployed sandbox runtime and portable exports as sandbox #146 remains blocked until Make SAM artifacts and scheduled-target previews identity-stable #217 is deployed through normal main and its post-deployment identity evidence passes.

[alexeygrigorev]

PM GROOMED — independent architecture preparation is agent-ready

#217 now isolates the two prerequisites exposed by #146’s blocked preview: deterministic shared Lambda payloads under exact Node 20.20.2, and six identity-preserving explicit schedule/permission pairs fed by three stable existing-function inputs. The normal main SAM/OIDC path, application stack, and runtime resource identities remain mandatory.

The issue explicitly excludes commit 3491f65 and every #146 sandbox provenance/export change from this preparation release. Agent-verifiable local work is separated from the HUMAN immutable no-execute preview and HUMAN sanitized read-only post-deployment identity evidence. The preview must be deleted and never executed; the only deployment is the normal push-triggered OIDC workflow.

needs grooming is removed. #217 has no implementation dependency, blocks #146, and can ship independently. Next owner: Software Engineer, starting from a clean canonical/deployed baseline that excludes 3491f65.

[alexeygrigorev]

SOFTWARE ENGINEER — READY FOR TEST

Implemented the complete groomed #217 architecture-preparation candidate from deployed baseline 6236865e509c0e142d364e6c56f7856d8f932076. The branch remains uncommitted and excludes 3491f65.

Summary

• Added the canonical exact Node 20.20.2 pin and fail-closed builder check; both deployment-workflow jobs consume the pin.
• Kept all host/toolchain/cache provenance in the external .tmp cache sidecar. Each deployed payload now contains only deterministic .dataops-sam-bundle.json metadata with schemaVersion, format, bundledOutputs, and inputs; the verifier rejects extra metadata keys and uses the deterministic output allowlist.
• Preserved all six Lambda payload identities/configuration shapes while keeping integrity, runtime-boundary, frontend-parity, cache isolation, cancellation, and concurrency gates.
• Migrated exactly the six groomed inline schedules to the twelve existing transformed logical IDs, backed by three required stable target-name parameters.
• Updated the sole deploy job to resolve the three existing function-name outputs after OIDC setup, validate stack/account/region and exact Lambda-name shape, and pass them to the ordinary sam deploy invocation.
• Preserved baseline DataOpsEnvironment behavior and introduced none of Label deployed sandbox runtime and portable exports as sandbox #146's provenance/export semantics.

Changed files

• .node-version
• .github/workflows/deploy-dataops-v1.yml
• backend/scripts/build-sam-artifact.mjs
• backend/scripts/frontend-parity-sam-smoke.mjs
• backend/scripts/runtime-boundary.test.mjs
• backend/scripts/sam-build-artifact.test.mjs
• backend/scripts/sam-workspace-isolation.test.mjs
• backend/scripts/verify-runtime-boundary.mjs
• infra/template.full.yaml
• tests/infra/test_template.py

Verification — all PASS

Exact toolchain wrapper used for Node-sensitive checks:

npm_config_cache=.tmp/npm-node-cache npm exec --yes --package=node@20.20.2 -- sh -c '<command>'
node --version -> v20.20.2

• node --test backend/scripts/runtime-boundary.test.mjs backend/scripts/sam-build-artifact.test.mjs — 17 passed, 0 failed.
• Wrong-version execution under host Node 24.13.1 — rejected before destination creation with SAM artifact build requires Node.js 20.20.2; actual version is 24.13.1.
• uv run pytest tests/infra/test_template.py — 37 passed.
• make sam-validate — valid SAM template.
• npm --prefix backend test — 1,093 tests; 1,088 passed, 5 intentional skips, 0 failed.
• npm --prefix backend run typecheck — passed all source/test/script TypeScript checks.
• npm --prefix backend run build — passed compile, runtime-boundary, 72-file frontend copy, and frontend verification.
• make test-sam-workspace-isolation — passed: one shared build, six identical artifacts, source/root snapshots unchanged, cold phases once, warm reuse with zero install/compile/bundle phases, no surviving SAM/import processes.
• Ordinary make sam-build && make verify-sam-runtime-boundary && make verify-sam-frontend under Node 20.20.2 — passed on final candidate fingerprint 27fd159bcfea7965b0c0b22972e25548c4392a04207c830d415c352ad8affe72.
• Two independent clean-cache make sam-build runs plus one warm-cache reuse under .tmp/issue-217-proof-final/ — all passed.
• Standalone runtime-boundary and frontend gates against the final clean artifact — passed.
• git diff --check — passed.
• git merge-base --is-ancestor 3491f65acad1986bd002cd1dabbb3e6b8eca0cfa HEAD — exit 1 as required; HEAD/base remains 6236865e509c0e142d364e6c56f7856d8f932076.
• Candidate path and semantic audit — only the ten Make SAM artifacts and scheduled-target previews identity-stable #217 files above changed; no source repositories or ../aws-infra changed; no DataOpsEnvironment override or Label deployed sandbox runtime and portable exports as sandbox #146 archive/provenance implementation is present.

Reproducibility fingerprints

For each of the six functions in clean build A, clean build B, and warm reuse (18 inventories total):

files: 79
bytes: 12,441,304
canonical path+byte inventory SHA-256:
88861626534ae502b53588823e6c596d03c5313604ffbd60eefc2e2b5b34f2ad

All 18 values are identical. Payload metadata bytes also compare equal across clean A, clean B, and warm reuse. Payload metadata contains 6 bundled outputs and 753 deterministic inputs. It contains no Node/platform/architecture/cache fingerprint. The external sidecar alone records Node 20.20.2, platform, architecture, and cache fingerprint 27fd159bcfea7965b0c0b22972e25548c4392a04207c830d415c352ad8affe72.

The independent workspace-isolation verifier's own canonical inventory algorithm also reported the same hash for all six artifacts:

fd4c9bd125511419df00c6bcb891bb4a0e74ffcf15c12042d26302e1204d18c1

Acceptance status

All agent-verifiable acceptance criteria and proportional checks are implemented and passing. The [HUMAN] immutable no-execute preview, post-merge normal OIDC deployment, and sanitized post-deployment identity evidence remain deliberately pending for their later lifecycle gates. No preview, deploy, AWS/provider access, runtime invocation, commit, merge, push, or HUMAN gate was performed.

Next owner: Tester.

[alexeygrigorev]

ARCHITECTURE PASS

Affected paths:

• .node-version
• .github/workflows/deploy-dataops-v1.yml
• backend/scripts/build-sam-artifact.mjs
• backend/scripts/frontend-parity-sam-smoke.mjs
• backend/scripts/runtime-boundary.test.mjs
• backend/scripts/sam-build-artifact.test.mjs
• backend/scripts/sam-workspace-isolation.test.mjs
• backend/scripts/verify-runtime-boundary.mjs
• infra/template.full.yaml
• tests/infra/test_template.py

Concrete evidence:

• The candidate is still the uncommitted slice on deployed baseline 6236865e509c0e142d364e6c56f7856d8f932076. git merge-base --is-ancestor 3491f65acad1986bd002cd1dabbb3e6b8eca0cfa HEAD exits 1, and the only changed paths are the ten Make SAM artifacts and scheduled-target previews identity-stable #217 paths above. DataOpsEnvironment remains Default: prod with the baseline allowed pattern; there is no deploy override and none of Label deployed sandbox runtime and portable exports as sandbox #146's sandbox provenance/export implementation is present.
• .node-version contains exactly 20.20.2; both workflow jobs use node-version-file, and build-sam-artifact.mjs checks the running version before fingerprint calculation, cache lookup/reuse, or artifact publication. The canonical version file is itself a build input.
• Host/toolchain/cache provenance is outside every CodeUri payload. The external cache sidecar contains the fingerprint and Node/platform/architecture, while copied payloads contain only deterministic .dataops-sam-bundle.json keys schemaVersion, format, bundledOutputs, and inputs. The cache sidecar inventories and hashes every payload file before acceptance and again after each isolated copy. Inspection of the final six built payloads found no Node 20.20.2, Linux/x64 provenance, or cache fingerprint; all six payload metadata hashes match. The engineer's two cold builds and warm reuse report the same 79-file, 12,441,304-byte inventory hash 88861626534ae502b53588823e6c596d03c5313604ffbd60eefc2e2b5b34f2ad for all 18 inventories.
• Local SAM processing of baseline and candidate produced 87 resources before and 87 after, with zero additions/removals, identical conditions and outputs, and exactly the intended twelve changed logical IDs: the six named rules and their six permissions. Every other processed resource is byte-structurally equal to baseline.
• The six explicit rules preserve the transformed descriptions, expressions, conditional/enabled states, target IDs, and inputs. The six permissions preserve action, principal, and source-rule relationships. No migrated rule or permission has a Ref/GetAtt dependency on ConversationalExecutionWorkerFunction, ConversationalResultDispatcherFunction, or BackendFunction; targets use the three required no-default name parameters and partition/region/account ARN construction. No physical rule, permission, or function name is introduced.
• The deploy workflow resolves exactly the three existing stack outputs after OIDC credential setup, verifies the stack ARN against the configured stack/account/region, requires one valid Lambda name for each output, and passes each once to the sole ordinary sam deploy --config-env full-sandbox invocation. It adds no packaged-template edit, live S3 pointer, alternate stack, manual deploy, or long-lived credentials.
• The processed graph leaves the pre-existing Function URL and DynamoDB stream dependencies intact while removing only the six schedule-to-function dependencies diagnosed in Label deployed sandbox runtime and portable exports as sandbox #146. Sponsor functions, Sponsor send recovery, and the explicit SES rule are unchanged.
• Focused tests cover wrong-Node rejection, external-sidecar integrity, payload-metadata schema, warm/cold isolation, all twelve explicit resources, default-off/on conversational transforms, stable target resolution, baseline resource inventory, ordinary OIDC ordering, and Label deployed sandbox runtime and portable exports as sandbox #146 exclusion. The engineer reports 17 focused Node tests, 37 infrastructure tests, 1,088 passing backend tests with 5 intentional skips, typecheck/build/SAM/runtime/frontend gates, and workspace isolation all passing.

Human-preview allowlist retained exactly: from the later immutable accepted commit, the non-executing preview must have zero adds/removes and no Replacement=True; modifications may include only the reviewed one-time six-function payload normalization, these six rules, these six permissions, and ordinary deployment-ID drift. A permission may be Conditional only after sanitized evidence proves its resolved old/new function name is identical. Any other resource/action is BLOCK/ABORT. The preview must be deleted and never executed; normal main GitHub Actions OIDC remains the only deployment path.

Minimum fixes: none for the architecture gate. Tester, PM acceptance, immutable commit, HUMAN no-execute preview, normal OIDC deployment, and HUMAN post-deployment identity evidence remain required in the issue's exact sequence.

[alexeygrigorev]

TEST PASS

Independent Tester verification completed against the exact uncommitted #217 candidate on baseline 6236865e509c0e142d364e6c56f7856d8f932076, branch issue-217-identity-stable. Candidate content SHA-256 across the ten changed paths is a0b57a5d477c6573caafb220be91a9fb418d976faec1c1d8c15ce3c01d4bf729. No files were edited or staged; no commit, push, preview, provider access, deployment, runtime invocation, or HUMAN gate was performed.

Commands and results

All commands ran from the existing project-local worktree. Local path prefixes are omitted from public evidence.

• npm_config_cache=.tmp/npm-node-cache npm exec --yes --package=node@20.20.2 -- sh -c 'node --version && node --test backend/scripts/runtime-boundary.test.mjs backend/scripts/sam-build-artifact.test.mjs' — exit 0; Node v20.20.2; 17 passed, 0 failed.
• node --version && test ! -e .tmp/tester-217-wrong-version && ARTIFACTS_DIR=.tmp/tester-217-wrong-version node backend/scripts/build-sam-artifact.mjs — exit 1 as required under Node v24.13.1; clear required/actual-version diagnostic. Follow-up test ! -e .tmp/tester-217-wrong-version — exit 0, proving failure occurred before destination creation.
• uv run pytest -q tests/infra/test_template.py::test_six_schedules_are_explicit_identity_stable_resources_with_exact_behavior tests/infra/test_template.py::test_processed_template_keeps_baseline_resource_identity_inventory tests/infra/test_template.py::test_deploy_workflow_pins_toolchain_and_resolves_only_existing_schedule_targets tests/infra/test_template.py::test_issue_217_candidate_preserves_baseline_environment_and_excludes_146_provenance — exit 0; 4 passed.
• uv run pytest -q tests/infra/test_template.py::test_conversational_result_delivery_schedule_transforms_default_off_and_ingress_on tests/infra/test_template.py::test_conversational_execution_schedules_transform_with_exact_rules_permissions_and_inputs — exit 0; 2 passed.
• uv run pytest tests/infra/test_template.py — exit 0; 37 passed in 18.16s.
• npm --prefix backend test — exit 0; 1,093 tests: 1,088 passed, 5 intentional skips, 0 failed.
• npm --prefix backend run typecheck — exit 0.
• npm --prefix backend run build — exit 0; compile and runtime boundary passed; 72 canonical frontend assets copied and verified.
• make sam-validate — exit 0; valid SAM template.
• Pinned clean build A: npm_config_cache=.tmp/npm-node-cache npm exec --yes --package=node@20.20.2 -- sh -c 'node --version && DATAOPS_SAM_CACHE_ROOT=$PWD/.tmp/tester-217-proof/cache-clean-a make sam-build SAM_BUILD_DIR=.tmp/tester-217-proof/build-clean-a' — exit 0.
• Pinned clean build B: same command with cache-clean-b / build-clean-b — exit 0.
• Pinned warm reuse: same command reusing cache-clean-a with build-warm-a — exit 0; all six functions reported cache reuse and no install/compile/bundle phase.
• Independent recursive path+byte hash comparator over all six functions in clean A, clean B, and warm A — exit 0; all 18 inventories identical: 79 files, 12,441,304 bytes, SHA-256 88861626534ae502b53588823e6c596d03c5313604ffbd60eefc2e2b5b34f2ad. JSON inventory hash was also identical: fd4c9bd125511419df00c6bcb891bb4a0e74ffcf15c12042d26302e1204d18c1.
• Ordinary pinned make sam-build — exit 0; six-artifact warm reuse succeeded.
• make verify-sam-frontend — exit 0; 72 packaged assets byte-verified.
• make verify-sam-runtime-boundary — exit 0.
• make test-sam-frontend-isolation — exit 0; 13 passed, 0 failed.
• node backend/scripts/frontend-parity-sam-smoke.mjs — exit 0; packaged health/reset and authenticated admin/operator work API checks passed.
• npm_config_cache=.tmp/npm-node-cache npm exec --yes --package=node@20.20.2 -- sh -c 'node --version && make test-sam-workspace-isolation' — exit 0; one shared cold build, six identical artifacts, tracked/root snapshots unchanged, warm install/compile/bundle phases 0, no surviving SAM/import processes. Six artifacts each reported 12,441,304 bytes and inventory hash fd4c9bd125511419df00c6bcb891bb4a0e74ffcf15c12042d26302e1204d18c1.
• Independent source/processed graph comparator against baseline 6236865… — exit 0. Source: 61 -> 73 resources, exactly the twelve named rule/permission IDs added, no removals, exactly the three consumer functions changed solely by removal of the six named inline schedules, and exactly three required parameters added. Processed: 87 -> 87 resources, zero additions/removals/renames, identical Conditions and Outputs, and exactly the twelve named resources structurally changed. All twelve contain zero Ref/GetAtt references to the three mutable functions.
• Payload provenance inspection — exit 0. Each payload metadata file has only bundledOutputs, format, inputs, schemaVersion; 6 outputs and 753 deterministic inputs; no Node/platform/architecture/cache fingerprint. Searches found no process.version, 20.20.2, or cache fingerprint in any clean-A payload. External sidecar alone records Node 20.20.2, linux, x64, fingerprint 27fd159bcfea7965b0c0b22972e25548c4392a04207c830d415c352ad8affe72, and 79-file inventory.
• Workflow audit — exit 0: credential setup -> existing-output resolution -> build -> frontend/runtime/isolation gates -> sole deploy step; exactly one sam deploy, one --config-env full-sandbox, two .node-version consumers, zero legacy node-version: "20", zero aws cloudformation deploy, zero sam package.
• git diff --check — exit 0. git diff --cached --name-only — empty. Status remains exactly nine modified tracked paths plus untracked .node-version; diff stat remains 569 insertions/118 deletions. HEAD remains 6236865….
• git merge-base --is-ancestor 3491f65acad1986bd002cd1dabbb3e6b8eca0cfa HEAD — exit 1 as required; merge-base is 6236865…. Candidate has no backend test/docs paths from Label deployed sandbox runtime and portable exports as sandbox #146, no sandbox override, and DataOpsEnvironment remains baseline Default: prod with the existing allowed pattern.

One discarded harness invocation used a relative DATAOPS_SAM_CACHE_ROOT; SAM runs custom builders from temporary directories, so the safety boundary correctly rejected it with exit 2. It created no accepted artifact. The required clean builds were rerun with project-local absolute cache roots and all passed; this was a tester invocation correction, not a candidate defect.

Acceptance criteria in issue order

1. PASS — canonical .node-version is exactly 20.20.2; both workflow jobs consume it; local pinned builds pass; wrong Node fails before destination publication/reuse.
2. PASS — host/toolchain/cache provenance is external to all six payloads; deterministic payload metadata schema and explicit negative searches pass.
3. PASS — clean A, clean B, and warm reuse are byte-identical for all 18 inventories with hashes above.
4. PASS — all six functions retain logical IDs, handlers, CodeUri: sam-build, runtime/configuration/roles/policies/physical-name behavior; processed graph shows no function changes or replacement constructs.
5. PASS — exactly the six schedules become the twelve existing transformed logical IDs; behavior matches baseline except stable target reference.
6. PASS — three parameters are required/no-default; workflow resolves exactly the three existing outputs after OIDC, validates stack/account/region/name shape, and passes each once.
7. PASS — source and processed graph evidence is exact: zero processed identity additions/removals/renames and no unrelated resource/policy/function changes.
8. PASS — exact expressions, descriptions, states/conditions, inputs, target IDs, permissions, and default-off/on conversational transforms are covered and pass.
9. PASS — normal main GitHub Actions -> OIDC -> build -> gates -> sole sam deploy --config-env full-sandbox remains the only executing path.
10. PASS — Label deployed sandbox runtime and portable exports as sandbox #146 sandbox provenance/export semantics and tests are absent; baseline environment shape is preserved.
11. PASS — candidate contains only the ten declared Make SAM artifacts and scheduled-target previews identity-stable #217 paths and no external/source-repository or unrelated work.
12. DEFERRED [HUMAN] — immutable no-execute preview is intentionally later; Tester did not access providers.
13. DEFERRED release sequence — merge/push and normal OIDC deployment remain later gates.
14. DEFERRED [HUMAN] — sanitized read-only post-deployment identity evidence remains later.
15. DEFERRED closure/reconstruction sequencing — Make SAM artifacts and scheduled-target previews identity-stable #217 remains open and Label deployed sandbox runtime and portable exports as sandbox #146 remains unshipped as required.

Verdict: the exact current candidate passes every agent-verifiable acceptance criterion. Concrete fixes required: none. Next owner: Product Manager acceptance; HUMAN preview/release gates remain mandatory in the issue's exact sequence.

[alexeygrigorev]

ACCEPTED — final PM code-slice acceptance

I reviewed the exact current uncommitted candidate in the existing project-local worktree against the full groomed issue, Software Engineer handoff, Architecture PASS, and latest independent TEST PASS. This acceptance is for the agent-verifiable code slice only; it does not claim preview, deployment, or post-deployment identity completion.

Concrete acceptance reasons:

• The candidate remains based on deployed baseline 6236865e509c0e142d364e6c56f7856d8f932076, with no staged/conflicted paths and exactly the ten declared Make SAM artifacts and scheduled-target previews identity-stable #217 paths. Commit 3491f65acad1986bd002cd1dabbb3e6b8eca0cfa is not an ancestor, and the candidate contains none of Label deployed sandbox runtime and portable exports as sandbox #146's sandbox provenance/export semantics, tests, docs, or DataOpsEnvironment override.
• .node-version is the single exact 20.20.2 declaration; both workflow jobs consume it. The builder checks it before fingerprinting, cache acceptance/reuse, destination creation, or publication and reports required versus actual versions on failure.
• Toolchain/host/cache provenance stays in the project-local external cache sidecar. The deployed payload metadata is deterministic and schema-limited, while sidecar inventory validation preserves artifact integrity and runtime-boundary verification. Independent cold A/cold B/warm evidence proves all 18 six-function inventories byte-identical at 79 files, 12,441,304 bytes, SHA-256 88861626534ae502b53588823e6c596d03c5313604ffbd60eefc2e2b5b34f2ad.
• The source migration is exactly the three required no-default target-name parameters plus the six named rules and six permissions replacing only their six inline schedules. Processed-template evidence keeps 87 resources before and after, with zero additions/removals/renames, unchanged Conditions/Outputs, and every non-migrated resource structurally unchanged. Expressions, descriptions, states/conditions, inputs, target IDs, invocation action/principal, and source-rule relationships remain the baseline behavior.
• All six shared Lambda consumers retain logical IDs, handlers, CodeUri: sam-build, Node runtime, configuration, IAM, and physical-name behavior. No aliases, versions, replacement functions, physical rule names, packaged-template edits, live S3 pointers, or alternate resources were introduced. Sponsor recovery, Sponsor SES rule, Function URL, DynamoDB stream events, retained resources, alarms, queues, tables, and buckets remain outside the changed processed graph.
• The operator path remains the existing main workflow through GitHub Actions OIDC, make sam-build, artifact gates, and the sole ordinary sam deploy --config-env full-sandbox. After OIDC setup, the resolver reads only the three existing stack outputs, verifies stack/account/region identity, rejects missing/duplicate/invalid Lambda names, and passes each required value once. It adds no long-lived credentials, manual deployment implementation, alternate stack, or bypass.
• Configuration and failure states are clear and fail closed: exact wrong-Node diagnostics, required/no-default SAM parameters, stack identity rejection, and missing/ambiguous/invalid output rejection all happen before deployment. The latest independent Tester ran the complete proportional workflow: 17 focused Node tests, 37 infrastructure tests, 1,088 backend passes with 5 intentional skips, typecheck/build, SAM validation, two clean builds plus warm reuse, runtime/frontend/isolation gates, processed-graph comparison, provenance inspection, and Label deployed sandbox runtime and portable exports as sandbox #146 ancestry/scope checks. Architecture also passed with no required fixes.

Required gates still pending, explicitly and intentionally:

1. Software Engineer commits exactly this accepted slice with Refs #217; do not merge or push yet.
2. [HUMAN] From that immutable commit, an authorized sandbox operator uses exact Node 20.20.2 and the ordinary SAM path to create, review, and delete one never-executed --no-execute-changeset preview. It must show zero adds/removes, no Replacement=True, and only the issue allowlist; any other action is BLOCK/ABORT. Conditional permissions require proof that resolved old/new function names are equal.
3. Only after preview PASS, the orchestrator merges and pushes through normal main; the existing OIDC workflow is the sole deployment, and On-Call records terminal CI/CD status.
4. [HUMAN] An authorized operator then supplies sanitized read-only evidence that the same six functions, six rules, and six permissions retained physical identities and schedule behavior. Make SAM artifacts and scheduled-target previews identity-stable #217 stays open until this passes; Label deployed sandbox runtime and portable exports as sandbox #146 remains unmerged/unshipped and must later be reconstructed from the deployed Make SAM artifacts and scheduled-target previews identity-stable #217 baseline with fresh lifecycle evidence.

Next lifecycle owner: Software Engineer, to commit the exact accepted paths with Refs #217, followed by the immutable HUMAN preview gate. No preview, deployment, provider access, commit, merge, push, closure, or HUMAN gate was performed during PM acceptance.

[alexeygrigorev]

SOFTWARE ENGINEER — ACCEPTED COMMIT READY FOR HUMAN PREVIEW

Committed the exact Architecture-passed, independently tested, and PM-accepted #217 candidate.

• Commit: 676a4f66cb4aafa2eef13287bc9c06e6c5f6e115
• Parent/deployed baseline: 6236865e509c0e142d364e6c56f7856d8f932076
• Git tree fingerprint: d29cca5b5ca7fcac061668794e464548ca8abe10
• Pre-commit accepted candidate content SHA-256: a0b57a5d477c6573caafb220be91a9fb418d976faec1c1d8c15ce3c01d4bf729 (exact match to independent TEST PASS)
• Subject: Stabilize SAM artifact and schedule identities
• Body: Refs #217

The commit contains exactly these accepted paths:

• .node-version
• .github/workflows/deploy-dataops-v1.yml
• backend/scripts/build-sam-artifact.mjs
• backend/scripts/frontend-parity-sam-smoke.mjs
• backend/scripts/runtime-boundary.test.mjs
• backend/scripts/sam-build-artifact.test.mjs
• backend/scripts/sam-workspace-isolation.test.mjs
• backend/scripts/verify-runtime-boundary.mjs
• infra/template.full.yaml
• tests/infra/test_template.py

Pre-commit verification proved no staged content, exactly those nine modified tracked paths plus untracked .node-version, no unrelated tracked path, and an exact match to the Tester's accepted content fingerprint. Post-commit git status --porcelain=v1 --untracked-files=normal is empty; the worktree has a clean tracked and untracked status. Generated and ignored evidence was not staged.

No merge, push, preview, deployment, provider access, issue closure, or HUMAN gate was performed.

Next gate: authorized HUMAN immutable --no-execute-changeset preview from exact commit 676a4f66cb4aafa2eef13287bc9c06e6c5f6e115, following the issue allowlist. Do not execute the preview.