[workflow][infra] Bind and run both break cluster-wide from ~14:15 SGT; identical definition passed 12 minutes earlier (admission SOURCE_CREDENTIAL_REQUIRED + code_execute 502 + Orleans forwarding failure)

[jianwei-su]

Workflow bind and run both break cluster-wide from ~14:15 SGT; identical definition passed 12 minutes earlier

Summary

Between 14:03 and 14:20 SGT on 2026-08-07 (06:03–06:20 UTC), our scope went from fully working to unable to bind or run any workflow. Nothing changed on our side in that window — we can prove this because the identical workflow definition that passed admission at 14:03 and completed all 14 steps at 14:15 is rejected at admission when resubmitted byte-for-byte at 14:20, and still rejected at 14:34.

Four distinct symptoms appeared together. All are infrastructure-level; none reproduce against NyxID or the downstream service directly.

Timeline (SGT, UTC+8)

Time	Event
14:03	workflows:explicit-request-preview returns 6 call sites; bind succeeds
14:04	Run rejected with workflow run is already active while every run in the list reads failed; step count regresses 9/10 → 6/10
14:15	Same member runs 14/14, completed
14:20	Identical definition rejected: NYXID_ADMISSION_SOURCE_CREDENTIAL_REQUIRED
14:20	Runtime code_execute fails: NYXID_PROXY_HTTP_502
14:30	Second workflow (27 steps) fails at step 1/27: Forwarding failed: tried to forward message Request [<internal-silo-address> runtimeactor/sco…
14:34	Admission still rejected (4th reproduction)

Symptom 1 — admission rejects a definition it accepted 17 minutes earlier

POST /api/scopes/{scopeId}/workflows:explicit-request-preview
400 INVALID_USER_WORKFLOW_REQUEST
External workflow capability admission failed:
NYXID_ADMISSION_SOURCE_CREDENTIAL_REQUIRED: A source-readable caller NyxID credential is required.

Reproduced 4 times (14:20, 14:21, 14:22, 14:34) with the byte-identical request body that succeeded at 14:03.

Isolation probes — we first assumed this was specific to a write call site we were adding, and tried to narrow it. All four returned the same error:

Probe	Result
POST /open-apis/im/v1/messages (the new write)	SOURCE_CREDENTIAL_REQUIRED
same call site changed to GET	SOURCE_CREDENTIAL_REQUIRED
same call site pointed at a different path	SOURCE_CREDENTIAL_REQUIRED
same call site degraded to the exact shape of a read step that had just passed	SOURCE_CREDENTIAL_REQUIRED

That assumption was wrong. The control settled it: removing the new call site entirely and resubmitting the definition that ran 14/14 minutes earlier is also rejected. So this is not about any call site we authored.

Both capability selectors are affected — capability.nyxid_operation and capability.nyxid_request behave identically.

Symptom 2 — runtime code_execute returns 502 while the sandbox is healthy

An already-bound workflow (bound 14:03, ran 14/14 at 14:15) now fails at step 8:

tool 'code_execute' execution failed: NYXID_PROXY_HTTP_502: The service request failed.

Calling the same sandbox service directly through NyxID at the same time succeeds:

{"success":true,"output":{"stdout":"ok","stderr":"","exit_code":0,"execution_time_ms":1089}}

So the sandbox is up and the NyxID proxy path to it is healthy. The break is on the aevatar → sandbox leg.

Symptom 3 — Orleans message forwarding failure on a second workflow

A different member (27-step workflow, bound 2026-08-06) fails at step 1 of 27, before any of our code runs:

step_executor_failed: step 'readctx' (tool_call) failed during executor:
Forwarding failed: tried to forward message Request [<internal-silo-address> runtimeactor/sco

The message is truncated by the platform at ~100 characters, so we cannot supply more of it. The internal silo address is redacted here and can be provided privately.

Symptom 4 — run-actor state inconsistency

At 14:04, new runs were refused with workflow run is already active while the runs list showed every run in failed state — no active run existed. In the same window, the observatory step count for one run went backwards, 9/10 → 6/10, across successive polls of the same runId.

We initially logged this as an isolated anomaly and moved on when a retry succeeded at 14:15. In hindsight it looks like the earliest symptom of the same problem: a grain reactivating on a different silo with stale state would produce exactly this.

What is healthy (controls run at 14:34, same session, same token)

• chrono-sandbox direct via NyxID — 200, stdout: "ok", 1130 ms
• Lark bot service direct via NyxID — reaches upstream, returns the upstream's own scope error
• Caller identity — whoami resolves; token valid for another ~4.5 h
• aevatar read endpoints — scope, members, bindings, observatory runs all respond normally

So: NyxID is up, the downstream services are up, our credential is valid, and aevatar's read plane works. Admission, the runtime tool leg, and actor dispatch are what changed.

Our reading

One hypothesis covers all four symptoms: aevatar lost the ability to read or use the forwarded caller NyxID credential, alongside silo churn. The aevatar service is configured Auth: none and relies entirely on the forwarded caller token — admission needs it to resolve capabilities (symptom 1), and the runtime needs it as the bearer to the sandbox (symptom 2). Symptoms 3 and 4 point at grain relocation happening at the same time.

We are not attached to this explanation; the timeline and the controls are the durable part.

Impact

Both of our production-migration workflows are fully blocked — one that had just reached end-to-end success for the first time, and one that cannot get past its first step. Our production automation still runs on the previous system, so there is no business outage; the migration is what is stalled.

Environment

• Scope, member, workflow and run IDs: available privately on request
• Reproduced continuously from 14:20 to 14:34 SGT on 2026-08-07
• All requests issued through the NyxID proxy with a caller token valid until 19:00 SGT

[jianwei-su]

One-line summary: P2 first ran 14/14 at 14:09 today with numbers matching our current production system exactly, but since 14:15 the identical definition fails admission (NYXID_ADMISSION_SOURCE_CREDENTIAL_REQUIRED), runtime code_execute returns 502, and P1 dies on step 1 with Forwarding failed (0/27) — while the sandbox answers 200 directly and both the NyxID proxy and our token are fine, so both migration lines are down. Please start with cluster silo stability.

---

Additional evidence gathered after filing.

Three consecutive runs on the same member, same binding, same definition, inside nine minutes — we changed nothing between them:

Started (SGT)	Result	Steps	Error
14:04:06	failed	6/10	workflow run is already active (no run was actually active)
14:09:19	completed	14/14	—
14:18:27	failed	7/8	code_execute → NYXID_PROXY_HTTP_502

The middle row is the one to look at: that binding is healthy and the definition is correct. Nine minutes later the same thing fails on a tool call, and the step count in the failing run is smaller than the successful run's total, which is consistent with the state regressions described in the issue body.

Admission has now been reproduced five times (14:20, 14:21, 14:22, 14:34, 14:42) with the byte-identical request body that succeeded at 14:03.

Controls re-run at 14:42, unchanged: sandbox direct through NyxID returns 200 (stdout: ok, 1130 ms); the NyxID proxy is healthy; our caller token is valid for several more hours and whoami resolves.

Impact on us: P2's migration is functionally complete and independently reconciled — the only remaining step is one re-bind, which requires explicit-request-preview to work. P1 cannot be assessed at all, because the forwarding failure happens before any of our workflow code executes.

Scope, member, run and revision IDs, plus the full internal silo address in the truncated Orleans message, are available privately on request.

[eanz17]

补充一条与本 issue 中 code_execute 502 相关的归因更正，来自对部署版 chrono-sandbox 源码的直接核对。

Aevatar 此前把 502 一律报成「容量不足」

NyxIdManagedCodexChronoTransport.ProxyFailure 仅按 HTTP 状态分类，429 or 502 or 503 => CapacityUnavailable。而验收面看到的稳定码由 CodexExecutionFailureKind 派生，因此所有 502 都显示为 codex_execution_capacity_unavailable，与真实故障阶段无关。

部署版 chrono-sandbox 的真实语义（feat/managed-codex-execution @ 1e8134d8）

• CODEX_CAPACITY_UNAVAILABLE 只在容量许可被拒时产生，且走 429（src/codex/service.rs 的 too_many_requests）——不产生 502。
• 502 来自 create_codex_sandbox 失败，携带 CODEX_SANDBOX_CREATION_FAILED / CODEX_OPENSANDBOX_UNAVAILABLE / CODEX_OPENSANDBOX_TIMEOUT（src/services/sandbox/codex.rs:358,804,825）。

即：看到 502 时，真实原因是沙箱创建 / OpenSandbox 侧故障，不是容量不足。 生产观测到的「约 11.1 秒后 502」也符合「尝试创建沙箱后失败」而非瞬时容量拒绝。

5f51f6d0 已让非 2xx 保留 body 并提取 allowlisted 上游码，但该码此前只影响 Failure.Code、不影响 Kind，而验收稳定码由 Kind 派生，所以真实 stage 在验收侧不可见。

已修复

0d900b66f（已推 feature/integrate）：存在 allowlisted 上游码时由该码决定 Kind —— sandbox creation / workspace preparation / OpenSandbox unavailable → ProvisioningFailed；各类 timeout → TimedOut；capacity → CapacityUnavailable；无上游码时保持按状态回退。不新增 fallback、不放宽 identity/credential 契约、不改请求形态。

另修正两条把该误标固化成契约的既有断言（CODEX_SANDBOX_CREATION_FAILED 与 CODEX_WORKSPACE_PREPARATION_FAILED 原本被断言为 CapacityUnavailable）。

验证：ChronoSandbox.Tests 222 passed（新增 6 例 theory）、AI.Tests 2480 passed、Integration.Tests 486 passed / 4 skipped、architecture + stability guards 通过。

对本 issue 的意义

如果本 issue 记录的 code_execute 502 也经由 managed codex 通道归因过，那条「容量」结论需要重新审视——502 指向的是沙箱创建/OpenSandbox，与 SOURCE_CREDENTIAL_REQUIRED 和 Orleans forwarding failure 是否同源需要用新的 typed stage 重新观测。部署包含 0d900b66f 的镜像后，同类失败会给出可区分的 ProvisioningFailed / TimedOut / CapacityUnavailable。

🤖 Generated with Claude Code

[jianwei-su]

Re-tested today, 2026-08-11, around 14:00 SGT. Two of the three problems in this issue are fixed. The third has changed shape twice and still blocks both workflows.

First, thank you for the 502 attribution correction — we had taken the "capacity" label at face value and reasoned from it, so knowing that a 502 points at sandbox creation / OpenSandbox rather than capacity is genuinely useful. We had no way to see that from the outside.

Fixed — confirmed by live test

Orleans message forwarding. The 27-step workflow that previously died at dispatch with Forwarding failed: tried to forward message Request […] now reaches its first step and fails there instead. Dispatch itself is working.

Admission. workflows:explicit-request-preview accepts the definition again and returns all 6 call sites; the subsequent bind was accepted. This is the same definition that was rejected five times on Aug 7.

One thing worth flagging: the preview response shape changed. It used to return a callSites array and now returns items. Our tooling keyed on callSites and read the new response as a failure until we looked at the raw body. If that was intentional, it is a breaking change for anyone parsing the response; if not, it may be worth a look.

Still blocking — and it is a new error

Both workflows now fail on their first code_execute step:

tool 'code_execute' execution failed:
code_execution_route_unavailable: The code execution service is not uniquely available.

• 27-step workflow: 0/27, fails at step 1
• 14-step workflow: 1/14, fails at step 2

"Not uniquely available" reads like an ambiguity rather than an outage — as if the router sees more than one candidate and cannot choose.

We have ruled out our side of that ambiguity. Our NyxID account has exactly one sandbox service, on a single endpoint, with no duplicate registration among our 37 connected services. We cannot enumerate what the scope sees on your side — the scope-level service and capability endpoints all return 404 to us — so if there is a duplicate candidate, it is in aevatar's own registry.

The error has changed twice in four days

When	Trigger	Error
Aug 7, interactive	manual invoke	NYXID_PROXY_HTTP_502
Aug 10 09:00, scheduled	weekly cron	code_execution_credential_unavailable: A source-readable NyxID credential is required for code execution.
Aug 11, interactive	manual invoke	code_execution_route_unavailable: The code execution service is not uniquely available.

The middle row may be worth separating out. That run was fired by the scheduler, so there was no interactive caller whose credential could be forwarded. If code execution resolves its credential from the forwarded caller token, scheduled runs have no such token by construction, and the weekly schedule would keep failing even after the routing problem is fixed. We would rather raise that now than discover it later.

Control, run in the same window

Calling the sandbox directly through the NyxID proxy works normally:

{"success": true, "output": {"stdout": "42", "exit_code": 0, "execution_time_ms": 1167}}

So the sandbox is healthy and the NyxID path to it is healthy. What cannot resolve is aevatar's route to it.

Where that leaves us

The migration itself is finished and independently reconciled; both workflows are waiting purely on code execution. Happy to re-run either one on request once a fix lands — we can turn a verification around quickly, and the definitions have not changed.

Scope, member, run and revision IDs available privately on request.

[nwnwnw413]

Independent reproduction of the third symptom, on a different scope. We hit the identical error today with no contact between our two migrations, which should rule out anything account-specific.

Scope 2c5c9b72-5b63-4c57-89bc-5d5492efebc3. All times UTC, 2026-08-11.

code_execution_route_unavailable: The code execution service is not uniquely available.

Smallest possible repro

A one-step workflow with no external call at all — pure JS arithmetic:

- id: probe_js
  type: tool_call
  parameters:
    tool: code_execute
    arguments:
      language: javascript
      code: |
        console.log(JSON.stringify({ok: true, sum: 1 + 1}));

Workflow probe_code_exec, revision rev-e3cf718d-63e3-4f5d-82c9-e6fd03c3b396, still bound in our scope. Four consecutive runs, identical result, 670–3300ms each:

def22b6e631946cc908b4b2185b3e04e · c710a9f6e75b4f97939f3ffe498090f4 · d41fb857fb5a4103bbbcc5c260c03f13 · a6e8b5a0aeb54736a57950dd3c4e7946

Full response body:

{"success":false,"error":"code_execution_route_unavailable",
 "code":"code_execution_route_unavailable",
 "message":"The code execution service is not uniquely available.",
 "diagnostic_id":null}

Same failure in our real workflow: community_radar rev rev-13c497d0-65c6-4d74-850c-a5539bf22f2e, run aa5ea0e18f4540c5b591f60924e40a81 (started 06:43:09.553). Step parse_group1 ran 06:44:38.926 → 06:44:42.255. read_seen plus four nyxid_proxy GET fetches in that same run all succeeded first — so the run credential is fine for nyxid_proxy and fails only for code_execute.

Same conclusion on the ambiguity, reached independently

We also have exactly one code-execution-capable service — chrono-sandbox = 2716aa5a-5ece-4e8f-a238-6e39e8b03020, active, no duplicate among our connected services. So there is no second candidate on our side either.

Given that, "not uniquely available" reading as ambiguity may be worth questioning: with exactly one candidate visible to each of two unrelated accounts, resolving to zero fits the evidence at least as well as resolving to many. That is an inference from the outside, not a claim about your resolver.

Our chrono-sandbox shows Auth: none / Credential: none and access: members, so we cannot alter its delegation configuration ourselves.

Control, same window

$ nyxid proxy request chrono-sandbox "/health"
{"status":"healthy","opensandbox_connected":true}

Sandbox healthy, NyxID path healthy, aevatar's route to it is what fails — matching what @jianwei-su reported above.

Cross-reference

We filed #3284 for the earlier NYXID_PROXY_UNAUTHORIZED form of this on Aug 7; it was fixed by image cb04697d, and the failure then became this one. Our re-test is in #3284 (comment). Given the error progression NYXID_PROXY_UNAUTHORIZED → code_execution_route_unavailable on our side and 502 → code_execution_credential_unavailable → code_execution_route_unavailable on @jianwei-su's, these look like the same underlying resolver work in progress, plausibly #3328's delegation-only migration. If so, it would help to know which step of #3328 makes it safe to re-test.

We can re-run the one-line probe as a canary in under a minute whenever it is useful.

cc @eanzhao

[eanz17]

Fixes for the remaining code_execute blocker are now deployed to mainnet in image aelfdevops/aevatar-console-backend:fd30ce36.

Included fixes:

• 620cb2674 - select and freeze the exact catalog-backed code_execute UserService during v5 admission, including personal routes and organization routes allowed for the caller.
• e15008cda - authorize scheduled code execution from the admitted durable route.
• 783a4d3fa - use the correct delegation credential kind for scheduled workflow dispatch.
• f1644e6bb - preserve the admitted delegation credential through scheduled execution.

Production verification on the deployed image:

• Ran a one-step JavaScript code_execute canary through the authenticated NyxID proxy path.
• Run suffix: cbca4e5f2e954860b9cde203a5bb6375.
• The v5 admission proof contained one exact catalog-backed chrono-sandbox UserService ID.
• Result: success=true, exit_code=0, stdout {"ok":true,"sum":2}, execution time 1141 ms.
• Committed read model: lastSuccess=true, stateVersion=17.
• Recent production logs contain no code_execution_route_* or code_execution_credential_unavailable rejection.
• The temporary canary workflow was archived after verification.

Verification at the deployed revision:

• Route and execution-port tests: 47/47 passed.
• Scheduled durable credential mapping test: 1/1 passed.
• Additional NyxIdCodeExecuteToolTests and NyxIdCodeExecutionWorkflowCapabilitySourceTests: 37/37 passed.
• tools/ci/test_stability_guards.sh: passed.

Please re-run the existing interactive one-line probe on both affected accounts. For scheduled execution, please re-preview and re-bind on the current image before testing the schedule: delegation-only scheduled execution requires the exact v5 admission proof produced by the new admission path.

I am not claiming a live scheduled-run success yet. My schedule canary was rejected with HTTP 400 before any schedule or run was created because that temporary workflow had interactive rather than durable admission. The scheduled path is covered by the deployed implementation and tests, but it still needs reporter-side validation against a correctly re-bound durable workflow.

I am keeping this issue open until both affected routes are confirmed, especially the organization-inherited route. The preview response currently intentionally serializes the collection as items; the earlier callSites compatibility concern was not changed as part of this fix.

[jianwei-su]

Re-ran on the deployed image as asked. The interactive path still fails for us, and the reason looks specific: our sandbox service is not catalog-backed.

All times SGT (UTC+8), 2026-08-12.

What we did, in order

Step	Result
09:43 — invoke the existing binding (pre-deploy)	ran to 1/2, failed at code_execute with code_execution_route_unavailable
11:08 — invoke the same binding (post-deploy)	INVALID_SCOPE_SERVICE_INVOKE_REQUEST: Failed to initialize Orleans actor scope-workflow:…
11:1x — re-preview explicit-request-preview	succeeded, 6 call sites, new revision issued
11:1x — re-bind with the fresh confirmations	rejected

The re-bind rejection:

STUDIO_MEMBER_EXTERNAL_CAPABILITY_NOT_READY
readiness.status: contract_drift
blocker:     CODE_EXECUTION_ROUTE_POLICY_MISMATCH
             "The canonical platform code execution route policy is incompatible."
remediation: refresh_source — "Restore platform code route"

So the old binding is correctly invalidated by the new image, re-preview works, and the new admission path rejects us at bind time.

Why we think it is the catalog-backed requirement

Your note says admission now selects and freezes the exact catalog-backed code_execute UserService. We checked our side:

• We have exactly one code-execution service: chrono-sandbox, active, Auth: none, Credential: none, endpoint https://sandbox.chrono-ai.fun/. No duplicates among our 37 connected services.
• chrono-sandbox is not in the NyxID catalog. The catalog lists 31 services; chrono-llm is there, chrono-sandbox is not. Ours was added as a custom HTTP service, not from a catalog entry.

If admission now requires the selected code-execution UserService to resolve to a catalog entry, then an account whose sandbox was added as a custom service has no way to satisfy it — which would match the contract_drift classification and the "Restore platform code route" remediation, neither of which we can act on from our side.

That would also explain why your canary passed and ours does not: if your sandbox is catalog-backed and ours is not, the deployed fix is correct but does not cover this shape of account.

We are not certain this is the cause — it is the one difference we can see from outside. If you can tell from the admission proof which UserService was selected (or why none was), that would settle it quickly.

What would unblock us

Either of these, whichever matches the intended design:

1. Add chrono-sandbox to the NyxID catalog so our service can be catalog-backed, or
2. Have admission accept a custom (non-catalog) code-execution UserService when it is the caller's only one.

One more observation, possibly unrelated

The two access paths have swapped behaviour across the last day, which made diagnosis harder:

	Aug 11 evening	Aug 12 morning
Direct call with the NyxID bearer	scope endpoints 401	200, scopeResolved: true
Through the NyxID proxy	worked	scope endpoints 401, studio/context returns scopeId: null, scopeResolved: false

Same token, valid in both windows. We mention it because your canary ran "through the authenticated NyxID proxy path", which is currently the failing path for us — so the two of us may not be exercising the same route.

Happy to re-run anything on request; the workflow and bindings are unchanged and a retest takes a few minutes.

[jianwei-su]

Following up on the catalog point, because it is the kind of claim that is easy to get wrong and I wanted to rule out the obvious false positive before you spend time on it.

The obvious objection, tested and ruled out

nyxid catalog list prints under the heading "Available Services" with a "How to Add" column. That phrasing suggests it might only list services the caller has not added yet — in which case chrono-sandbox would be missing from the listing simply because we already have it, and my earlier conclusion would be wrong.

It does not work that way. Services we have already added still appear in the catalog listing:

Slug	Instances we have added	Present in catalog listing
api-lark-bot	3	yes
llm-openai	2	yes
api-lark	2	yes
chrono-sandbox	1	no

So the listing is not filtered by what the caller already has, and chrono-sandbox's absence is real rather than a display artifact. The only chrono-* entry in the catalog is chrono-llm.

What our sandbox actually is

Service:    Chrono Sandbox (chrono-sandbox)
Type:       http
Status:     active
Auth:       none
Credential: none
Endpoint:   https://sandbox.chrono-ai.fun/
OpenAPI:    https://sandbox.chrono-ai.fun/openapi.json

It was added as a custom HTTP service, not from a catalog entry. It is our only code-execution-capable service, with no duplicates among 37 connected services.

How this lines up with the deployed fix

Your change selects and freezes the exact catalog-backed code_execute UserService during v5 admission. If our sandbox has no catalog entry behind it, there is nothing for that selection to resolve to — which would produce exactly the contract_drift / CODE_EXECUTION_ROUTE_POLICY_MISMATCH we now get at bind time, and the "Restore platform code route" remediation we cannot act on.

It would also explain the asymmetry: your canary passes because your sandbox is catalog-backed; ours cannot, because it never was.

Still a hypothesis — we can only see our side. What would confirm or kill it in one step is the admission proof for our rejected bind: which UserService did the selector consider, and why was none accepted.

Two possible resolutions

1. NyxID side — add chrono-sandbox to the catalog the way chrono-llm is, after which we re-add our service so it is catalog-backed.
2. aevatar side — have admission accept a custom (non-catalog) code-execution UserService when it is unambiguously the caller's only one.

We have no preference; whichever matches the intended design. Option 1 needs someone with catalog write access, so it is not something we can do ourselves.

Ready to retest within minutes of either landing — the workflow and bindings are unchanged, and we have an automated check that re-binds and re-runs on its own.

[eanz17]

Thanks for the immediate retest. I correlated your bind request at 2026-08-12 11:15:11 SGT with production logs.

The server-side evidence is:

GET /api/v1/user-services -> 200
failureKind=PolicyMismatch
canonicalCandidates=1
accessibleCandidates=1
activeCandidates=1
eligibleCandidates=0

This rules out the non-catalog hypothesis. If the UserService had no catalog_service_id, Aevatar would have reported CODE_EXECUTION_ROUTE_MISSING with canonicalCandidates=0. Aevatar is seeing exactly one catalog-backed chrono-sandbox route for your caller, but that route's stored delegation policy is incompatible with the platform contract.

The deployed resolver intentionally accepts both supported transition shapes:

• forward_access_token=true, inject_delegation_token=true, scope proxy:* or proxy:* sandbox:execute; or
• forward_access_token=false, inject_delegation_token=true, scope sandbox:execute or proxy:* sandbox:execute.

Your route matches neither shape. This is consistent with a stale historical NyxID UserService identity-policy snapshot, not a missing catalog entry and not a reason to allow an arbitrary custom same-slug endpoint.

NyxID already has the generic, audited recovery path for this exact class of drift from NyxID#1378, merged in NyxID#1379:

POST /api/v1/services/15875e8d-abb0-4be8-92ef-5fd753ad50f0/resync-identity

That endpoint is platform-admin-only and force-aligns all UserService instances for the chrono-sandbox catalog entry to the catalog's current identity policy, with an audit record and affected count. Before running it, the operator must verify the current catalog policy; after it succeeds, please re-preview and re-bind once more.

I have not claimed or performed that production mutation from a normal user credential. The issue remains open until the platform-admin resync is completed and both reporter accounts pass the interactive probe. The separate NyxID-proxy 401 / scopeResolved=false observation is a different ingress/identity boundary and should not be inferred from this policy-mismatch result.

[jianwei-su]

Correction: I was wrong. chrono-sandbox IS in the NyxID catalog. Please disregard my two previous comments on this point — the NyxID owner is right.

What I got wrong

I ran nyxid catalog list, did not find chrono-sandbox, and concluded it had no catalog entry. That command hides it by design:

nyxid catalog list --help
  --all    Include all active services (including system services without auth)

chrono-sandbox has Auth Method: none, so it is a system service without auth and is omitted from the default listing. With the flag it is there:

$ nyxid catalog list          →  31 entries, no chrono-sandbox
$ nyxid catalog list --all    →  36 entries, chrono-sandbox present
$ nyxid catalog show chrono-sandbox
    Service: Chrono Sandbox
    Slug:            chrono-sandbox
    Type:            http
    Default URL:     https://sandbox.chrono-ai.fun/
    Auth Method:     none
    Credential Mode: admin

Worse, I then "ruled out" the obvious objection with a table showing api-lark-bot, llm-openai and api-lark appearing in the listing despite being already added. That table was correct but irrelevant: those three all have auth methods, so the no-auth filter never applied to them. I tested the wrong hypothesis and reported the result as if it settled the question.

So the catalog-backed theory is dead, and with it the two remediations I proposed. Nothing needs to be added to the catalog.

Where that leaves the actual problem

The failure itself is unchanged and still reproduces. On the deployed image, today:

Step	Result
invoke the pre-deploy binding	INVALID_SCOPE_SERVICE_INVOKE_REQUEST: Failed to initialize Orleans actor
re-preview	succeeds, 6 call sites, new revision
re-bind with fresh confirmations	CODE_EXECUTION_ROUTE_POLICY_MISMATCH, readiness.status: contract_drift

Our side, restated without the bad inference:

• Exactly one code-execution service, chrono-sandbox, active, and it does have a catalog entry (Credential Mode: admin, Auth Method: none).
• No duplicates among 37 connected services.
• The sandbox answers directly through the NyxID proxy: success=true, exit_code=0, ~1.1s.

I am not going to offer another theory. The one thing that would identify the cause directly is the admission proof for our rejected bind — which UserService the selector considered, and the specific reason each was rejected. If that can be surfaced, or if you can read it server-side for our scope, it should be a short step from there.

Apologies for the detour, and for stating a hypothesis with more confidence than the evidence supported.