Desktop core E2E harness, tiered testing, and release blessing gate

[Git-on-my-level]

Summary

• Adds a tiered desktop core E2E harness (desktop-core-harness.sh, CORE_E2E.md, flow lint, T0 CI self-check) so agents can validate real app behavior via the automation bridge — not just compile/unit tests.
• Introduces hermetic T2 testing with PROVIDER_MODE=offline, Rust OMI_LLM_STUB, capture/chat/memories/tasks bridge seams, and falsifiable YAML flow assertions (12 flows + spatial overlay verified green locally).
• Adds release blessing: bless-release.sh runs T2 against a rebuild tag, writes blessed metadata to GitHub release KEY_VALUE blocks; prod promotion (desktop_promote_prod) is gated unless --override-unblessed + confirmation string; admin Releases page shows blessing status.

Test plan

• ./scripts/desktop-core-harness.sh --self-check --skip-backend-contracts (T0)
• PROVIDER_MODE=offline make dev-up + make desktop-run-local DESKTOP_APP_NAME=omi-core-e2e DESKTOP_USER=alice
• ./scripts/desktop-core-harness.sh --tier 2 --bundle omi-core-e2e --port <automation-port> --keep-stack → passed: true, provider_mode: offline
• python3 scripts/release-keyvalue.py check-manifest accepts good manifest, rejects missing provider_mode: offline
• python3 .github/scripts/test_check_desktop_release_promotion.py (bless gate unit tests)
• Pre-push hooks: Swift compile, Rust compile, changelog validation, prod promotion policy check

Notes

• T2 requires Docker/OrbStack for Typesense; dev-up now restarts Typesense when the container dies but manifest PID lingers.
• Bless runs are intended 1–2×/day on agent-local macOS runners, not per-PR.
• Prod override requires override_unblessed=true and override_confirm=I-ACCEPT-UNBLESSED-PROD-RISK.

Made with Cursor

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a9cc72e705

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Honor non-stream chat completion requests in the stub

When OMI_LLM_STUB=1, this stub always returns text/event-stream, even when req.stream is false. Non-stream desktop callers such as AgentDelegationResolver.swift and AgentPill.swift send "stream": false and parse a JSON choices[0].message.content response, so the offline/T3 harness paths that exercise those callers will get HTTP 200 with an SSE body and fail with an unexpected response shape. Return an OpenAI-compatible JSON completion for non-stream requests.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Use the actual named-bundle id for T3 gauntlet

For any required omi-* bundle, the app-config logic derives the bundle id as com.omi.<full slug> (for example omi-core-e2e becomes com.omi.omi-core-e2e). Stripping the omi- prefix here makes the T3 gauntlet target com.omi.core-e2e, which will not match the running named bundle, so tier-3 harness runs fail to connect to the app even though the bridge is healthy.

Useful? React with 👍 / 👎.

[cubic-dev-ai]

22 issues found across 56 files

Confidence score: 2/5

• desktop/macos/Desktop/Sources/DesktopAutomationBridge.swift (reset_main_chat) can erase a signed-in user’s main chat when automation bridge is enabled on a production bundle, which is a direct user-data loss risk if merged as-is — add the same non-prod gating used by adjacent harness-only actions before allowing reset.
• desktop/macos/Desktop/Sources/AppState/AppState+Transcription.swift allows already_recording to proceed through inject/stop, so harness capture can interfere with a real in-progress recording and tear it down unexpectedly — track harness-owned sessions (or hard-stop on preexisting recordings) before merge.
• Release promotion integrity is currently weak across desktop/macos/scripts/release-keyvalue.py and .github/workflows/desktop_promote_prod.yml: blessing can accept non-T2 manifests, malformed metadata blocks can appear successful, and blessed_override=true can bypass blessedSha commit mismatch checks — tighten these validations so only correctly tiered, well-formed metadata tied to the target commit can be promoted.
• desktop/macos/scripts/desktop-flow-lint.py checks for a non-existent do: key, so typed flows can silently include legacy do steps and pass lint, reducing confidence in flow correctness — fix the key detection to do and re-run lint on typed flows before merging.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="desktop/macos/Backend-Rust/src/routes/chat_completions.rs">

<violation number="1" location="desktop/macos/Backend-Rust/src/routes/chat_completions.rs:616">
P2: Stub always returns SSE even for non-streaming requests — `req.stream` is ignored. A client sending `stream: false` or omitting `stream` gets `Content-Type: text/event-stream` with SSE line events instead of the expected JSON `ChatCompletionResponse`. Consider checking `req.stream` in the stub: when `false`, return a single-chunk JSON response matching the real handler's non-streaming shape.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

4 issues found across 19 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="desktop/macos/Backend-Rust/src/routes/chat_completions.rs">

<violation number="1" location="desktop/macos/Backend-Rust/src/routes/chat_completions.rs:616">
P2: Stub always returns SSE even for non-streaming requests — `req.stream` is ignored. A client sending `stream: false` or omitting `stream` gets `Content-Type: text/event-stream` with SSE line events instead of the expected JSON `ChatCompletionResponse`. Consider checking `req.stream` in the stub: when `false`, return a single-chunk JSON response matching the real handler's non-streaming shape.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

1 issue found across 10 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/lint.yml">

<violation number="1" location=".github/workflows/lint.yml:92">
P3: Two assertions in `test_unblessed_release_fails` lack failure context, making CI failures harder to debug. If the check script's output format or exit code changes, the failed assertion produces only `AssertionError` without showing what the actual stdout/stderr contained. The other two test functions already include `result.stderr` in their assertions for this reason.</violation>
</file>

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

P3: Two assertions in test_unblessed_release_fails lack failure context, making CI failures harder to debug. If the check script's output format or exit code changes, the failed assertion produces only AssertionError without showing what the actual stdout/stderr contained. The other two test functions already include result.stderr in their assertions for this reason.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/lint.yml, line 92:

<comment>Two assertions in `test_unblessed_release_fails` lack failure context, making CI failures harder to debug. If the check script's output format or exit code changes, the failed assertion produces only `AssertionError` without showing what the actual stdout/stderr contained. The other two test functions already include `result.stderr` in their assertions for this reason.</comment>

<file context>
@@ -89,6 +89,9 @@ jobs:
       - name: Check desktop prod promotion policy
         run: python3 .github/scripts/check-desktop-prod-promotion-policy.py
 
+      - name: Test desktop release promotion gate
+        run: python3 .github/scripts/test_check_desktop_release_promotion.py
+
</file context>

[cubic-dev-ai]

1 issue found across 6 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="desktop/macos/Backend-Rust/src/routes/chat_completions.rs">

<violation number="1" location="desktop/macos/Backend-Rust/src/routes/chat_completions.rs:616">
P2: Stub always returns SSE even for non-streaming requests — `req.stream` is ignored. A client sending `stream: false` or omitting `stream` gets `Content-Type: text/event-stream` with SSE line events instead of the expected JSON `ChatCompletionResponse`. Consider checking `req.stream` in the stub: when `false`, return a single-chunk JSON response matching the real handler's non-streaming shape.</violation>
</file>

<file name=".github/workflows/lint.yml">

<violation number="1" location=".github/workflows/lint.yml:92">
P3: Two assertions in `test_unblessed_release_fails` lack failure context, making CI failures harder to debug. If the check script's output format or exit code changes, the failed assertion produces only `AssertionError` without showing what the actual stdout/stderr contained. The other two test functions already include `result.stderr` in their assertions for this reason.</violation>
</file>

<file name="desktop/macos/scripts/check-e2e-flow-coverage.py">

<violation number="1" location="desktop/macos/scripts/check-e2e-flow-coverage.py:181">
P2: The harness recommendation in `check-e2e-flow-coverage.py` prints a command that uses the old `omi-harness` runner and the old app name `omi-e2e`, but this PR introduces the `desktop-core-harness.sh` tiered harness and the `omi-core-e2e` bundle name. Developers relying on the coverage report will get stale or incorrect invocation instructions that don't align with the new desktop core E2E contract. Consider updating `harness_command` to generate a command that matches the new harness interface (e.g., `desktop-core-harness.sh` with `--bundle omi-core-e2e` or the appropriate tier/port).</violation>
</file>

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[Git-on-my-level]

@cursor address merge conflict

[cursor]

Merge conflict resolved and pushed.

Conflict: desktop/macos/e2e/flows/harness-smoke.yaml

main dropped the AuthService.swift covers entry; this branch added tier: 1 and kept that coverage. Resolution keeps both:

• tier: 1
• covers for DesktopAutomationBridge.swift and AuthService.swift

Also fixed the stale harness recommendation in check-e2e-flow-coverage.py (cubic review): it now suggests desktop-core-harness.sh --tier N --bundle omi-core-e2e instead of the legacy omi-harness / omi-e2e command. The coverage test was updated to match.

Verification: bash desktop/macos/tests/test-e2e-flow-coverage.sh passed.

Commit: 2986c1faa on cursor/1eaeca88, pushed to origin. The PR should be mergeable with main now.

  

[cubic-dev-ai]

2 issues found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/desktop-swift-ci.yml">

<violation number="1" location=".github/workflows/desktop-swift-ci.yml:25">
P2: For push events to `main`, the diff base is hard-coded to `HEAD~1` in both the path gate and the e2e coverage check. On a multi-commit push, this only looks at the last commit, so desktop changes in earlier pushed commits can be missed and the Swift build, tests, and e2e coverage validation will be skipped.

Use `github.event.before` as the diff base for push events so the full pushed range is evaluated.</violation>

<violation number="2" location=".github/workflows/desktop-swift-ci.yml:63">
P2: The CI build only validates `arm64-apple-macosx`, while the Codemagic release pipeline builds both `arm64` and `x86_64` and merges them into a universal binary. This means x86_64-specific compile or link failures won't be caught in CI and will only surface during the release build, which has historically caused issues (see the "Burned a build in May 2026" comment in the release pipeline). Consider adding an x86_64 build step or using the same build matrix the release pipeline uses.</violation>
</file>

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: The CI build only validates arm64-apple-macosx, while the Codemagic release pipeline builds both arm64 and x86_64 and merges them into a universal binary. This means x86_64-specific compile or link failures won't be caught in CI and will only surface during the release build, which has historically caused issues (see the "Burned a build in May 2026" comment in the release pipeline). Consider adding an x86_64 build step or using the same build matrix the release pipeline uses.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/desktop-swift-ci.yml, line 63:

<comment>The CI build only validates `arm64-apple-macosx`, while the Codemagic release pipeline builds both `arm64` and `x86_64` and merges them into a universal binary. This means x86_64-specific compile or link failures won't be caught in CI and will only surface during the release build, which has historically caused issues (see the "Burned a build in May 2026" comment in the release pipeline). Consider adding an x86_64 build step or using the same build matrix the release pipeline uses.</comment>

<file context>
@@ -0,0 +1,68 @@
+      - name: Build desktop Swift app
+        if: steps.changed.outputs.should_run == 'true'
+        working-directory: desktop/macos
+        run: xcrun swift build -c release --package-path Desktop --triple arm64-apple-macosx
+
+      - name: Test desktop Swift app
</file context>

[cubic-dev-ai]

P2: For push events to main, the diff base is hard-coded to HEAD~1 in both the path gate and the e2e coverage check. On a multi-commit push, this only looks at the last commit, so desktop changes in earlier pushed commits can be missed and the Swift build, tests, and e2e coverage validation will be skipped.

Use github.event.before as the diff base for push events so the full pushed range is evaluated.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/desktop-swift-ci.yml, line 25:

<comment>For push events to `main`, the diff base is hard-coded to `HEAD~1` in both the path gate and the e2e coverage check. On a multi-commit push, this only looks at the last commit, so desktop changes in earlier pushed commits can be missed and the Swift build, tests, and e2e coverage validation will be skipped.

Use `github.event.before` as the diff base for push events so the full pushed range is evaluated.</comment>

<file context>
@@ -0,0 +1,68 @@
+          if [ "${{ github.event_name }}" = "pull_request" ]; then
+            DIFF_BASE="origin/${{ github.base_ref }}"
+          else
+            DIFF_BASE="HEAD~1"
+          fi
+          FILES=$(git diff --name-only --diff-filter=ACMRD "$DIFF_BASE"...HEAD)
</file context>