-
Notifications
You must be signed in to change notification settings - Fork 113
CI hygiene: fail loudly, gate the merge queue, log on failure #876
Copy link
Copy link
Closed
Labels
consumerBlocks consumer adoption — must ship for the v0.20.0 consumer launch windowBlocks consumer adoption — must ship for the v0.20.0 consumer launch windowdevopsDevOps/infrastructure changesDevOps/infrastructure changesdomain:qualityTests, CI/CD, security, performance, evalsTests, CI/CD, security, performance, evalsp0high priorityhigh prioritytech debttier-0Tier 0 — must ship before parallel agent work begins (test/CI prerequisites)Tier 0 — must ship before parallel agent work begins (test/CI prerequisites)track:platformFoundation that both consumer-app and oem-pc tracks consumeFoundation that both consumer-app and oem-pc tracks consume
Milestone
Metadata
Metadata
Assignees
Labels
consumerBlocks consumer adoption — must ship for the v0.20.0 consumer launch windowBlocks consumer adoption — must ship for the v0.20.0 consumer launch windowdevopsDevOps/infrastructure changesDevOps/infrastructure changesdomain:qualityTests, CI/CD, security, performance, evalsTests, CI/CD, security, performance, evalsp0high priorityhigh prioritytech debttier-0Tier 0 — must ship before parallel agent work begins (test/CI prerequisites)Tier 0 — must ship before parallel agent work begins (test/CI prerequisites)track:platformFoundation that both consumer-app and oem-pc tracks consumeFoundation that both consumer-app and oem-pc tracks consume
Type
Fields
Give feedbackNo fields configured for issues without a type.
Part of #875 (Tier 1).
Goal
Restore CI signal integrity. Today, multiple workflows swallow failures, lack timeouts, delete logs, and only two workflows actually gate the merge queue.
Why
Per
CLAUDE.md's "no silent fallbacks — fail loudly" rule, hiding failures at the workflow layer is the same anti-pattern as catching exceptions in code. Contributors learn to ignore CI red.Scope (single PR)
A. Stop swallowing failures
.github/workflows/test_code_agent.yml:98,107,120,128,143,153,179— remove|| trueand[WARNING] non-blocking for nowechoes; mark genuinely-broken testsxfailwith reason.github/workflows/test_gaia_cli_linux.yml:169,183,198— replace|| TEST_EXIT=$?+echowith properexit $TEST_EXIT.github/workflows/test_mcp.yml:201,226— drop|| truefromgaia mcp testandgaia mcp stop.github/workflows/build_cpp.yml:374-379— promote integration/benchmark::warning::to job failure (or move to a separate non-required job with explicit naming).github/workflows/test_sd.yml:46,80,116,148— remove threecontinue-on-error: trueblocks; if SD requires Lemonade ≥ 9.2.0, fail loudly with an actionable error instead of silentlySKIP_SD_TESTS=trueB. Tighten the merge-queue gate
.github/workflows/merge-queue-notify.yml:18-21— expand theworkflow_run.workflowswatch list to include the integration suites that should block merges, OR wiretest_gaia_cli.yml's existingtest-summary(line 84) as the single required check by adding the rightpull_requesttrigger and removing theworkflow_call-only constraintdocs/reference/dev.mdxC. Add timeouts and log uploads
timeout-minutesto:lint.yml,test_unit.yml:42,test_chat_agent.yml:46,test_code_agent.yml:40,test_security.yml:42,133,test_rag.yml:39,73,pypi.yml:16. Default ≤ 60.if: failure()artifact upload oflemonade.log, pytest XML, and any agent stdout to:test_gaia_cli_linux.yml,test_chat_agent.yml,test_code_agent.yml,test_security.yml,test_unit.ymlrm -f lemonade.login.github/workflows/test_gaia_cli_linux.yml:213D. Re-enable Dependabot
.github/dependabot.ymllines 21, 37, 57, 74, 89, 106, 121 — change all sevenopen-pull-requests-limit: 0to a real number (suggest 5)assignees: [kovtcharov-amd]to security ecosystemsE. Post-publish smoke
.github/workflows/publish.ymlthat runs on a fresh hosted Ubuntu after the PyPI upload completes:pip install gaia==<published-version>+gaia --version+gaia --help. Fail the workflow (and ideally roll back the GitHub release tag) if this fails.Acceptance criteria
grep -RE '\|\| true|TEST_EXIT' .github/workflows/test_*.ymlreturns 0 hitsOut of scope