docs(gap-baseline): measure PR job fan-out, disprove the workflow-waste hypothesis - #1905
Open
seonghobae wants to merge 3 commits into
Open
Conversation
…te hypothesis Static analysis of all 35 workflows (matrix expansion + three-valued if: evaluation, evaluator checked against 7 real conditions first): one PR synchronize dispatches 33 job slots, not the 36 a naive count reports. Every candidate reduction was checked and rejected on evidence: 12 slots are required status-check contexts (live branch protection), all 12 push triggers are already restricted to main/master/develop so no PR-branch duplication exists, every small quality-CI workflow already carries paths filters, and adding paths-ignore to sast-semgrep/python-security is explicitly forbidden by tests/test_docs_only_pr_runner_admission.py. The changed-scope gate jobs are documented deliberate design, not waste. Conclusion: queue depth is 76-repo ruleset fan-out (where on: filters are discarded) against a shared concurrency ceiling -- observed in_progress 0-1 for 3+ hours while queued climbed 372->424 with GitHub reporting all systems operational. Further .github YAML optimisation is not a productive lever; the open question is org-level Actions capacity, readable only with admin:org. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…odex-found exception An independent Codex audit, run against the same repo under the same constraints but without knowledge of the first pass's conclusion, found one genuine redundant slot the first pass missed: opencode-review.yml's coverage-source-tree is a full runner slot whose whole body is one echo, and which exists only to carry needs/if: into the required coverage-evidence context (itself another echo-only job). Recorded, not shipped: it is 1 slot of 33 (3%) in a queue measured 424 deep against near-zero execution, and collapsing it edits the job graph of a required review workflow the ruleset injects into 76 repositories. The capacity-bound conclusion is unchanged. The first pass classified both jobs as load-bearing gate jobs by pattern- matching the documented changed-scope design without opening them to confirm they do any work -- kept as a method note, since two agents re-reading one agent's evidence is not corroboration. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
3 tasks
… dimension This entry counted runner slots and concluded reclaiming 1 of 33 could not matter in a capacity-bound queue. Wrong. The two echo-only jobs sit in series on the review critical path, and a job is not created until its needs: complete, so each link waits out the whole queue again. Measured on naruon#1528: 9h40m and 13h01m of queue wait for 4s and 5s of runtime, ~22h41m total holding the real review behind it, on every PR across 76 repositories. Slot count made that look like 3%; critical-path latency is the dimension that governs how long a PR actually takes to clear. Fix shipped as #1910 (serial depth 5 -> 3, queue waits 4 -> 2). Also records the two non-obvious safety conditions: coverage-evidence had no if: of its own and relied on transitive skipping, and opencode-review-dispatch .yml has same-named jobs whose edge is a real artifact dependency -- job names are unique only within a workflow file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
[P2] 확인된 낭비와 모순되는 최종 감사 결론을 수정해 주세요검토 HEAD는 직접 확인한 반례는 다음과 같습니다.
최소 수정은 기존 관측표를 유지하면서 제목·첫 문단·마지막 결론을 “관측한 trigger 범위와 남은 구조 감사”로 좁히는 것입니다. runner 용량 확인과 workflow 낭비 제거는 병행할 수 있습니다. 정적 분석기와 실행 명령이 PR에 없으므로 33이라는 수치는 재현 가능한 산출물을 연결하기 전에는 독립 검증된 하한으로 쓰지 말아 주세요. 현재 #1899의 이는 사용자 요청을 더 작은 목표로 바꾸지 않기 위한 문서 정확성 finding입니다. org 설정·보안 gate 변경이나 추가 모델 timeout을 요청하지 않습니다. |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed.
Findings
1. HIGH Current-head GitHub Checks - Fix failed required checks before approval
- Problem: Failed same-head checks remain for
f7f480a820b1a8278eb4537943b0fcf830b9e97c. - Root cause: The model-unavailable evidence fallback is allowed only when peer GitHub Checks are complete and clean.
- Fix: Read and fix the failed check logs below, then rerun the current-head checks.
- Regression test: Keep the model-unavailable fallback gated on an empty failed-check rollup.
Failed checks:
- Required Noema Review/noema-review: FAILURE (https://github.com/ContextualWisdomLab/.github/actions/runs/33955810883/job/101306378575)
- noema-review check run: failure (https://github.com/ContextualWisdomLab/.github/actions/runs/33955810883/job/101306378575)
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Docs: product-technical-gap-baseline.md"]
S1 --> I1["operator or user guidance"]
I1 --> R1["Review risk: Docs: product-technical-gap-baseline.md"]
R1 --> V1["docs review"]
Contributor
OpenCode Review Overview
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #1903 (same file, avoids a guaranteed append-conflict).
The standing directive has repeatedly asked to find workflows that trigger unnecessarily, on the hypothesis that gratuitous triggering inside
.githubis what fills the org's Actions concurrency ceiling. I measured it directly. The hypothesis does not hold — and the negative result is worth recording so no future session re-runs this search.Method: static analysis of all 35 workflows — resolve
on:blocks, expand statically-enumerable matrices (each combination takes its own runner slot), and evaluate every job'sif:with a three-valued true/false/unknown evaluator that counts a job as running unless provably false, so it never under-reports. The evaluator was validated against 7 real conditions fromnoema-review.yml,opencode-review.yml,strix.yml, andrepository-metadata-reconcile.ymlbefore its output was trusted.Result: one PR synchronize dispatches 33 job slots (a naive count says 36; three jobs are provably skipped for a synchronize event).
Every candidate reduction checked and rejected on evidence:
main/master/develop, so no push workflow fires on a PR branch.cloudflare-dns.yml.paths-ignoreonsast-semgrep.yml/python-security.yml— the one apparent remaining lever — is explicitly forbidden bytests/test_docs_only_pr_runner_admission.py, because ruleset18156473discardson:filters in target repos and.github's classic branch protection would strand a filtered required context.changed-scope/admit-current-headgate jobs are documented, contract-tested deliberate design, not waste.What actually explains the queue: 76 repositories × ruleset-injected central workflows (with
on:filters discarded in target repos) × open PRs, against one shared ceiling. Observed the same day:in_progress0–1 for 3+ hours whilequeuedclimbed 372 → 424, with githubstatus.com reporting all systems operational — the shape of a capacity ceiling, not of workflow waste.Consequence: further
.githubYAML optimisation is not a productive lever. The open question is org-level Actions capacity, readable only with anadmin:orgcredential no agent session holds (gh api orgs/.../rulesets/18156473returns a scope error). That is an owner action.Documentation only — no code or workflow changes.
Test plan
pytest tests/test_product_technical_gap_baseline.py— 5 passed🤖 Generated with Claude Code