From 997219fa0ed7b8cc796ecd6bc31af49135791ed7 Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Sun, 26 Jul 2026 16:34:43 -0400 Subject: [PATCH 1/6] CI: a PR re-push cancels that PR's superseded in-flight runs Concurrency groups for build.yml and sql-validation.yml, keyed per PR number for pull_request events only. Push and release runs get a unique per-run group (run_id), so dev/main integration builds, release builds, and every push commit's required-check result are never queued behind, replaced by, or cancelled through this mechanism. Rationale: the shared Windows runner pool is what serializes everyone's CI (#1697 sat queued behind two dev builds), so a stale PR run cancelled at re-push time is reclaimed capacity for whatever sits queued behind it. claude-review.yml already had exactly this shape; build.yml and sql-validation.yml now match it. Co-Authored-By: Claude Fable 5 --- .github/workflows/build.yml | 11 +++++++++++ .github/workflows/sql-validation.yml | 7 +++++++ 2 files changed, 18 insertions(+) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 32e09eb8..173e0cc0 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -13,6 +13,17 @@ permissions: id-token: write actions: read +# A re-push to a PR cancels that PR's superseded in-flight run: finishing a build of code +# that is no longer the PR's head helps nobody, and the shared Windows runner pool is what +# serializes everyone's CI (#1697 sat queued behind two dev builds), so a cancelled stale +# run is reclaimed capacity for whatever is queued. Push and release runs deliberately get +# a UNIQUE group per run (run_id): dev/main integration builds and release builds are never +# queued behind, replaced by, or cancelled through this mechanism — every push commit keeps +# its own reported check result, which release cuts and merge gating read. +concurrency: + group: ${{ github.event_name == 'pull_request' && format('build-pr-{0}', github.event.pull_request.number) || format('build-run-{0}', github.run_id) }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + jobs: build: runs-on: windows-latest diff --git a/.github/workflows/sql-validation.yml b/.github/workflows/sql-validation.yml index c6fc01fc..3a1ad8f9 100644 --- a/.github/workflows/sql-validation.yml +++ b/.github/workflows/sql-validation.yml @@ -8,6 +8,13 @@ on: branches: [dev] paths: ['install/**', '.github/sql/**', '.github/workflows/sql-validation.yml'] +# A re-push to a PR cancels that PR's superseded in-flight matrix — four SQL Server +# containers per run on the shared runner pool, validating code that is no longer the +# PR's head. Push runs get a unique per-run group (run_id) and are never cancelled. +concurrency: + group: ${{ github.event_name == 'pull_request' && format('sql-validation-pr-{0}', github.event.pull_request.number) || format('sql-validation-run-{0}', github.run_id) }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + jobs: validate-sql: runs-on: ubuntu-latest From 677c5a2753cdd7649be064483d48b680ca9dd6df Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Sun, 26 Jul 2026 16:35:15 -0400 Subject: [PATCH 2/6] CI: the docs allowlist is extension-explicit, and the version gate mirrors it Ratifies #1712's two flagged judgment calls (CITATION.cff and the Screenshots/ entry stay on the documentation allowlist - citation metadata and screenshot images cannot affect any build.yml job) but replaces the bare directory globs docs/** and Screenshots/** with extension-explicit patterns for markdown, SVG, and image files. Both directories hold exactly those types today, so nothing classified as documentation before this commit stops being documentation - but a .sql or script dropped into either directory tomorrow now defaults to being CODE, which restores the allowlist's own stated principle (an unfamiliar file type should default to the safe direction) that a directory-wide grant quietly undermined. check-version-bump.yml gets the same allowlist as exclusions: its gate still said *.md-only, so a LICENSE or .gitignore correction in a dev to main PR would demand a version bump while build.yml calls the same diff documentation. The two filters now state the same policy and say so. Co-Authored-By: Claude Fable 5 --- .github/workflows/build.yml | 14 ++++++++++---- .github/workflows/check-version-bump.yml | 13 +++++++++++-- 2 files changed, 21 insertions(+), 6 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 173e0cc0..e5259056 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -97,14 +97,20 @@ jobs: # *.yml (workflows), *.csproj / *.props / packages.lock.json (build inputs), # and *.cs regardless of how comment-only the change looks — an XML doc # comment still recompiles, and the compiler is what proves it still builds. + # + # The docs/ and Screenshots/ entries are extension-explicit rather than bare + # directory globs for the same reason: everything in them today is markdown, + # SVG, or a screenshot image, and a .sql or script dropped into either + # directory tomorrow should default to being code, not inherit a free pass + # from its parent directory. docs: - '**/*.md' - 'LICENSE' - 'CITATION.cff' - '.gitignore' - '.gitattributes' - - 'docs/**' - - 'Screenshots/**' + - 'docs/**/*.{md,svg,png,jpg,jpeg,gif}' + - 'Screenshots/**/*.{md,svg,png,jpg,jpeg,gif}' # True when any file OUTSIDE the documentation allowlist changed. A docs-only # change skips .NET setup, restore and versioning below — there is nothing to # compile — while the job itself still runs, so the required 'build' check @@ -122,8 +128,8 @@ jobs: - '!CITATION.cff' - '!.gitignore' - '!.gitattributes' - - '!docs/**' - - '!Screenshots/**' + - '!docs/**/*.{md,svg,png,jpg,jpeg,gif}' + - '!Screenshots/**/*.{md,svg,png,jpg,jpeg,gif}' # Decides the docs fast path ONCE, in one place, and says so out loud. Two guards # keep it off the paths where a skipped restore would be a real loss: diff --git a/.github/workflows/check-version-bump.yml b/.github/workflows/check-version-bump.yml index 226d267b..d64e3971 100644 --- a/.github/workflows/check-version-bump.yml +++ b/.github/workflows/check-version-bump.yml @@ -16,7 +16,10 @@ jobs: # version. The workflow still runs so the required check reports a result # — using paths-ignore would skip the run entirely and leave the check # stuck pending. Instead, the version comparison below is gated on whether - # any non-*.md file changed. + # any non-documentation file changed. The exclusions MIRROR build.yml's + # documentation allowlist (its docs: filter) — keep the two in step, or a + # LICENSE-only correction will demand a version bump here while build.yml + # calls it documentation. - name: Detect non-documentation changes id: changes uses: dorny/paths-filter@v4 @@ -25,6 +28,12 @@ jobs: code: - '**' - '!**/*.md' + - '!LICENSE' + - '!CITATION.cff' + - '!.gitignore' + - '!.gitattributes' + - '!docs/**/*.{md,svg,png,jpg,jpeg,gif}' + - '!Screenshots/**/*.{md,svg,png,jpg,jpeg,gif}' - name: Get PR version if: steps.changes.outputs.code == 'true' @@ -67,4 +76,4 @@ jobs: - name: Skip notice if: steps.changes.outputs.code != 'true' - run: echo "Only documentation (*.md) files changed — version bump check skipped." + run: echo "Only documentation files changed — version bump check skipped." From d9ba6b7ba5949bcc34708054368b769df7d3317a Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Sun, 26 Jul 2026 16:36:15 -0400 Subject: [PATCH 3/6] Nightly: the scheduled run re-dispatches onto dev, so dev's file is what executes Scheduled workflows always execute the DEFAULT branch's copy of the workflow file, while nightly artifacts deliberately build from dev's tree. That skew is exactly how the 2026-07-26 scheduled nightly failed (run 30194606068): main's stale copy still read Dashboard/Dashboard.csproj, a path #1612 moved to deprecated/ on dev, so 'Set nightly version' died on a file missing from the tree it had just checked out. Same trap as before (#1550/#1551) - syncing the file again would fix tonight and re-break at the next pre-release layout drift. Structural fix instead: on schedule the workflow does nothing but re-dispatch itself onto the dev ref (GITHUB_TOKEN can create workflow_dispatch runs; the Actions recursion guard exempts workflow_dispatch, and the dispatched run cannot loop back because it arrives as workflow_dispatch, not schedule). A workflow_dispatch run executes the DISPATCHED ref's copy of the file - dev's, current by definition - so after a one-time sync of this file to main, main's copy has exactly one tree-independent job that must keep working, and nightly logic changes take effect the night they merge to dev. Semantics preserved: manual dispatches (from_schedule defaults false) always build, exactly as before; the scheduled path keeps the 24h new-commit check via from_schedule=true; the artifact-publishing build job stays pinned to ref dev so a branch dispatch still cannot ship branch binaries. Co-Authored-By: Claude Fable 5 --- .github/workflows/nightly.yml | 47 ++++++++++++++++++++++++++++++++--- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 0be8bc0b..24f040b2 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -4,13 +4,49 @@ on: schedule: # 6:00 AM UTC (1:00 AM EST / 2:00 AM EDT) - cron: '0 6 * * *' - workflow_dispatch: # manual trigger + workflow_dispatch: # manual trigger — and the vehicle the scheduled re-dispatch below rides + inputs: + from_schedule: + description: 'Set true by the scheduled re-dispatch so the 24h new-commit check applies. Leave false for manual runs, which always build.' + type: boolean + required: false + default: false permissions: contents: write jobs: + # Scheduled workflows always execute the DEFAULT branch's copy of this file, while nightly + # artifacts deliberately build from dev's tree. That skew is how the 2026-07-26 nightly + # failed (run 30194606068): main's stale copy still read Dashboard/Dashboard.csproj, a path + # #1612 moved to deprecated/ on dev, so 'Set nightly version' died on a file missing from + # the tree it had just checked out — and the same trap bit before (#1550/#1551). The cure + # is structural, not another sync: on schedule this workflow does NOTHING but re-dispatch + # itself onto the dev REF, because a workflow_dispatch run executes the dispatched ref's + # copy of this file — dev's, current by definition. Once main carries this shape, its copy + # has exactly one job that must keep working, and that job references no tree paths at + # all; every future change to the real nightly logic lands on dev and takes effect the + # night it merges, no promotion to main needed. GITHUB_TOKEN can create workflow_dispatch + # runs (the Actions recursion guard exempts workflow_dispatch and repository_dispatch), + # and the dispatched run cannot loop back here because it arrives as workflow_dispatch, + # not schedule. Until main is synced once, the scheduled run still executes main's OLD + # copy and keeps failing nightly — the one-time sync is in the PR that introduced this. + redispatch: + if: github.event_name == 'schedule' + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + actions: write + steps: + - name: Re-dispatch this workflow onto the dev ref + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: gh workflow run nightly.yml --repo ${{ github.repository }} --ref dev -f from_schedule=true + + # Everything below runs only in a workflow_dispatch run — the scheduled re-dispatch or a + # manual one — which executes from the dispatched ref (dev for the scheduled path). check: + if: github.event_name == 'workflow_dispatch' runs-on: ubuntu-latest outputs: has_changes: ${{ steps.check.outputs.has_changes }} @@ -34,7 +70,10 @@ jobs: build: needs: check - if: needs.check.outputs.has_changes == 'true' || github.event_name == 'workflow_dispatch' + # Manual dispatches always build (from_schedule defaults false); the scheduled + # re-dispatch sets from_schedule=true and builds only when dev changed in the last + # 24h — the same policy the schedule applied when it ran these jobs directly. + if: needs.check.outputs.has_changes == 'true' || inputs.from_schedule != true runs-on: windows-latest steps: @@ -188,7 +227,9 @@ jobs: darling-pg: name: Darling PostgreSQL tests needs: check - if: needs.check.outputs.has_changes == 'true' || github.event_name == 'workflow_dispatch' + # Same gating as the build job: manual dispatches always run, the scheduled + # re-dispatch (from_schedule=true) runs only when dev changed in the last 24h. + if: needs.check.outputs.has_changes == 'true' || inputs.from_schedule != true runs-on: windows-latest steps: From b72de7822618f6a3b97a1c7161ea19dab0f87d0f Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Sun, 26 Jul 2026 16:36:57 -0400 Subject: [PATCH 4/6] CI: bound every non-release job with a timeout Every job that never carries the release/signing path gets a timeout-minutes ceiling sized at roughly 3x its worst observed or cold-cache path: build.yml darling-pg 30 (max observed 3m40s warm, cold pg-runtime fetch adds a ~340MB download), nightly check 10 / build 90 / darling-pg 60, sql-validation 30 per matrix leg, claude-review 30. The failure these bound is hung-not-slow - pg_ctl -w waiting forever on a cluster that will never come up, a SQL Server container that never reaches healthy - which otherwise holds a shared-pool runner for the 6h default while every other PR queues behind it. Deliberately NOT bounded: build.yml's build job. On release it waits on SignPath's manual approval gate, which can legitimately take hours; a timeout there could kill a release mid-signing. Co-Authored-By: Claude Fable 5 --- .github/workflows/build.yml | 7 +++++++ .github/workflows/claude-review.yml | 3 +++ .github/workflows/nightly.yml | 8 ++++++++ .github/workflows/sql-validation.yml | 4 ++++ 4 files changed, 22 insertions(+) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index e5259056..7b6a1fdb 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -504,6 +504,13 @@ jobs: darling-pg: name: Darling PostgreSQL tests runs-on: windows-latest + # Max observed on a warm cache is ~3m40s; a cold pg-runtime cache adds a ~340MB fetch. + # 30 minutes is 3x headroom over the cold path — past that, something is hung (pg_ctl -w + # waiting on a cluster that will never come up), and the default 6h timeout would hold a + # shared-pool Windows runner hostage for the duration. The build job above deliberately + # has NO timeout: on release it waits on SignPath's manual approval gate, which can + # legitimately take hours. + timeout-minutes: 30 permissions: contents: read diff --git a/.github/workflows/claude-review.yml b/.github/workflows/claude-review.yml index 5c3f2787..a096a7f9 100644 --- a/.github/workflows/claude-review.yml +++ b/.github/workflows/claude-review.yml @@ -17,6 +17,9 @@ concurrency: jobs: review: runs-on: ubuntu-latest + # Reviews are advisory and typically finish in minutes; a wedged run should release + # its runner rather than sit on the default 6h timeout. + timeout-minutes: 30 permissions: contents: read pull-requests: write diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 24f040b2..ea316b88 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -48,6 +48,7 @@ jobs: check: if: github.event_name == 'workflow_dispatch' runs-on: ubuntu-latest + timeout-minutes: 10 outputs: has_changes: ${{ steps.check.outputs.has_changes }} steps: @@ -75,6 +76,10 @@ jobs: # 24h — the same policy the schedule applied when it ran these jobs directly. if: needs.check.outputs.has_changes == 'true' || inputs.from_schedule != true runs-on: windows-latest + # Full pipeline (restore, tests, four publishes, cold pg-runtime fetch, vpk pack, + # release upload) is well under an hour; 90 minutes means hung-not-slow. Nightly ships + # unsigned, so unlike build.yml's release path there is no manual signing gate to wait on. + timeout-minutes: 90 steps: - uses: actions/checkout@v7 @@ -231,6 +236,9 @@ jobs: # re-dispatch (from_schedule=true) runs only when dev changed in the last 24h. if: needs.check.outputs.has_changes == 'true' || inputs.from_schedule != true runs-on: windows-latest + # Cold pg-runtime fetch + build + the full live-PG suite fits well inside an hour; a + # cluster that never comes up (pg_ctl -w) is the hang this bounds. + timeout-minutes: 60 steps: # Scheduled runs always test dev (schedules execute from the default branch, so ref_name diff --git a/.github/workflows/sql-validation.yml b/.github/workflows/sql-validation.yml index 3a1ad8f9..57fd7a06 100644 --- a/.github/workflows/sql-validation.yml +++ b/.github/workflows/sql-validation.yml @@ -18,6 +18,10 @@ concurrency: jobs: validate-sql: runs-on: ubuntu-latest + # Container pull + install scripts + seeded view execution runs in well under 15 + # minutes per version; a SQL Server container that never reaches healthy is the hang + # this bounds, and there are four of these jobs per run on the shared pool. + timeout-minutes: 30 strategy: fail-fast: false # SQL Server 2016 is the documented minimum but cannot appear here: SQL Server on Linux From 95097b22c7e7e515b8ee8be4014384795c375b95 Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Sun, 26 Jul 2026 16:48:26 -0400 Subject: [PATCH 5/6] CI: restore the change classification the dorny v4 bump silently broke The 2026-07-26 08:02 action-pin bump moved dorny/paths-filter v3 to v4. v4 evaluates every filter pattern as an INDEPENDENT predicate under the default predicate-quantifier 'some' (a filter is true when any changed file matches at least one rule), so a bare '!**/*.md' line stopped being a subtraction and became its own rule: 'any file that is not markdown'. Every area filter ending in that line went true for ANY non-markdown change anywhere in the repo, and the code: filter ('**' plus exclusions) could never be false again. Measured consequences, straight from run logs: - darling-pg ran the full TimescaleDB suite on every PR since the bump, including md-only ones. Run 30218459544 (PR #1709, two .md files): 'Filter darling = true, Matching files: CHANGELOG.md' - CHANGELOG.md matched '!Darling/**/*.md' by virtue of not being a Darling markdown. - A single root .gitignore edit built and tested every product: run 30219202642 (throwaway PR #1714) lit lite, dashboard, AND darling, each reporting 'Matching files: .gitignore' via its '!**/*.md' rule. - The #1712 docs fast path shipped unable to engage: every changed file matches '**', so code: was always true and setup/restore always ran. Its measured 1m43s md-only runs were real but came from the area filters (markdown matches no include), not from the fast path. The fix keeps v4 (v3 is on the deprecated-runtime track) and removes every bare negation instead: - Area filters state the markdown carve-out INSIDE each include as an extglob - 'Darling/**/!(*.md)' - where quantifier semantics cannot detach it. The positive-only docs allowlist needed no change. - The uninvertible code: filter becomes an all: counter. The classify step decides docs-only by all_count == docs_count, and additionally refuses to engage while any area filter is lit, because an area build step with restore skipped would run dotnet build --no-restore against nothing. Counts and areas derive from the same allowlist today; the veto exists for the day someone edits one and not the other. - check-version-bump.yml had the identical '**'-plus-exclusions shape, so its md-only skip had also silently stopped working. Same counter fix, same allowlist, and its gate now says why. Validation: probe commit on throwaway PR #1714 (this build.yml plus a Darling .txt probe and a Darling .md probe) makes the filter log a per-file truth table; results recorded in the CI-intelligence PR body. Co-Authored-By: Claude Fable 5 --- .github/workflows/build.yml | 116 +++++++++++++---------- .github/workflows/check-version-bump.yml | 41 ++++---- 2 files changed, 87 insertions(+), 70 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 7b6a1fdb..c6ad9dfa 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -57,37 +57,44 @@ jobs: # The shared PerformanceMonitor.* core libraries feed Lite, the Full Dashboard, AND # Darling (verified via ProjectReference), so a change here fans out to all three. # NOT the CLI Installer — it references only Installer.Core. + # + # Every area pattern says `dir/**/!(*.md)` — any non-markdown file under the + # area — instead of the old `dir/**` include plus a bare `!**/*.md` exclude. + # That is not style: dorny v4 evaluates each pattern as an INDEPENDENT + # predicate under the default predicate-quantifier 'some' (a filter is true + # when any changed file matches at least one rule), so a bare `!**/*.md` line + # is not a subtraction — it is its own rule meaning "any file that is not + # markdown", which silently made every area filter true for ANY non-markdown + # change anywhere in the repo. Measured proof: a single root .gitignore edit + # built and tested all four products and ran the full Darling PG suite + # (PR #1714, run 30219202642, filter log: "Filter darling = true, Matching + # files: .gitignore"). The extglob keeps the markdown carve-out INSIDE the + # include, where quantifier semantics cannot detach it. core: - - 'PerformanceMonitor.Alerting/**' - - 'PerformanceMonitor.Analysis/**' - - 'PerformanceMonitor.Collectors/**' - - 'PerformanceMonitor.Common/**' - - 'PerformanceMonitor.Notifications/**' - - 'PerformanceMonitor.PlanAnalysis/**' - - 'PerformanceMonitor.Ui/**' - - '!**/*.md' + - 'PerformanceMonitor.Alerting/**/!(*.md)' + - 'PerformanceMonitor.Analysis/**/!(*.md)' + - 'PerformanceMonitor.Collectors/**/!(*.md)' + - 'PerformanceMonitor.Common/**/!(*.md)' + - 'PerformanceMonitor.Notifications/**/!(*.md)' + - 'PerformanceMonitor.PlanAnalysis/**/!(*.md)' + - 'PerformanceMonitor.Ui/**/!(*.md)' # Installer.Core is shared by the CLI Installer AND the Full Dashboard's integrated # installer — a change rebuilds both, and nothing else. installer_core: - - 'deprecated/Installer.Core/**' - - '!**/*.md' + - 'deprecated/Installer.Core/**/!(*.md)' dashboard: - - 'deprecated/Dashboard/**' - - 'deprecated/Dashboard.Tests/**' - - '!**/*.md' + - 'deprecated/Dashboard/**/!(*.md)' + - 'deprecated/Dashboard.Tests/**/!(*.md)' lite: - - 'Lite/**' - - 'Lite.Tests/**' - - '!**/*.md' + - 'Lite/**/!(*.md)' + - 'Lite.Tests/**/!(*.md)' installer: - - 'deprecated/Installer/**' - - 'deprecated/Installer.Tests/**' - - 'install/**' - - 'upgrades/**' - - '!**/*.md' + - 'deprecated/Installer/**/!(*.md)' + - 'deprecated/Installer.Tests/**/!(*.md)' + - 'install/**/!(*.md)' + - 'upgrades/**/!(*.md)' darling: - - 'Darling/**' - - '!**/*.md' + - 'Darling/**/!(*.md)' # The DOCUMENTATION allowlist: files that cannot affect a build under any # job in this workflow. Deliberately an allowlist of non-executable content, # not a "everything that isn't code" subtraction — a new file type defaults @@ -111,42 +118,41 @@ jobs: - '.gitattributes' - 'docs/**/*.{md,svg,png,jpg,jpeg,gif}' - 'Screenshots/**/*.{md,svg,png,jpg,jpeg,gif}' - # True when any file OUTSIDE the documentation allowlist changed. A docs-only - # change skips .NET setup, restore and versioning below — there is nothing to - # compile — while the job itself still runs, so the required 'build' check - # reports a result rather than going missing and blocking the merge. - # - # The exclusions MIRROR the docs filter above; they are the same list stated - # negatively because paths-filter has no "not in that other filter" operator. - # Keep the two in step: a path added to docs: but not here still forces a - # restore, and a path excluded here but absent from docs: makes the fast-path - # log under-report what it classified. - code: + # Catch-all COUNTER, not a boolean gate: the classify step below decides + # "documentation-only" by comparing all_count to docs_count — they are equal + # exactly when every changed file sits on the docs allowlist. Stated as a + # count comparison because the previous shape ('**' plus '!' exclusions, + # a code: filter) could never be false under predicate-quantifier 'some' — + # every file matches '**', so the #1712 fast path shipped unable to engage + # (throwaway PR #1714: a .gitignore-only diff still paid setup + restore and, + # via the predicate bug above, a full build). + all: - '**' - - '!**/*.md' - - '!LICENSE' - - '!CITATION.cff' - - '!.gitignore' - - '!.gitattributes' - - '!docs/**/*.{md,svg,png,jpg,jpeg,gif}' - - '!Screenshots/**/*.{md,svg,png,jpg,jpeg,gif}' - - # Decides the docs fast path ONCE, in one place, and says so out loud. Two guards - # keep it off the paths where a skipped restore would be a real loss: + + # Decides the docs fast path ONCE, in one place, and says so out loud. Guards keep + # it off every path where a skipped restore would be a real loss: # release — the filter step does not even run there, and a release must always # compile and publish from a cold, fully restored tree. # push — dev/main pushes are the integration signal for what just merged, so # they restore unconditionally even for a docs-only commit. Cheap # insurance: this only forces the restore back on, it does not force # the per-product build/test steps, which stay path-gated as before. - # Everything else (pull_request) is eligible, and engages only when NOTHING outside - # the documentation allowlist changed. + # areas — belt and suspenders: even when the counts say docs-only, any lit + # area filter vetoes the fast path, because an area=true with restore + # skipped would run `dotnet build --no-restore` against nothing. The + # two classifications are built from the same allowlist so they cannot + # disagree today; this guard is for the day someone edits one and not + # the other. + # Everything else (pull_request) is eligible, and engages only when EVERY changed + # file is on the documentation allowlist (all_count == docs_count). - name: Classify change for the docs fast path id: fastpath shell: bash env: - CODE_CHANGED: ${{ steps.filter.outputs.code }} + ALL_COUNT: ${{ steps.filter.outputs.all_count }} + DOCS_COUNT: ${{ steps.filter.outputs.docs_count }} DOCS_FILES: ${{ steps.filter.outputs.docs_files }} + AREAS: 'root=${{ steps.filter.outputs.root }} core=${{ steps.filter.outputs.core }} installer_core=${{ steps.filter.outputs.installer_core }} dashboard=${{ steps.filter.outputs.dashboard }} lite=${{ steps.filter.outputs.lite }} installer=${{ steps.filter.outputs.installer }} darling=${{ steps.filter.outputs.darling }}' run: | set -euo pipefail @@ -162,14 +168,16 @@ jobs: exit 0 fi - if [ "${CODE_CHANGED}" = "false" ]; then + echo "Changed files: ${ALL_COUNT:-0} total, ${DOCS_COUNT:-0} on the documentation allowlist. Areas: ${AREAS}" + + if [ "${ALL_COUNT:-0}" -gt 0 ] && [ "${ALL_COUNT:-0}" -eq "${DOCS_COUNT:-0}" ] && [[ "${AREAS}" != *"=true"* ]]; then echo "engaged=true" >> "$GITHUB_OUTPUT" - echo "::notice title=DOCS FAST PATH ENGAGED::No file outside the documentation allowlist changed, so .NET setup, restore and versioning are skipped. This job still reports its result." + echo "::notice title=DOCS FAST PATH ENGAGED::All ${ALL_COUNT} changed files are on the documentation allowlist, so .NET setup, restore and versioning are skipped. This job still reports its result." echo "Documentation files classified in this change:" for f in ${DOCS_FILES}; do echo " - ${f}"; done else echo "engaged=false" >> "$GITHUB_OUTPUT" - echo "::notice title=Full build::At least one file outside the documentation allowlist changed." + echo "::notice title=Full build::At least one changed file is off the documentation allowlist (${DOCS_COUNT:-0} of ${ALL_COUNT:-0} classified as documentation)." fi - name: Setup .NET 10.0 @@ -530,10 +538,14 @@ jobs: # On push, compare against the previous commit on this branch (mirrors the build job); # on pull_request, an empty base makes the action diff against the PR base branch. base: ${{ github.event_name == 'push' && github.event.before || '' }} + # `Darling/**/!(*.md)` instead of a `Darling/**` include plus a `!Darling/**/*.md` + # exclude: dorny v4 treats each pattern as an independent predicate under the + # default quantifier, so the old bare negation was itself a match-all-non-Darling-md + # rule — this job ran the full TimescaleDB suite on every PR, including md-only + # ones (run 30218459544: "Filter darling = true, Matching files: CHANGELOG.md"). filters: | darling: - - 'Darling/**' - - '!Darling/**/*.md' + - 'Darling/**/!(*.md)' - '.github/workflows/build.yml' # This job's gate was already correct for documentation — a docs-only change leaves diff --git a/.github/workflows/check-version-bump.yml b/.github/workflows/check-version-bump.yml index d64e3971..1e158cda 100644 --- a/.github/workflows/check-version-bump.yml +++ b/.github/workflows/check-version-bump.yml @@ -15,28 +15,33 @@ jobs: # Documentation-only PRs (e.g. a CHANGELOG correction) do not bump the # version. The workflow still runs so the required check reports a result # — using paths-ignore would skip the run entirely and leave the check - # stuck pending. Instead, the version comparison below is gated on whether - # any non-documentation file changed. The exclusions MIRROR build.yml's - # documentation allowlist (its docs: filter) — keep the two in step, or a - # LICENSE-only correction will demand a version bump here while build.yml - # calls it documentation. + # stuck pending. Instead, the version comparison below is gated on a count + # comparison: all_count == docs_count means every changed file is on the + # documentation allowlist, which MIRRORS build.yml's docs: filter — keep + # the two in step, or a LICENSE-only correction will demand a version bump + # here while build.yml calls it documentation. A boolean '**'-plus- + # exclusions filter cannot express "any non-documentation file" under + # dorny v4's default predicate-quantifier ('some'): every changed file + # matches '**', so that filter shape is unconditionally true — which is + # why this gate had silently stopped skipping md-only PRs. - name: Detect non-documentation changes id: changes uses: dorny/paths-filter@v4 with: filters: | - code: + docs: + - '**/*.md' + - 'LICENSE' + - 'CITATION.cff' + - '.gitignore' + - '.gitattributes' + - 'docs/**/*.{md,svg,png,jpg,jpeg,gif}' + - 'Screenshots/**/*.{md,svg,png,jpg,jpeg,gif}' + all: - '**' - - '!**/*.md' - - '!LICENSE' - - '!CITATION.cff' - - '!.gitignore' - - '!.gitattributes' - - '!docs/**/*.{md,svg,png,jpg,jpeg,gif}' - - '!Screenshots/**/*.{md,svg,png,jpg,jpeg,gif}' - name: Get PR version - if: steps.changes.outputs.code == 'true' + if: steps.changes.outputs.all_count != steps.changes.outputs.docs_count id: pr shell: pwsh run: | @@ -45,14 +50,14 @@ jobs: Write-Host "PR version: $version" - name: Checkout main - if: steps.changes.outputs.code == 'true' + if: steps.changes.outputs.all_count != steps.changes.outputs.docs_count uses: actions/checkout@v7 with: ref: main path: main-branch - name: Get main version - if: steps.changes.outputs.code == 'true' + if: steps.changes.outputs.all_count != steps.changes.outputs.docs_count id: main shell: pwsh run: | @@ -61,7 +66,7 @@ jobs: Write-Host "Main version: $version" - name: Compare versions - if: steps.changes.outputs.code == 'true' + if: steps.changes.outputs.all_count != steps.changes.outputs.docs_count env: PR_VERSION: ${{ steps.pr.outputs.VERSION }} MAIN_VERSION: ${{ steps.main.outputs.VERSION }} @@ -75,5 +80,5 @@ jobs: echo "✅ Version bumped: $MAIN_VERSION → $PR_VERSION" - name: Skip notice - if: steps.changes.outputs.code != 'true' + if: steps.changes.outputs.all_count == steps.changes.outputs.docs_count run: echo "Only documentation files changed — version bump check skipped." From ae028750de8b7c46ebf33baed0da9c6536481c79 Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Sun, 26 Jul 2026 16:56:36 -0400 Subject: [PATCH 6/6] CHANGELOG: the CI-intelligence pass (#1715) Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6bb653d5..8cbcf962 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,6 +36,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **CI: the change classification the v4 pin bump silently broke is restored - and this time it is probe-validated** ([#1715]) - a timing baseline over the 30 most recent build.yml runs plus a throwaway probe PR (#1714) turned the planned build-time audit into a regression find. The 2026-07-26 08:02 pin bump moved dorny/paths-filter v3 -> v4, and v4 evaluates every filter pattern as an INDEPENDENT predicate under its default predicate-quantifier 'some' (a filter is true when any changed file matches at least one rule), so a bare `!**/*.md` exclusion stopped being a subtraction and became its own rule: "any file that is not markdown". Every area filter carrying that line went true for ANY non-markdown change anywhere in the repo - a single root .gitignore edit built and tested all four products (probe run 30219202642), darling-pg ran the full TimescaleDB suite on every PR since the bump including md-only ones (run 30218459544 matched CHANGELOG.md against the darling filter), and the [#1712] docs fast path shipped unable to engage, because every file matches '**' so its code: gate was never false - its measured md-only 1m43s runs were real, but they were the area filters at work (markdown matches no include), not the fast path. The regression was invisible by construction: the bump's own PR touched build.yml, so root=true forced a full build that looks identical to a correct run, and so does every over-built run after it. Fixed keeping v4 (v3 is on the deprecated-runtime track): area filters carry the markdown carve-out INSIDE each include as an extglob (`Darling/**/!(*.md)`) where quantifier semantics cannot detach it; the uninvertible code: filter is replaced by an `all:` counter with docs-only decided by all_count == docs_count; the classify step additionally refuses to engage while any area filter is lit, so the two classifications can never disagree into a `dotnet build --no-restore` with no restore behind it; and check-version-bump.yml, which had the identical '**'-plus-exclusions shape and an equally dead md-only skip, takes the same counter fix. Validated with a per-file truth table on the probe PR (run 30219765613: a Darling .txt probe, a Darling .md probe, .gitignore and the workflow file in one diff - each filter's matched-file list recorded in the PR body). **Also in this pass:** a PR re-push now cancels that PR's superseded in-flight build.yml/sql-validation.yml runs, while push and release runs keep unique per-run concurrency groups - never queued behind or cancelled by anything, every dev/main commit keeps its own check result; the [#1712] allowlist's flagged judgment calls are ratified (CITATION.cff, Screenshots/) but its directory-wide grants become extension-explicit (`docs/**/*.{md,svg,png,jpg,jpeg,gif}`) so a .sql dropped into docs/ tomorrow defaults to code; the scheduled nightly re-dispatches itself onto the dev ref instead of doing real work from main's copy of the workflow file - scheduled workflows execute the DEFAULT branch's copy against dev's checked-out tree, which is exactly how the 2026-07-26 06:00 nightly failed (run 30194606068: main's stale copy read Dashboard/Dashboard.csproj, moved to deprecated/ by #1612 - the #1550 trap again) - so after a ONE-TIME sync of nightly.yml to main (command in the PR body; the schedule stays red each morning until it happens) nightly logic changes take effect the night they merge to dev, with manual dispatches still always building and the artifact job still pinned to dev; and every job that never carries the release/signing path gets a timeout-minutes ceiling at ~3x its worst cold path (darling-pg 30, nightly build 90 / pg 60 / check 10, sql-validation 30 per leg, claude-review 30) so hung-not-slow failures stop holding a shared-pool runner for the 6h default - build.yml's build job stays unbounded on purpose, because the release path waits on SignPath's manual approval gate. **Measured and deliberately not done** (numbers in the PR body): per-area restore splitting (warm restore is 19-33s; four condition-mirrored restore steps buy seconds at the price of the drift risk [#1701] just retired) and cross-job test splitting (Run Lite tests ~2m25s dominates the full build, but a second Windows job costs ~2m45s of checkout/setup/restore/build before its first test - a net loss on a shared serialized pool). Merge queue remains a recommendation with exact settings in the PR body: it is a repo setting, and build.yml needs a merge_group trigger first or queued PRs stall on never-reporting required checks. - **CI: a documentation change stops paying for a .NET restore** ([#1712]) - non-executable changes now skip .NET setup, restore and versioning in the required `build` job, while the job still RUNS so the check reports and cannot block a merge. The gate is an explicit **allowlist** of non-executable content (`**/*.md`, `LICENSE`, `CITATION.cff`, `.gitignore`, `.gitattributes`, `docs/**`, `Screenshots/**`) rather than a subtraction, so an unfamiliar new file type defaults to being treated as code - the safe direction to be wrong in. `*.sql`, `*.yml`, `*.csproj`/`*.props`/`packages.lock.json` are deliberately excluded from the allowlist because some job compiles or consumes each of them, and `*.cs` is excluded however comment-only a change looks, since the compiler is what proves it still builds. **Two guards against under-building**: the fast path never engages on a `release` event or on a push to `dev`/`main` (those restore unconditionally - it only forces the restore back on, per-product build/test steps stay path-gated exactly as before), and both jobs now emit a `::notice::` naming WHY they took the path they took plus the file list classified as documentation, because a job that reports success having quietly run nothing is indistinguishable from one that tested everything. Measured against the real gap: markdown-only changes were ALREADY skipping every heavy step (PR #1707, pure `.md`, ran `build` in 1m43s), so what this actually fixes is non-markdown documentation - a `LICENSE`, `.gitignore` or screenshot edit previously matched the catch-all and paid a six-project locked-mode restore for nothing. The remaining ~1m45s floor is `actions/checkout` on a Windows runner, not work this gate can remove. - **CI: the Lite fast / analysis-heavy test split collapses back into one step** ([#1701]) - the split existed because the seven analysis classes rebuilt the full DuckDB schema inside every test and their subset alone cost ~9 CI minutes, so a narrower lite_analysis path gate let non-analysis Lite changes skip it. After [#1693]/[#1694]/[#1698] that subset runs in ~66s on the same runner, so the split stopped earning its second test-host spin-up - and its hand-maintained class-name filters were a standing drift risk (a renamed analysis class would silently fall out of the heavy filter into the fast bucket). One Run Lite tests step now runs the whole suite behind the lite path gate; the unconsumed lite_analysis filter block is gone. The lite gate is a strict superset of the old lite_analysis gate, so nothing that ran before is skipped now. - **Lite.Tests: the shared DuckDB fixture extends to 20 fast-bucket classes** ([#1698]) - rounds out [#1693]/[#1694]: a full-suite trx profile showed 400s+ of serial weight remaining in fast-bucket classes with the same per-test disease (a fresh DuckDB + full schema build inside every test; per-row single-connection seeding in several). All 20 eligible classes now take the class fixture with a data-only reset per test - including the four appender round-trip classes that previously built one database PER TABLE PER TEST - with the special cases handled rather than papered over: FindingStoreTests' v3->v4 analysis-schema migration test hand-creates the legacy table shape and therefore keeps its own private database file while its 15 siblings share; the two MCP classes keep a test-local temp dir for the ServerManager config directory; the server-time-helper collection trio keeps its collection attribute. Six classes are deliberately NOT converted because each needs a private database by design - ArchiveViewDedupTests (parquet + archive-view rebuilds), MuteRulesSurviveResetTests (ArchiveAllAndResetAsync deletes the database file), DuckDbSchemaTests (tests schema creation itself), and the three TestAlertDataHelper consumers (the helper writes parquet into the archive dir and rebuilds views) - that list is the output of reading every candidate class end to end, not a guess. Measured: total serial work across the suite 925s -> 760s (IndexObjectStats 60.6s -> 20.8s, SystemEventsReader 36.2s -> 13.8s, PerformanceCalendarData off the top-15); the local wall is parallelism-bound on a many-core box, but CI's 2-core runner is serial-bound and gets the elimination nearly 1:1. Full suite run twice consecutively: identical 1494/1494 results, zero warnings. @@ -1697,6 +1698,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [#1703]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1703 [#1708]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1708 [#1712]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1712 +[#1715]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1715 [#1710]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1710 [#1690]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1690 [#1693]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1693