Skip to content

Share one path gate between the two Darling CI jobs and report its decision - #3122

Merged
erikdarlingdata merged 8 commits into
devfrom
L3116-pg-gate-shared-libs
Sep 7, 2026
Merged

Share one path gate between the two Darling CI jobs and report its decision#3122
erikdarlingdata merged 8 commits into
devfrom
L3116-pg-gate-shared-libs

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Darling PostgreSQL tests skipped every change to a shared PerformanceMonitor.* library Darling consumes, reported success in 17s having run no PostgreSQL at all, and printed a notice asserting the change was documentation. Its sibling Darling Linux build, in the same file and on the same commit, ran — because that job's filter listed shared libraries and the PostgreSQL job's listed none.

What changed

.github/darling-paths-filter.yml is new and holds the Darling path gate. Both Darling jobs read it through dorny/paths-filter's documented filters: <relative path> form, so the gate is one file rather than one list per job. It lives outside .github/workflows/ because anything inside that directory is parsed as a workflow.

The gate's darling filter is Darling/**/!(*.md), the seven shared libraries in extglob form, build.yml, nightly.yml, and the gate file itself. It also carries an all: ['**'] counter, which is what the notices report against — never a gate, since every changed file matches '**'.

Both gate-decision notices report the decision and the changed files it was made from: which files are in the gate, or how many changed and what they were. The PG job's skip notice hard-coded documentation-only as the sole explanation for a false gate; on #3114 (a PostgreSQL collector and its test) that was false in both clauses, and it read as authoritative because it was the only log there was.

Darling/Darling.Tests/DarlingPathFilterGateTests.cs is new: four facts holding the gate to the dependency graph rather than to a remembered list.

All three filter steps in the file move from list-files: shell to list-files: json, including the build job's docs fast path. The lists reach these steps through env: rather than a ${{ }} interpolation into script text — deliberately, since that interpolation is the pull_request script-injection footgun — and a runtime parameter expansion does not honour the escapes shell format emits, so word splitting turned one path containing a space into several entries naming no file. Eight tracked paths under Screenshots/ carry spaces and that directory is on the documentation allowlist, so the docs notice is the one that meets a spaced path first. A JSON array is self-delimiting, so each notice prints it verbatim. Nothing in the file consumes *_files other than these three notices, and *_count is emitted regardless of the format.

CommentFilterAdoptionTests' class summary described the PG job's filter as naming "only Darling/** and the two workflow files". Corrected; the claim it supports — that no entry added to the build job's filter can stand up the throwaway cluster — is unchanged and still true. RepoFileAdoptionTests gains one entry, because the new pin's anchors span line breaks so it takes the shared LF reader.

How a third copy of the library list was avoided

Sharing between the two Darling jobs, not adding to them. The gate file replaces Darling Linux build's inline list; it does not sit alongside it. So the repository holds the same number of copies of this dependency fact as before — the build job's core filter, which decides what gets compiled and published, and this one, which decides what reaches Darling — and the job that had none now reads the one the other job reads.

YAML anchors were not the mechanism: GitHub Actions' workflow parser rejects anchors and aliases, and the filter body is a nested YAML document inside a string input anyway, so an anchor could not span the two jobs. A build-job output consumed via needs: was the other candidate and was rejected: it would serialise the PostgreSQL job behind the whole Windows build, and a failed build would then skip the PG job entirely rather than run it — losing coverage exactly when a commit is in trouble. darling-tree-guards already pays that price deliberately, for a job whose gate is by design the complement of the build job's.

Requirement derived, not enumerated. TheSharedGate_CoversEverySharedLibraryDarlingCompiles walks the ProjectReference closure of Darling.Tests and requires a NAME/**/!(*.md) entry for every PerformanceMonitor.* project at the repository root. All seven are in that closure: .Viewer accounts for .Ui, .Analysis and .Notifications, .Service adds .Alerting and .PlanAnalysis, .Storage and Darling.Analysis add .Collectors, and .Common is reached by nearly all of them. So "how far does the dependency graph reach" is not a judgement any more — add a shared library to a Darling project and the gate entry becomes mandatory.

EachDarlingJob_ReadsTheSharedGate_AndCarriesNoListOfItsOwn asserts both jobs name the gate file and that neither contains filters: |. That second half is the one that matters: re-inlining a list is how a private copy comes back, and a private copy is what nothing compares.

Deleting the gate file is a hard error rather than a silent pass — dorny throws Configuration file not found, so both Darling jobs red.

Both acceptance arms, each on its own PR

Neither arm can be demonstrated on this PR. Its change touches build.yml, which every filter in that file names, so it fires all of them — that self-validation is why the entry exists, and it is also why this PR cannot show a skip. And its run arm fires on build.yml rather than on the .Collectors entry, which is the entry that was missing. So both arms were exercised on throwaway PRs branched off this one, where the diff against the base is the single file each arm is about and the base already carries this gate.

Arm 1 — a PerformanceMonitor.Collectors-only PR runs Run Darling PG tests to completion. #3125, one changed file, PerformanceMonitor.Collectors/PgIndexBloatCollector.cs — the same file #3114 changed, whose PG job reported success in 17s having skipped every step. Filter darling = true; the notice reads 1 of 1 changed files are in the Darling gate and names the file. Darling Linux build also ran the publish and the image build on the same commit, so the two jobs now agree.

Arm 2 — a Darling/**/*.md-only PR still skips it. #3124, one added markdown file under Darling/docs/. Darling PostgreSQL tests succeeded in 16s with steps 5–14 skipped, Run Darling PG tests among them; Filter darling = false, Matching files: none; the notice reads None of the 1 changed files are in the Darling gate (.github/darling-paths-filter.yml), so the TimescaleDB suite did not run on this commit and then names the file. Darling Linux build skipped its steps 5–14 likewise.

Both probe branches carry this PR's gate file byte-for-byte and its build.yml with exactly one line different — on.pull_request.branches widened to include the throwaway base, without which build.yml never dispatches for a pull request into a feature branch. That widening exists only on the throwaway branches. The first attempt at arm 2 registered only check-branches for precisely that reason and was closed unrun.

Ahead of CI, the pattern set was evaluated through dorny's own matcher — picomatch(pattern, {dot: true}) under the some quantifier, which is what dorny/paths-filter v4 does — over 23 file-set cases. Each of the seven libraries is true individually. The negative cases all stay false: a root .gitignore edit, CHANGELOG.md alone, the root README.md, Lite/ source and XAML, a deprecated product, the installer tree, and markdown inside a shared library. The first two are the #1714 / run-30218459544 regressions the extglob form exists to prevent, which is why no bare negation appears in the new file. The same harness ran the same cases against the gate as it stands on dev, as a control that the check discriminates rather than agreeing with whatever it is shown: there, arm 1 is false — the defect — and arm 2 is false, correctly.

Each of the four new facts was mutation-tested: dropping the .Collectors entry, rewriting one entry as a bare dir/** include, re-inlining a filters: | list in the PG job, restoring the documentation-only skip notice, reverting list-files to shell, and removing it altogether. Each mutation reds the fact it should and no other. TheCoverageCheck_ReportsAnInjectedGap carries two of those mutations in-suite, so the coverage check cannot pass by parsing nothing.

Deliberately left out

Lite/** is absent from the gate, though the build job's own darling filter carries it. That entry exists because Darling.Tests reads Lite source in its cross-app parity guards — and those guards run in the build job, whose "Run Darling tests" step fires on that filter, with darling-tree-guards as the backstop when it does not. What these two jobs add over that is a live TimescaleDB cluster and a linux-x64 publish, neither of which a Lite XAML edit can affect. A Lite/** entry here would buy a four-minute cluster and no signal.

The build job's core filter is untouched. It is a description of what to compile and publish, not of what reaches Darling, and three tests read pattern strings out of it by position.

Two costs are accepted rather than engineered around, both from the two jobs sharing one answer. Darling Linux build gains PerformanceMonitor.Ui, which it does not compile — it publishes the service, not the WPF viewer — so a UI-only change now costs it one publish and one image build. It also gains nightly.yml, which it has no tests to run against. Both err toward building. In exchange it gains PerformanceMonitor.Alerting and PerformanceMonitor.PlanAnalysis, which the service it publishes references directly and its own list had been missing.

Proposed CHANGELOG entry

Not applied here, to avoid conflicting in the shared [Unreleased] block:

  • The gated-live PostgreSQL job no longer reports green having tested nothing when a shared library changes ([Darling PostgreSQL tests skips shared-library changes its two sibling jobs catch, reports green, and blames documentation #3116]) - Darling PostgreSQL tests gated on Darling/** and the two workflow files, so a change to any PerformanceMonitor.* library Darling compiles against skipped every step including Run Darling PG tests and reported success in seconds - while Darling Linux build, on the same commit in the same file, ran, because its filter listed three of those libraries. Both jobs now read one gate at .github/darling-paths-filter.yml covering all seven libraries in Darling.Tests' ProjectReference closure, and DarlingPathFilterGateTests derives that requirement from the closure rather than restating it. The skip notice named "documentation-only" as the reason for every skip; it now reports what the gate matched, or how many changed files it classified and what they were. The three changed-file listings also move to list-files: json, which survives the env: indirection that shell format's escaping does not - eight tracked paths under Screenshots/ carry spaces, and each would have printed as several entries naming no file.

The Darling PostgreSQL job and the Darling Linux job now read
.github/darling-paths-filter.yml instead of each carrying its own list of
paths, and that gate covers every shared PerformanceMonitor.* library in
Darling.Tests' ProjectReference closure.

Both gate-decision notices report the decision and the changed files it was
made from rather than a hard-coded cause.
Comment thread .github/workflows/build.yml Outdated
@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

Reviewed this as a CI/CD infra change (no T-SQL, so the T-SQL style section of CONTRIBUTING.md doesn't apply here) — two GitHub Actions jobs, a new shared path-filter YAML, and a new derived-coverage test.

What I checked:

  • Gate coverage math: walked Darling.Tests.csproj's ProjectReference closure by hand across all five Darling projects and confirmed the union is exactly the 7 libraries the gate lists (Alerting, Analysis, Collectors, Common, Notifications, PlanAnalysis, Ui) — matches what DarlingPathFilterGateTests derives.
  • Confirmed darling-linux really was missing Alerting/PlanAnalysis before this PR (verified PerformanceMonitor.Darling.Service.csproj references both directly), so that fix is real, and that it doesn't compile Ui (only Viewer does, which that job doesn't publish) — so carrying Ui in the shared gate is a deliberate, correctly-documented over-build rather than an oversight.
  • nightly.yml's own darling-pg job gates on "did dev change in 24h", not on paths, so it correctly has no coupling to the new shared gate file — no parity gap there.
  • The env:-indirection used to build the notice text (DARLING_FILES: ${{ steps.filter.outputs.darling_files }} then referenced as $DARLING_FILES in the script) is the right call security-wise — it avoids the classic GitHub Actions script-injection footgun of interpolating ${{ }} directly into run: text on a pull_request-triggered workflow.
  • No Lite/Darling code parity issue — this only touches Darling's own CI gating, and the PR's own reasoning for leaving Lite/** out of the new gate checks out against the build job's existing lite/darling filters.
  • Left one inline comment on a minor, low-impact edge case in the new notice-printing loops (pre-existing pattern elsewhere in the file, just now used two more times).

Overall this is unusually well self-verified (mutation-tested assertions, both acceptance arms demonstrated). Nothing blocking.

RepoFileAdoptionTests requires RepoFile to be the only repo-file reader in
Darling.Tests, and requires the LF-reading pins to be exactly the ones it
declares. The new pin's anchors span line breaks, so it takes the LF reader and
is declared in that set.
list-files: shell escapes for substitution into script text; the lists reach
these steps through env:, and a runtime parameter expansion does not honour
those escapes, so word splitting turned a path containing a space into several
entries naming no file. Eight tracked paths under Screenshots/ carry spaces and
that directory is on the documentation allowlist. A JSON array is
self-delimiting, so each notice prints it verbatim.
@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

Reviewed this against CONTRIBUTING.md's conventions (no T-SQL here, so the style section doesn't apply) and the Lite/Darling parity requirement. Summary: no correctness, security, or parity issues found.

Independently verified the closure claim. I walked the ProjectReference graph out of the .csproj files myself rather than trusting the PR body's assertion: Darling.TestsDarling.Analysis, Darling.Service, Darling.Storage, Darling.Viewer, PerformanceMonitor.Common, and transitively out to Alerting, Analysis, Collectors, Notifications, PlanAnalysis, Ui. That's exactly the seven root PerformanceMonitor.* directories that exist, and exactly the seven entries in the new gate's darling filter — the closure computation in DarlingPathFilterGateTests.TheSharedGate_CoversEverySharedLibraryDarlingCompiles matches reality, not just its own derivation.

Self-validation is real, not just claimed. Since the gate file lists itself and build.yml as members of its own darling filter, an edit to the gate that left both jobs skipping would fire the gate anyway and run the new tests against itself — confirmed this by tracing the entries rather than taking the PR body's word for it.

Lite/Darling parity. No Lite files touched, and the deliberate absence of Lite/** from the new shared gate is justified by the build job's own darling filter already covering Lite-affecting parity guards (CrossAppGuardCiGateTests backstop) — this isn't parity drift, it's an intentional, explained scope boundary.

Shell notice loops (for f in ${DARLING_FILES}; do echo " - ${f}"; done, both jobs): unquoted expansion, but this matches dorny/paths-filter's own documented list-files: shell consumption pattern (env-var indirection + unquoted for-loop), so not a defect.

One thing worth double-checking outside this review's reach: the PR body's "Arm 2" (a Darling/**/*.md-only PR still skips the PG job) was validated on a separate throwaway branch rather than in this diff — worth confirming that result is still attached/linked somewhere before merge, since it's the negative-case half of the fix.

No blocking findings.

@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

Review summary

Went through the full diff (the new .github/darling-paths-filter.yml, the build.yml job changes, and the new/updated Darling.Tests files). This is CI infrastructure only — no T-SQL, so the T-SQL style conventions don't apply here.

Correctness

  • Traced DarlingPathFilterGateTests.SharedLibraryClosure() by hand against the actual ProjectReference graphs (Darling.TestsDarling.Analysis/Service/Storage/Viewer → the seven PerformanceMonitor.* libs). The union is exactly the seven entries in the new gate (Alerting, Analysis, Collectors, Common, Notifications, PlanAnalysis, Ui) — the derived-requirement test is sound today, not just plausible.
  • The darling-pg and darling-linux job blocks are symmetric: same list-files: json, same shared filters: .github/darling-paths-filter.yml, same env:-indirection for DARLING/DARLING_COUNT/DARLING_FILES/ALL_COUNT/ALL_FILES, same release/skip/run branching. Job IDs and step names referenced by JobBlock/StepBlock in the new test (darling-pg, darling-linux, Report the Darling PG gate decision, Report the Linux gate decision) match the workflow exactly.
  • ReadRepoFileLf normalizes CRLF→LF before the new test's multi-line regexes and literal-string mutations run against it, so the LF-based anchors in DarlingPathFilterGateTests are correct even though the checkout itself is CRLF (.gitattributes). The new file was correctly added to RepoFileAdoptionTests.s_lfReaders.
  • The switch from list-files: shell to list-files: json for the docs fast-path step is safe — the old for f in ${DOCS_FILES} loop was removed in favor of printing the JSON array directly, so there's no leftover shell-splitting logic assuming the old format.
  • Security posture is actually improved here: file lists now reach the notice steps via env: rather than ${{ }} interpolated into script text, closing the classic pull_request-triggered script-injection footgun for attacker-controlled filenames (and sidestepping shell-format's broken backslash-escaping for spaced paths under Screenshots/**).

Lite/Darling parity

  • No parity drift. This only touches the Darling-specific CI gate; Lite/** is deliberately excluded from the new shared gate, and the reasoning (Lite-triggered Darling suite runs already happen through the build job's own darling filter + darling-tree-guards backstop) is documented inline and matches how that filter still reads today.

No correctness, security, or parity issues found. Nice self-validating design — the mutation-style tests (TheCoverageCheck_ReportsAnInjectedGap, the bare-dir/** mutation) actually exercise the failure modes the PR description calls out.

@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

Reviewed the diff (CI workflow + Darling.Tests only — no T-SQL, no Lite changes, so the T-SQL style rules and Lite/Darling app-parity concerns don't apply here; this is CI-gate parity between two jobs in the same workflow file, and that's what the PR fixes).

What I checked:

  • Gate correctness: walked the actual ProjectReference closures of Darling.Tests.Viewer/.Service/.Storage/.Analysis and confirmed all 7 PerformanceMonitor.* shared libraries the new darling filter lists (Alerting, Analysis, Collectors, Common, Notifications, PlanAnalysis, Ui) are in fact referenced, matching DarlingPathFilterGateTests.TheSharedGate_CoversEverySharedLibraryDarlingCompiles's claim.
  • list-files: json migration: confirmed DOCS_FILES/DARLING_FILES/ALL_FILES are all threaded through env: (never ${{ }} interpolated into script text), so the script-injection footgun is actually closed, not just described as closed.
  • Control flow: the if release … exit 0 / if DARLING … else … split in both gate-decision steps is equivalent to the old if/elif/else, and set -euo pipefail doesn't cause early-exit surprises anywhere in the new branches.
  • Regex/text-matching helpers in DarlingPathFilterGateTests (JobBlock, StepBlock, FilterPatterns) against the real build.yml: indentation assumptions (2-space job keys, 6-space step keys) match the actual file, and ReadRepoFileLf is correctly required given the file is CRLF-checked-out (hence the RepoFileAdoptionTests.s_lfReaders addition).
  • Cross-guard interference: Lite.Tests/CrossAppGuardCiGateTests.FilterPatterns and Darling.Tests/ViewerSidebarDotRendersTheCardStatusTests.TheGuard_RunsOnEveryTreeItScans both read the build job's own inline darling/core filter block (unindented differently, untouched by this PR) — they don't collide with or get fooled by the new filters: <path> line added to the two Darling jobs.
  • nightly.yml: its darling-pg job runs unconditionally (no path filter at all), so it needed no corresponding change — consistent with the PR not touching it.

No correctness bugs, no parity drift, and no security concerns found — if anything this closes a latent injection vector. Nice work on the mutation-tested facts and the two throwaway-PR arms (#3125/#3124) demonstrating both directions of the fix.

@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

Reviewed the diff (.github/darling-paths-filter.yml, build.yml, and the three Darling.Tests files). No T-SQL in this PR, so the style-guide section doesn't apply, and there's no Lite/Darling parity concern in the usual collector sense — this only touches CI gating for the two Darling-only jobs (darling-pg, darling-linux); nightly.yml's darling-pg job runs unconditionally with no path filter, so it's unaffected, and the build job's own separate darling/core filters (used for the in-process "Run Darling tests" step and Lite/Darling parity guards) are untouched, as claimed.

Things I specifically checked and found correct:

  • Both jobs now point at the same filters: .github/darling-paths-filter.yml, and the env:-indirection for DARLING/DARLING_COUNT/DARLING_FILES/ALL_COUNT/ALL_FILES in both "Report the ... gate decision" steps is wired consistently and avoids interpolating attacker-controlled filenames into the script text (the classic pull_request script-injection footgun) — good call flagging that in the comments.
  • The skip-branch messages use ALL_COUNT/ALL_FILES rather than DARLING_COUNT/DARLING_FILES, which is correct: when darling=false, none of the changed files matched, so "none of ALL_COUNT" is the accurate statement.
  • list-files: json switch is applied consistently everywhere DOCS_FILES/DARLING_FILES/ALL_FILES are consumed later — no leftover shell-format consumer that would now choke on JSON.
  • DarlingPathFilterGateTests.SharedLibraryClosure() correctly excludes the Darling.* projects from the "shared library" set via the parent-directory check (their folders sit under Darling/, not repo root), matching the doc comment's claim.
  • The new test file is correctly added to RepoFileAdoptionTests's s_lfReaders pin (it uses ReadRepoFileLf, and its JobBlock/StepBlock helpers do rely on matching literal \n-based anchors that only work post-normalization).
  • Gate file itself is straightforward valid YAML; the dir/**/!(*.md) form (vs. a bare !**/*.md exclude) is applied consistently, matching the documented dorny v4 predicate-quantifier gotcha.

No correctness, security, or parity issues found. This is unusually well self-validated (four new/adjusted xunit facts pin the gate's shape, with an explicit mutation test), so I don't have anything to flag.

@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

Reviewed this as a CI/tooling-only change (no T-SQL, no Lite/Darling app code touched), so the T-SQL style/parity checklist mostly doesn't apply — checked instead for correctness of the new shared gate, the two rewritten Report the ... gate decision steps, and the new DarlingPathFilterGateTests.

Verified independently (not just taking the PR body's word for it):

  • Manually walked the ProjectReference closure from Darling.Tests.csproj.Darling.Analysis/.Darling.Service/.Darling.Storage/.Darling.Viewer → their PerformanceMonitor.* references. The union is exactly {Common, Ui, PlanAnalysis, Alerting, Analysis, Notifications, Collectors} — matches all seven entries in .github/darling-paths-filter.yml's darling filter. TheSharedGate_CoversEverySharedLibraryDarlingCompiles's derivation is accurate today.
  • Confirmed no other step in build.yml consumes root_files/core_files/lite_files/etc. from the first filter step, so switching that step's list-files from shell to json globally (not just for docs_files) doesn't silently break another consumer.
  • Confirmed nightly.yml has no paths-filter step of its own (it always runs unconditionally), so there's no third copy of this list left out of sync there.
  • The env:-indirection pattern used for DARLING_FILES/ALL_FILES/DOCS_FILES is the correct fix for the stated risk: a value substituted into a bash variable and then expanded with "$VAR" is never re-parsed for command substitution or word-splitting, whereas the old ${{ }}-into-script-text form (were it ever used here) would be. Good call using env: rather than direct interpolation given filenames are attacker-influenceable on a PR.
  • Traced JobBlock/StepBlock/FilterPatterns in DarlingPathFilterGateTests.cs against the actual build.yml layout (2-space job indent, 6-space step indent, column-zero filter keys in the gate YAML) — the string/regex boundaries line up with the real file, so these aren't testing a fixture that's drifted from reality.

Not verified: couldn't run dotnet restore/dotnet test in this sandbox (no network egress to NuGet), so the new tests were checked by static reasoning rather than execution. Worth confirming CI is green on this PR (particularly DarlingPathFilterGateTests and RepoFileAdoptionTests) before merge.

No correctness, security, or Lite/Darling parity issues found. The Lite/** omission from the new gate and the PerformanceMonitor.Ui/nightly.yml cost on the Linux job are both called out and justified in the PR body, and I agree with the tradeoffs as described.

@erikdarlingdata
erikdarlingdata merged commit 641c4db into dev Sep 7, 2026
8 checks passed
@erikdarlingdata
erikdarlingdata deleted the L3116-pg-gate-shared-libs branch September 7, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant