Skip to content

Darling PostgreSQL tests skips shared-library changes its two sibling jobs catch, reports green, and blames documentation #3116

Description

@erikdarlingdata

Darling PostgreSQL tests reports success having run nothing whenever a change lands in a shared PerformanceMonitor.* library that Darling consumes. Its two sibling jobs in the same workflow file do not have this problem — they list those libraries explicitly, and only the job that runs the tests omits them.

Three jobs, three filters, one file

.github/workflows/build.yml on dev:

job does its filter include PerformanceMonitor.Collectors?
build (core filter, ~line 101) yesPerformanceMonitor.Collectors/**/!(*.md), alongside .Alerting, .Analysis, .Common, .Notifications, .PlanAnalysis
Darling Linux build (~line 857) yesPerformanceMonitor.Collectors/**, plus PerformanceMonitor.Common/** and PerformanceMonitor.Analysis/**
Darling PostgreSQL tests (~line 674) NODarling/**/!(*.md), build.yml, nightly.yml, and nothing else

The Darling Linux build job already carries a deliberate list of the shared libraries that reach Darling. The PostgreSQL test job carries none of them. So this is not a judgement call about how far the dependency graph extends — the answer is already written twice in this file, and the test job disagrees with both copies.

Behavioural proof from one run

PR #3114 changes exactly two files:

PerformanceMonitor.Collectors/PgIndexBloatCollector.cs
Lite.Tests/PgIndexBloatCollectorDefinitionTests.cs

Same commit, same workflow, opposite decisions:

check duration outcome
Darling Linux build 1 m 47 s ran — its filter includes PerformanceMonitor.Collectors
Darling PostgreSQL tests 17 s skipped — steps 5–14 all skipped, including Run Darling PG tests

For scale, the same PG job took 4 m 5 s on a sibling PR whose only change was a comment. The PR that changes a PostgreSQL collector ran no PostgreSQL tests; the PR that changes prose ran all of them.

And Darling genuinely consumes the changed code:

  • PerformanceMonitor.Collectors.csproj is referenced by four Darling projects — .Service, .Viewer, .Storage, .Analysis.
  • DarlingWorker.cs registers and runs the changed class: ["pg_index_bloat"] = (r, s, ct) => r.RunAsync(PgIndexBloatCollector.Instance, s, ct).
  • Darling/Darling.Tests/PgSchemaGeneratorTests.cs asserts on that same class — so there is Darling-side coverage of the change and the gate skipped the job that would have run it.

The gate keys on where the change LIVES rather than on what the tests COVER.

The skip notice asserts a false cause

::notice title=Darling PG tests skipped::No Darling code changed - documentation-only
Darling edits do not trigger the TimescaleDB suite.

For #3114 that is wrong on both clauses: Darling code changed (transitively, via a referenced project and a collector it runs), and the edit was C# source, not documentation.

The comment directly above that step explains why it exists:

"a job that reports success having quietly run nothing looks identical to one that tested everything. Costs one step; buys a log you can point at when asking 'did this actually get tested?'"

That is the right instinct, and this is the case where the log you point at gives the wrong answer confidently. The notice does not report the gate's decision — it reports one hypothesis about why the decision came out that way, with "documentation-only" hard-coded as the only explanation for darling == false. Note the Darling Linux build job's equivalent notice says "No Darling/service code changed" and makes no documentation claim; the wrong message is specific to this job.

Scope

Every shared library Darling consumes, not just .Collectors. That is the whole PerformanceMonitor.* core set the build job enumerates — including .Alerting, .Notifications and .Analysis, all shared seams where "Lite silently reads a permanently-empty value" is a known failure mode. A change to any of them currently ships with a green TimescaleDB check that ran no TimescaleDB.

This is not the dorny predicate-quantifier bug documented at length around these filters (a bare !**/*.md acting as an independent match-all rule). The patterns here are correct for what they say. The filter is simply missing entries its siblings have.

Suggested direction

Do not hand-copy the list a third time — three copies of the same dependency knowledge drifting apart is what produced this. Prefer sharing one filter decision across the jobs (the build job already computes a shared-core signal), or at minimum a single YAML anchor both Darling jobs consume.

And make the notice print the decision rather than a guessed cause — list-files: shell is already enabled on the build job's filter for exactly this purpose.

Acceptance criteria, both arms:

  • a PR touching only PerformanceMonitor.Collectors/** runs Run Darling PG tests to completion;
  • a PR touching only Darling/**/*.md still skips it.

A fix that makes the gate always-true removes the guard rather than correcting it, and the second arm is what catches that.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions