Skip to content

Retry GetBuildInfo to absorb Artifactory index propagation lag - #3682

Merged
agrasth merged 1 commit into
masterfrom
fix/pnpm-test-buildinfo-race
Sep 1, 2026
Merged

Retry GetBuildInfo to absorb Artifactory index propagation lag#3682
agrasth merged 1 commit into
masterfrom
fix/pnpm-test-buildinfo-race

Conversation

@agrasth

@agrasth agrasth commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • pm-version-monitor's pnpm compatibility test runs showed a sharp, date-correlated pass-rate collapse (100% → ~25%) on days with heavy concurrent CI load — spread evenly across every pnpm line including the officially-supported 10.x baseline, so it wasn't a pnpm-version issue at all.
  • Every failure shared the same signature: Error: Should be true / Messages: Build info was not found, immediately after a successful jfrog rt bp publish.
  • Root cause: tests.GetBuildInfo queried Artifactory right after the publish call returned success, with no retry. BuildInfoService.GetBuildInfo's own doc comment confirms a 404 surfaces as (found=false, err=nil) — exactly the shape of "published, but the search index hasn't caught up yet," which gets more likely the more concurrent CI load hits the same Artifactory instance.
  • Add a short bounded retry (4 attempts, 500ms exponential backoff, ~7.5s worst case) inside GetBuildInfo itself, so all ~20 call sites across pnpm_test.go, npm_test.go, maven_test.go, etc. benefit without touching each one individually. Only the found=false, err=nil case retries — any real error still returns immediately, unchanged from today's behavior.

Test plan

  • go build ./... — succeeds
  • go vet ./utils/tests/... — clean
  • Traced the exact failing runs from pm-version-monitor's pnpm dashboard (August 20 mass-retest window) back to this call site and confirmed the signature matches on every one
  • Confirmed via BuildInfoService.GetBuildInfo's doc comment in jfrog-client-go that found=false, err=nil is the documented 404 signal, so retrying only that case is safe and won't mask real failures

pm-version-monitor's pnpm compatibility runs showed a sharp,
date-correlated pass-rate collapse (100% -> ~25%) on days with heavy
concurrent CI load, spread across every pnpm line including the
officially-supported 10.x baseline -- not correlated with pnpm version
at all. Root cause traced to every failure sharing the same signature:

    Error: Should be true
    Messages: Build info was not found

tests.GetBuildInfo queried Artifactory immediately after
artifactoryCli.Exec("bp", ...) reported success, with no retry.
BuildInfoService.GetBuildInfo's own doc comment says a 404 surfaces as
(found=false, err=nil) -- exactly the shape of a build that was
published but whose search index entry hasn't propagated yet, which
happens more often under concurrent load hitting the same instance.

Add a short bounded retry (4 attempts, 500ms exponential backoff,
~7.5s worst case) inside GetBuildInfo itself so every one of its ~20
call sites across pnpm_test.go, npm_test.go, maven_test.go, etc.
benefits without individual changes. Only the found=false/err=nil
case retries; any real error still returns immediately, unchanged
from before.
@agrasth
agrasth force-pushed the fix/pnpm-test-buildinfo-race branch from 6e10efb to 3af0959 Compare September 1, 2026 09:13
@agrasth
agrasth deployed to build-gate September 1, 2026 09:13 — with GitHub Actions Active
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

@agrasth
agrasth merged commit a302742 into master Sep 1, 2026
128 checks passed
@agrasth
agrasth deleted the fix/pnpm-test-buildinfo-race branch September 1, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants