feat(cli): make the lockfile prune resilient to slow registries [RED-930] [ship]

CLAUDE.md

@@ -120,3 +120,4 @@ Source in `src/ai-context/`, built during `prepare`. Generates examples from fix
 - `CHECKLY_CACHE_DIR` — override the CLI's cache directory (embedded-package tarball downloads)
 - `CHECKLY_SKIP_NODE_VERSION_CHECK` — set to `1` to bypass the bin's hard Node version preflight (unsupported Node may then fail in unexpected ways); must be set in the shell environment — the preflight runs before `.env` is loaded
 - `CHECKLY_LOCKFILE_PRUNE` — set to `0` to disable pruning the bundled lockfile to the code bundle's contents; when a lockfile is bundled this also disables `bundle.packages.prune` (the manifest rewrite rolls back)
+- `CHECKLY_LOCKFILE_PRUNE_TIMEOUT` — lockfile prune time budget in seconds (default 30; `0` disables the timeout); invalid values are ignored

packages/cli/src/ai-context/references/configure-playwright-checks.md

@@ -16,8 +16,8 @@
 - For pnpm projects, the workspace-root pnpmfile (`.pnpmfile.cjs` / `.pnpmfile.mjs`) is bundled automatically so the remote install can reproduce the lockfile — but only when the lockfile records a pnpmfile checksum (run `pnpm install` with the pnpmfile in place to record one; without it the pnpmfile is silently not bundled). The pnpmfile must also be self-contained: it may only load side-effect-free Node.js builtins (such as `path`, `crypto`, `util`) via literal `require('...')`/`import` specifiers, and must not reference `process`, `__dirname`, `import.meta`, or dynamically computed module paths. A pnpmfile that is not self-contained is skipped with a warning, and the remote install may re-resolve dependencies instead of using the lockfile. The pnpmfile is uploaded, so avoid embedding secrets in it.
 - In a workspace (monorepo) whose code bundle covers only part of the workspace, the bundled lockfile is pruned automatically so the remote install only fetches what the bundle needs, instead of failing on dependencies of workspace members that were left out.
   - Supported for `pnpm-lock.yaml` (v6/9), `package-lock.json` (v2/3), the text `bun.lock`, and Yarn Berry `yarn.lock`. Yarn Classic v1 lockfiles are not supported; for bun's binary `bun.lockb`, regenerate a text lockfile with `bun install --save-text-lockfile`.
-  - The workspace's package manager binary must be available on the machine running the CLI.
-  - When pruning was needed but cannot run or cannot verify its result, the original lockfile ships unchanged and the CLI prints a note explaining why — follow it. Skips are silent when there is nothing to prune, or when pruning is disabled via `CHECKLY_LOCKFILE_PRUNE=0`; silent skip reasons and per-entry embed match details are visible under `DEBUG='checkly:cli:services:check-parser:*'` and `DEBUG='checkly:cli:services:embedded-packages'`.
+  - The workspace's package manager binary must be available on the machine running the CLI. For pnpm and npm, pruning reuses cached registry metadata and only contacts the registry for packages missing from the cache; yarn Berry and bun regenerate offline.
+  - When pruning was needed but cannot run or cannot verify its result, the original lockfile ships unchanged and the CLI prints a note explaining why — follow it (a timed-out prune's note names `CHECKLY_LOCKFILE_PRUNE_TIMEOUT=<seconds>`, which raises the default 30 s budget). Skips are silent when there is nothing to prune, or when pruning is disabled via `CHECKLY_LOCKFILE_PRUNE=0`; silent skip reasons, a timed-out prune's partial package manager output, and per-entry embed match details are visible under `DEBUG='checkly:cli:services:check-parser:*'` and `DEBUG='checkly:cli:services:embedded-packages'`.
   - For pnpm projects with `patchedDependencies`, patches belonging to unbundled workspace members are filtered out automatically. A note naming stale patch declarations means the lockfile is out of date with the config — refresh it with a regular install.
 - Checkly caches installed dependencies between runs, keyed off the workspace's dependency inputs (the lock file, every workspace member's `package.json` and `.npmrc` — whether or not the member is in the bundle) plus the bundle's own install inputs, so the key can change without a file edit, e.g. when a different set of workspace members lands in the bundle. To force a reinstall for deployed checks, set `runner.cache.install.version` (a string or a safe integer) at the top level of `checkly.config.ts` — not per check — and change its value whenever the cache should be invalidated; deployed, scheduled checks pick up the change on the next `checkly deploy`. If the config still declares the deprecated `caching.dependencyCache.version`, remove it — setting both fails config loading. Unset or empty values leave the cache key unchanged, so a dynamic value such as `version: process.env.DEPENDENCY_CACHE_VERSION` is safe when the variable is not always set. For a one-off reinstall during an ad-hoc run, use the `--refresh-cache` flag on `checkly test`, `checkly pw-test`, `checkly trigger`, or `checkly checks run` instead.
 - If dependencies come from a private registry that Checkly's infrastructure cannot reach (for example an intranet-only Nexus mirror), list them in `bundle.packages.embed` in `checkly.config.ts` — a top-level section: `bundle: { packages: { embed: ['@acme/private-utils', 'legacy-private-pkg@2.1.0'] } }`. The CLI resolves the entries against the workspace-root lockfile (Yarn Classic v1 lockfiles are not supported), downloads their tarballs, verifies each tarball's integrity (recorded in the lockfile, or fetched from registry metadata for Yarn Berry), and ships them inside the code bundle so the runner can install them without reaching the registry. Applies to Playwright Check Suites only, not Browser or Multistep Checks.

packages/cli/src/services/check-parser/__tests__/bundler.spec.ts

@@ -297,6 +297,7 @@ describe('Bundler.finalize() lockfile prune reporting', () => {
       return true
     })
     vi.stubEnv('CHECKLY_LOCKFILE_PRUNE', '')
+    vi.stubEnv('CHECKLY_LOCKFILE_PRUNE_TIMEOUT', '')
   })
 
   afterEach(async () => {
@@ -477,6 +478,7 @@ describe('Bundler.finalize() embedded package materialization', () => {
       return true
     })
     vi.stubEnv('CHECKLY_LOCKFILE_PRUNE', '')
+    vi.stubEnv('CHECKLY_LOCKFILE_PRUNE_TIMEOUT', '')
   })
 
   afterEach(async () => {
@@ -806,6 +808,7 @@ describe('Bundler.finalize() patch filtering', () => {
       return true
     })
     vi.stubEnv('CHECKLY_LOCKFILE_PRUNE', '')
+    vi.stubEnv('CHECKLY_LOCKFILE_PRUNE_TIMEOUT', '')
   })
 
   afterEach(async () => {
@@ -1189,6 +1192,7 @@ describe('Bundler.finalize() package pruning', () => {
       return true
     })
     vi.stubEnv('CHECKLY_LOCKFILE_PRUNE', '')
+    vi.stubEnv('CHECKLY_LOCKFILE_PRUNE_TIMEOUT', '')
   })
 
   afterEach(async () => {
@@ -1910,6 +1914,7 @@ describe('Bundler.finalize() patch filtering with real pnpm', () => {
       return true
     })
     vi.stubEnv('CHECKLY_LOCKFILE_PRUNE', '')
+    vi.stubEnv('CHECKLY_LOCKFILE_PRUNE_TIMEOUT', '')
   })
 
   afterEach(async () => {

packages/cli/src/services/check-parser/__tests__/lockfile-pruner.spec.ts

@@ -3,18 +3,23 @@ import fs from 'node:fs/promises'
 import os from 'node:os'
 import path from 'node:path'
 
+import Debug from 'debug'
 import { describe, it, expect, afterAll, vi } from 'vitest'
 
 import {
   pruneBundledLockfile,
   selectMaterializationEntries,
   shouldPruneLockfile,
+  redactDetail,
+  PruneBundledLockfileResult,
 } from '../lockfile-pruner.js'
 import { createFauxPackageFiles } from '../faux-package.js'
+import { UNPRINTABLE_URL } from '../../embedded-packages/diagnostics.js'
 import { BunDetector, NpmDetector, PackageManager, PNpmDetector, Runnable, YarnDetector } from '../package-files/package-manager.js'
 import { Package, Workspace } from '../package-files/workspace.js'
 import { Err, Ok } from '../package-files/result.js'
 import { File } from '../parser.js'
+import { captureStderr } from '../../../testing/capture-stderr.js'
 
 const PNPM_FIXTURE_ROOT = path.join(__dirname, 'lockfile-pruner-fixtures', 'pnpm-workspace')
 // Same shape as PNPM_FIXTURE_ROOT, plus two patched dependencies: `ms` is
@@ -83,9 +88,13 @@ const rewriteBunLockScript = (...replacements: Array<[string, string]>): string
 const rewriteYarnLockScript = (...replacements: Array<[string, string]>): string =>
   rewriteLockfileScript('yarn.lock', ...replacements)
 
-// Ambient values (particularly CHECKLY_LOCKFILE_PRUNE) must not leak into
-// test outcomes.
-const testEnv = (): NodeJS.ProcessEnv => ({ ...process.env, CHECKLY_LOCKFILE_PRUNE: undefined })
+// Ambient values (CHECKLY_LOCKFILE_PRUNE and its timeout override) must not
+// leak into test outcomes.
+const testEnv = (): NodeJS.ProcessEnv => ({
+  ...process.env,
+  CHECKLY_LOCKFILE_PRUNE: undefined,
+  CHECKLY_LOCKFILE_PRUNE_TIMEOUT: undefined,
+})
 
 // A PackageManager whose lockfile-only install command is replaced, for
 // exercising failure paths without a real package manager.
@@ -297,6 +306,20 @@ describe('lockfile-pruner', () => {
     })
   })
 
+  describe('redactDetail()', () => {
+    it('collapses every URL to scheme and host and drops bare userinfo, leaving plain text alone', () => {
+      expect(redactDetail('GET https://user:pw@registry.example/secret-token/pkg?token=abc failed'))
+        .toEqual('GET https://registry.example failed')
+      expect(redactDetail('see http://localhost:4873/p and https://[bad failed'))
+        .toEqual(`see http://localhost:4873 and ${UNPRINTABLE_URL} failed`)
+      expect(redactDetail('HTTPS://user:pw@registry.example/secret-token/pkg'))
+        .toEqual('https://registry.example')
+      expect(redactDetail('registry //alice:p@ss@registry.example/ rejected'))
+        .toEqual('registry //registry.example/ rejected')
+      expect(redactDetail('  did you mean this? yes  ')).toEqual('did you mean this? yes')
+    })
+  })
+
   describe('pruneBundledLockfile()', () => {
     it('skips notably for an unsupported package manager even when the lockfile is unreadable', async () => {
       // Pins the ordering invariant in pruneBundledLockfile: the capability
@@ -345,18 +368,100 @@ describe('lockfile-pruner', () => {
       })
     })
 
-    it('fails when the command times out', async () => {
+    it('fails when the command times out and debug-logs the partial output', async () => {
+      const { workspace, files } = makePnpmScenario()
+      // The timeout reason stays a one-liner; the child's partial output is
+      // only visible on the debug channel, so pin that it gets there.
+      const previouslyEnabled = Debug.disable()
+      Debug.enable('checkly:cli:services:check-parser:lockfile-pruner')
+      try {
+        let result: PruneBundledLockfileResult | undefined
+        const written = await captureStderr(async () => {
+          result = await pruneBundledLockfile({
+            workspace,
+            // The URL travels via the environment (passed through to the
+            // child) rather than the script text, which the pruner echoes
+            // in its own "Running ..." debug line.
+            packageManager: stubPackageManager(new Runnable('node', ['-e', `
+              process.stdout.write('x'.repeat(10_000) + '\\nOUTMARK ' + process.env.PRUNE_TEST_URL + '\\n')
+              process.stderr.write('ERRMARK\\n')
+              setInterval(() => {}, 1000)
+            `])),
+            files,
+            // Generous: the assertions need the child to have started and
+            // written before the kill, even on a loaded CI host.
+            timeoutMs: 2_000,
+            env: { ...testEnv(), PRUNE_TEST_URL: 'https://user:secret@registry.example/pkg?token=abc' },
+          })
+        })
+        expect(result).toMatchObject({ status: 'failed', reason: expect.stringContaining('timed out') })
+        // Assertions do not span the newline after the stream label: with
+        // DEBUG_COLORS on, debug re-prefixes every line.
+        const debugOutput = written.join('')
+        expect(debugOutput).toContain('partial stdout:')
+        expect(debugOutput).toContain('OUTMARK https://registry.example')
+        expect(debugOutput).toContain('partial stderr:')
+        expect(debugOutput).toContain('ERRMARK')
+        expect(debugOutput).not.toContain('secret')
+        expect(debugOutput).not.toContain('token=abc')
+        expect(debugOutput).not.toContain('/pkg')
+        // Only the tail survives the cap, marked as truncated.
+        expect(debugOutput).toContain('…')
+        expect(debugOutput).not.toContain('x'.repeat(10_000))
+      } finally {
+        Debug.enable(previouslyEnabled)
+      }
+    }, 30_000)
+
+    it('takes the time budget from CHECKLY_LOCKFILE_PRUNE_TIMEOUT in seconds', async () => {
+      const { workspace, files } = makePnpmScenario()
+      const result = await pruneBundledLockfile({
+        workspace,
+        packageManager: stubPackageManager(new Runnable('node', ['-e', 'setInterval(() => {}, 1000)'])),
+        files,
+        env: { ...testEnv(), CHECKLY_LOCKFILE_PRUNE_TIMEOUT: '1' },
+      })
+      expect(result).toMatchObject({
+        status: 'failed',
+        reason: 'node timed out after 1s; set CHECKLY_LOCKFILE_PRUNE_TIMEOUT=<seconds> to raise it',
+      })
+    }, 30_000)
+
+    it('lets an explicit timeout option outrank CHECKLY_LOCKFILE_PRUNE_TIMEOUT', async () => {
       const { workspace, files } = makePnpmScenario()
       const result = await pruneBundledLockfile({
         workspace,
         packageManager: stubPackageManager(new Runnable('node', ['-e', 'setInterval(() => {}, 1000)'])),
         files,
         timeoutMs: 500,
-        env: testEnv(),
+        env: { ...testEnv(), CHECKLY_LOCKFILE_PRUNE_TIMEOUT: '60' },
       })
-      expect(result).toMatchObject({ status: 'failed', reason: expect.stringContaining('timed out') })
+      expect(result).toMatchObject({ status: 'failed', reason: expect.stringContaining('timed out after 500ms') })
     }, 30_000)
 
+    it.each(['abc', '-5', '1.5', '2147484'])('ignores an unusable CHECKLY_LOCKFILE_PRUNE_TIMEOUT (%j)', async value => {
+      const { workspace, files } = makePnpmScenario()
+      // The stub exits at once, so the rejection is only observable on the
+      // debug channel; the outcome shows the parse did not fail the prune.
+      const previouslyEnabled = Debug.disable()
+      Debug.enable('checkly:cli:services:check-parser:lockfile-pruner')
+      try {
+        let result: PruneBundledLockfileResult | undefined
+        const written = await captureStderr(async () => {
+          result = await pruneBundledLockfile({
+            workspace,
+            packageManager: stubPackageManager(new Runnable('node', ['-e', ''])),
+            files,
+            env: { ...testEnv(), CHECKLY_LOCKFILE_PRUNE_TIMEOUT: value },
+          })
+        })
+        expect(result?.status).not.toEqual('failed')
+        expect(written.join('')).toContain(`Ignoring CHECKLY_LOCKFILE_PRUNE_TIMEOUT=${JSON.stringify(value)}`)
+      } finally {
+        Debug.enable(previouslyEnabled)
+      }
+    })
+
     it('skips when the regenerated lockfile is identical and nothing was backfilled', async () => {
       const { workspace, files } = makePnpmScenario()
       // With the absent member's real manifest in the bundle there is
@@ -618,7 +723,7 @@ describe('lockfile-pruner', () => {
       // child's output, not in the displayed command line.
       const scriptPath = path.join(await makeTempDir(), 'fail.cjs')
       await fs.writeFile(scriptPath, `
-        process.stdout.write('GET https://alice:sup3rsecret@registry.example.com/pkg failed ' + 'x'.repeat(600))
+        process.stdout.write('GET https://alice:sup3rsecret@registry.example.com/pkg?token=abc failed ' + 'x'.repeat(600))
         process.exit(1)
       `)
       const result = await pruneBundledLockfile({
@@ -632,7 +737,8 @@ describe('lockfile-pruner', () => {
         return
       }
       expect(result.reason).not.toContain('sup3rsecret')
-      expect(result.reason).toContain('registry.example.com')
+      expect(result.reason).not.toContain('token=abc')
+      expect(result.reason).toContain('https://registry.example.com failed')
       expect(result.reason).toContain('…')
     })