* perf(ci): shard smoke tests + cache CLI and fixtures [IDE-2006]
CP-1: Add `cache: true` to all actions/setup-go@v6 steps so the Go
module and build caches are preserved across CI runs.
CP-2: Add a 4-shard smoke-tests job that runs the 46 smoke tests in
parallel across separate ubuntu-latest runners. Remove SMOKE_TESTS from
the 3-OS integration-tests job so it runs integration tests only.
CP-3: Add resolveCliDir() — reads SNYK_LS_CLI_CACHE_DIR; when set the
CLI binary survives across runs via actions/cache; cleanup is a no-op.
CP-4: Add cloneRepoOnceCached() — reads SNYK_LS_FIXTURE_CACHE_DIR;
skips network clone on cache hit. Refactor cloneRepoOnce to share
cloneIntoBase logic.
CP-7: Pin sharedFakeLeaksCommit to f15521a so all three fixture repos
can be cached (was empty = HEAD, uncacheable).
* fix(ci): fix tilde expansion + add stale-cache eviction for smoke fixtures [IDE-2006]
Tilde paths in GitHub Actions env: blocks are literal strings — Go's
os.MkdirAll/os.Stat received "~/.cache/..." and created a literal "~"
directory, causing every smoke shard to fail with a cache miss.
Fix: add a "Set cache dir paths" step that writes the expanded $HOME
paths to GITHUB_ENV before the actions/cache and go test steps. The
cache action's path: field now references env.SNYK_LS_CLI_CACHE_DIR so
both the action and the Go code use the same absolute path.
Also add repoIsAtCommit() and stale-cache eviction to cloneRepoOnceCached:
when the cached dir's HEAD does not match the expected commit, the dir is
removed and a fresh clone is performed, preventing stale fixtures from
silently running against the wrong commit.
* perf(ci): shard smoke race tests across 4 parallel runners [IDE-2006]
Remove SMOKE_TESTS from the serial race-tests job (which was running all
46 smoke tests with -race on a single runner, same bottleneck as before
sharding). Smoke tests under the race detector now run as the new
smoke-race-tests job with the same 4-shard matrix used by smoke-tests,
reducing the wall time from ~22-25 min to the longest shard.
* perf(ci): run smoke shards on all 3 OS; upgrade actions/cache to v5 [IDE-2006]
Expand smoke-tests and smoke-race-tests to a 4-shard × 3-OS matrix
(12 jobs each). OS-specific steps handle Xvfb on Linux/macOS and skip
clipboard install on Windows; shell: bash ensures the -run pattern with
pipe characters works on all platforms.
Also bump actions/cache@v4 → v5 across both jobs to avoid the Node 20
deprecation warning.
* chore(ci): upgrade actions/cache to v5 in setup-tools composite action [IDE-2006]
* fix(ci): smoke-race-tests runs on ubuntu-latest only, not all OS [IDE-2006]
* fix(smoke): call cleanupCLI() explicitly before os.Exit — defer is skipped by os.Exit [IDE-2006]
* perf(ci): add catch-all 'other' shard (all OS) + pre-push coverage check [IDE-2006]
Add 'other' as a 5th shard in the smoke-tests matrix (5 shards × 3 OS =
15 jobs). The other shard uses source-based discovery rather than
`go test -list` (which would mix smoke and non-smoke tests):
application/server: greps *_smoke_test.go for function names, subtracts
named-shard patterns, and runs the remainder.
other packages: greps for files containing SmokeTestEnvVar or
testutil.SmokeTest, then runs those packages' full test suite —
TestMain and testutil.SmokeTest handle the smoke/non-smoke filtering
(infrastructure/oss Test_Scan, infrastructure/learn Test_GetLesson, etc.).
Remove verify-smoke-coverage CI job; replace with pre-push hook via
scripts/check-smoke-shard-coverage.sh + .pre-commit-config.yaml entry
that warns when a new smoke test is not in a named shard.
* fix(ci): replace paste -sd with tr+sed for macOS BSD compat [IDE-2006]
* fix(ci): add pact PATH in smoke 'other' shard for infrastructure/learn [IDE-2006]
* perf(ci): reduce smoke shard timeout from 30m to 15m [IDE-2006]
* perf(ci): self-declaring smoke shards via SmokeTestWithEngine shard param [IDE-2006]
Replace regex run-patterns + catch-all 'other' shard with a self-declaring
system: each smoke test passes its shard env var ("SMOKE_SHARD_N") to
SmokeTestWithEngine/SmokeTest. CI jobs set export SMOKE_SHARD_N=1; tests
for other shards skip themselves. Numbered shards [1,2,3,4] replace the
old named shards (workspace, code, precedence-org, infra).
- Add shardEnvVar param to testutil.SmokeTest and SmokeTestWithEngine;
shard check runs before engine init to avoid wasteful setup on wrong-shard runners
- Update 32 call sites across 11 test files with their shard assignment
- Shard 4 (infra) also covers infrastructure/learn, infrastructure/cli,
infrastructure/oss packages previously discovered dynamically by the 'other' shard
- Remove NAMED pattern string, UNCOVERED discovery, and SMOKE_PKGS grep from build.yaml
- smoke-race-tests job converted to same numeric-shard approach
- Remove check-smoke-shard-coverage.sh pre-push hook and script (shard
assignment now lives in the test, not in a separate coverage check)
- Fix Makefile: test-smoke, test-all, race-test, benchmark-real, instance-test
now set all required SMOKE_SHARD_N vars so local runs aren't silently skipped
* perf(ci): move Test_SmokeScanUnmanaged from shard 1 to shard 4 [IDE-2006]
Shard 1 was running ~2× longer than shard 4 (594s vs 289s on macOS).
Moving the unmanaged OSS scan test to shard 4 balances both shards to
~440s. Shard assignment based on actual CI timing from run 25801123088.
* perf(ci): run smoke shards against ./... for automatic package discovery [IDE-2006]
Hardcoded package lists require manual updates when new packages with
smoke tests are added. Running ./... lets tests self-select via the
SMOKE_SHARD_N env var — no YAML change needed for new packages.
Also removed the pact-path guard (now always set, harmless on other shards).
* fix(ci): guard analytics pact tests with IntegTestWithEngine [IDE-2006]
Switching smoke-race to ./... exposed infrastructure/analytics pact tests
in all smoke-race shards. Those tests use UnitTestWithEngine (no guard)
so they always run, call pact.Setup(true), and fail the pact-broker
version check in the smoke-race environment where INTEG_TESTS is not set.
Pact tests are integration tests by nature — they start a real mock server
and make HTTP calls. Changing to IntegTestWithEngine gives them the correct
INTEG_TESTS guard so they skip in smoke-race and only run in jobs that
explicitly set INTEG_TESTS=true (integration-tests, race-tests).
* fix(test): remove hardcoded SMOKE_SHARD_4 from OSS TestMain [IDE-2006]
TestMain must not know which shard its package belongs to — that
coupling means moving an OSS test to a different shard silently breaks
CI because TestMain exits early before any test can run.
Shard selection is the responsibility of individual tests via
SmokeTestWithEngine. TestMain only gates on SMOKE_TESTS.
Also align the CLI dir with the smoke_main_test.go caching pattern:
when SNYK_LS_CLI_CACHE_DIR is set the binary is reused across runs
rather than downloaded fresh in every shard.
* fix(ci): guard learn pact with IntegTest; increase smoke timeout to 25m [IDE-2006]
infrastructure/learn pact tests used UnitTest (no guard) so they ran in
all smoke-race shards, hitting the pact-broker version check and failing.
Same root cause as analytics — pact tests are integration tests and must
use IntegTest so they skip when INTEG_TESTS is not set.
Increase smoke-tests and smoke-race timeouts from 15m to 25m: running
./... in a monorepo adds ~500s of unit tests from other packages on top
of the smoke tests themselves, pushing past the 15m limit.
* chore: give each shard its own token
* chore: fix export
* chore: fix token assignment
* fix(ci): guard snyk_api pact test with IntegTest [IDE-2006]
Same root cause as analytics and learn: UnitTest has no guard so
pact.Setup(true) runs in smoke-race, hitting the pact tool version
check and failing. All three pact test files now use IntegTest.
* fix(ci): fix scanner race, rebalance smoke shards, fix hover test [IDE-2006]
scanner.go: SetScanInProgress moved before goroutine launch so
waitForDeltaScan never observes a stale "all done" snapshot in the
window between goroutine creation and first execution. This was the
root cause of the Test_SmokeInstanceTest panic (scan goroutine
processing results after t.TempDir() cleanup).
server_smoke_test.go:
- Test_SmokeScanUnmanaged moved back to SMOKE_SHARD_1. In SHARD_4 it
ran concurrently with infrastructure/oss CLI scans (via ./...) causing
resource contention that inflated its runtime from ~100s to 958s.
- Test_SmokeSnykCodeDelta_NoNewIssuesFound_JavaGoof moved from SHARD_2
to SHARD_3 to reduce Code API call density in SHARD_2 (429 rate limits).
server_test.go: Test_IntegrationHoverResults restricted to OSS-only
products (the hover assertion targets package.json, not Code/IaC) and
uses require.Eventually so the test stops immediately on timeout rather
than continuing into assertions with missing scan data.
* fix(server): add missing internal/product import in server_test.go [IDE-2006]
* ci: add all-checks summary job as single merge gate [IDE-2006]
Gates lint, integration-tests, smoke-tests, proxy-test, race-tests,
smoke-race-tests under one status check. Add "all-checks" to branch
protection required status checks; remove the old per-matrix entries.
* feat(ci): add parallel smoke test targets and fix scanner WaitGroup deadlock [IDE-2006]
Add test-smoke-serial (sequential shards) and test-smoke-parallel (4 concurrent
shell processes, output buffered to build/smoke-shard-N.log) Makefile targets.
Both save test-smoke and their own stage name to .tests-hash. Update
check-tests-run.sh to hint at make test-smoke-parallel when the stage is stale.
Fix deadlock window in scanner.go: SetScanInProgress now runs before
waitGroup.Add so a panic there leaves the WaitGroup untouched and Wait() safe.
* fix(ci): fix treeview race and SNYK_TOKEN_OSTEST for shard 4 [IDE-2006]
Test_SmokeTreeView was flaky when run with the race detector: OSS and
Code scans run concurrently, so a $/snyk.treeView notification from the
OSS scan could arrive before Code finished. The second subtest then called
getTreeView while app.js was not yet in the results. Fix: enable Code-only
in the test so there is exactly one scan; waitForScan guarantees it is
complete before the subtests run.
TestUnifiedTestApiSmokeTest requires the risk-score unified OSS API, which
needs a specific Snyk token (SNYK_TOKEN_OSTEST). When that env var is unset
GetEnvironmentToken falls back to SNYK_TOKEN, which in shard 4 is
SNYK_TOKEN_4 — a token that lacks the required feature flag, causing the
scan to fail with "Client request cannot be processed". Fix: export
SNYK_TOKEN_OSTEST=${{ secrets.SNYK_TOKEN }} in both smoke-tests and
smoke-race-tests steps so the test always uses the main token.
* chore: remove all checks job
* fix(scanner): fix referenceBranchScanWaitGroup deadlock and path resolution bugs [IDE-2006]
Move referenceBranchScanWaitGroup.Add(1) from the outer for-loop to
just before the inner reference-scan goroutine is launched. Previously,
if processResults panicked or executePreScanCommand returned an error
before the inner goroutine started, Done() was never called, permanently
deadlocking referenceBranchScanWaitGroup.Wait() in Scan() and causing
waitForScan to time out instead of detecting the panic.
Extract stat-based heuristics in getAbsTargetFilePath into a dedicated
resolveNonRelativePath helper (reduces cyclomatic complexity). Fix two
bugs in that logic:
- When all stat heuristics fail, fall back to the context file path
only when its basename matches displayTargetFile, preventing
mis-attribution in --all-projects scans where path is the workspace
root rather than the specific result file.
- In the relative-path branch, return joinedRelative when os.Stat
succeeds instead of falling through to return "".
Together these prevent empty GetAffectedFilePath() values from reaching
updateGlobalCacheAndSeverityCounts, which panicked via os.Exit(2) and
killed the test process in Test_SmokeInstanceTest (smoke-race shard 2).
Add two unit tests covering the new fallback behaviour.
* fix(make): install git hooks automatically via make tools [IDE-2006]
make tools previously installed only binary tooling, leaving the pre-push
hook (check-tests-run.sh) uninstalled. As a result, git push bypassed the
test-stage verification check, allowing pushes without tests being recorded
at the current HEAD.
Add a hook-installation step to the tools recipe: when pre-commit is
present, run pre-commit install and pre-commit install --hook-type pre-push
(both idempotent). When pre-commit is not found (e.g. minimal CI runners),
print an advisory and continue — make tools does not fail.
* chore: use a 5th service account for integration & proxy tests [IDE-2006]
---------
Co-authored-by: Nick Yasnohorodskyi <nikita.yasnohorodskyi@snyk.io>