ci: CPD without compile + changed-modules-only lint lane#1457
Draft
joaodinissf wants to merge 6 commits into
Draft
ci: CPD without compile + changed-modules-only lint lane#1457joaodinissf wants to merge 6 commits into
joaodinissf wants to merge 6 commits into
Conversation
joaodinissf
force-pushed
the
perf/lint-lane-step-5
branch
from
3ecbf45 to
e6a7274
Compare
joaodinissf
force-pushed
the
perf/lint-lane-step-5
branch
2 times, most recently
from
b78df4f to
c7f7b05
Compare
This was referenced Jul 11, 2026
joaodinissf
force-pushed
the
perf/lint-lane-step-5
branch
2 times, most recently
from
bbf2dde to
063479f
Compare
Replace the pmd/checkstyle jobs with a parallel shape that gives early,
inline feedback and stops re-running analysis inside the build:
- lint: compile + pmd:pmd + checkstyle:checkstyle (SARIF) + pmd:cpd-check,
-T 2C, --fail-never; gates by counting the merged SARIF (+ cpd.xml grep).
Fails in ~3-5 min on its own check, independent of the build.
- spotbugs: compile + spotbugs:spotbugs (SARIF), -Xmx4g, own parallel lane
(the slow analysis).
- maven-verify: build + tests only; the redundant checkstyle/pmd/spotbugs
goals are dropped (now owned by lint/spotbugs).
- line-endings: unchanged.
- both new lanes restore the master-produced Linux-maven-publish-* cache,
restore-only, mirroring snapshot.yml's path spec and key recipe exactly
(the path spec is hashed into the cache version, so the mirroring must
be literal).
All three emit SARIF 2.1.0, merged per tool and uploaded to Code Scanning
(security-events: write) for inline annotations on the PR diff + Security tab.
No custom Python annotator.
Count-gate rather than the *:check goals: the check goals @Execute-fork a
second analysis and cannot emit SARIF, and without the full compile classpath
they false-positive on type-resolving rules. Each report goal runs once
(full-reactor compile -> correct + SARIF) and the gate counts the result.
Rationale + tables in docs/ci-static-analysis-design.md; measurement protocol
in docs/ci-measurement-protocol.md.
CPD gating is wired but inert until dsldevkit#1339 lowers the token threshold.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SpotBugs' per-module analysis is the spotbugs job's long pole. A PR only needs its changed modules scanned, so a pre-step injects <spotbugs.skip>true> into every unchanged reactor module's pom — the plugin then skips the goal, and the per-module JVM fork, for them. The full-reactor compile is kept (a changed module keeps its complete aux-classpath); a build/config change falls back to a full scan. pull_request only — master/snapshot run a full scan. -Dspotbugs.onlyAnalyze was the cleaner-looking alternative but screens too late (after the per-module fork), ~17% vs ~88% measured; the script header documents the migration if an upstream SpotBugs early-exit ever lands. - .github/scripts/compute-spotbugs-skip.sh: diff -> changed modules -> inject skip into the unchanged ones (idempotent; build/config change -> full scan). - verify.yml spotbugs job: fetch-depth 0 + a scope step before compile; -Djgit.dirtyWorkingTree=ignore because the scope step dirties poms on purpose and this job releases nothing (releases/maven-verify keep =error); SARIF upload guarded so an empty scan set (no module scanned) doesn't fail the upload. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
spotbugs-maven-plugin forks a fresh JVM per module by default (maxHeap 2048), so a full scan pays 59 JVM startups. -Dspotbugs.fork=false runs the analysis inside the Maven JVM; the lane's MAVEN_OPTS -Xmx4g governs the shared heap (the plugin's maxHeap setting applies to forks only). Measured locally (full scan, clean tree, JDK 21): 75s vs 95s wall at -T 3C, and stable at CI thread width (-T 8, 4 GB heap) with identical findings (59 module SARIFs, same result count). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
joaodinissf
force-pushed
the
perf/lint-lane-step-5
branch
2 times, most recently
from
c3fcc98 to
0f1f731
Compare
compute-spotbugs-skip.sh now also exports SPOTBUGS_SCOPE_ARGS
("-pl ../ddk-target,<changed modules> -am") and the spotbugs lane
passes it to mvn, so a scoped run builds only the PR's changed modules
plus their upstream dependencies instead of the full 64-module
reactor. The -am-pulled unchanged dependencies keep the injected
spotbugs.skip: they compile (complete aux-classpath) but are not
analysed.
ddk-target is pinned into every scoped reactor: the target-definition
artifact is referenced by target-platform-configuration, not by any
bundle MANIFEST, so -am alone never pulls it — Tycho then falls back
to a local-repository copy of the .target, which fails on a cold cache
and can silently resolve a stale target definition on a warm one
(verified: a June install of the same sequenceNumber still pointed at
the 2026-03 release train while the tree's points at 2026-06).
The gate verifies that every scanned source-bearing module produced
its SARIF (and that a full scan produced any at all): --fail-never
swallows even target-resolution failures, so without a presence check
a dead build uploads nothing, counts zero violations, and passes
vacuously.
Fail-safes are unchanged: a build/config change means a full reactor
and full scan, and a PR touching no reactor module builds the full
reactor with every analysis skipped.
Measured locally (single-module change, clean tree, JDK 21): 34s wall
vs 75s for the full-reactor equivalent, resolving the current target
platform (jdt.core 3.46.0) with findings identical to the full scan.
Verified against a repository with no installed ddk-target: without
the pin the run reproduces the swallowed resolution failure with zero
SARIFs; with it the scoped run succeeds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CPD tokenizes sources under src/ and needs neither bytecode nor a resolved target platform, so the second lint invocation drops its compile goals. cpd.xml outputs are identical with and without the compile pass, verified at the current token threshold and at the PMD default of 100 (timestamp attributes aside). Measured locally (warm tree, JDK 21): 6.8s vs 44.3s at the current threshold; 4.5s vs 28.5s at threshold 100. In CI the invocation was 53s, ~40s of it redundant recompilation and JVM startup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compute-spotbugs-skip.sh becomes compute-analysis-skip.sh with a mode argument: `spotbugs` injects spotbugs.skip as before, `lint` injects pmd.skip, cpd.skip and checkstyle.skip, and each mode exports its -pl/-am reactor scope args. The lint lane gains the scope step and passes LINT_SCOPE_ARGS to both invocations; `compile` stays in the PMD/Checkstyle invocation because PMD's type-resolving rules need Tycho's aux-classpath (skip-injected -am dependencies compile but are not analysed). Changes under ddk-configuration (rulesets, filters) now also trigger the full-scan fail-safe in both lanes. Code Scanning note: repo-wide alert state reflects the default branch, which receives no lint/spotbugs analyses (verify runs on pull_request only), so a scoped upload that omits unchanged modules can only affect PR-context annotations — the same property the spotbugs category has had since the per-module skip landed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
joaodinissf
force-pushed
the
perf/lint-lane-step-5
branch
from
0f1f731 to
8161060
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Stacked on #1456. Earlier commits belong to the perf stack; review only the last two commits here.
Two reductions to the
lintlane (~3.5 min in CI):src/and needs neither bytecode nor the target platform. cpd.xml outputs verified identical with and without compile, at both the current token threshold and the PMD default of 100 (so the result holds after chore: enable CPD detection (pmd.cpd.min → 100), de-dup and baseline #1397). Locally 6.8s vs 44.3s; the CI step was 53s of which ~40s was redundant recompilation.compute-spotbugs-skip.shbecomescompute-analysis-skip.sh <base-sha> <spotbugs|lint>: the lint mode injectspmd.skip/cpd.skip/checkstyle.skipinto unchanged modules' poms and exportsLINT_SCOPE_ARGS(-pl ../ddk-target,<changed> -am), mirroring the spotbugs lane (including theddk-targetreactor pin and the report-presence gate from ci: in-process SpotBugs + changed-modules-only reactor in the spotbugs lane #1456; only source-bearing bundles are expected to report, so a source-less branding bundle can never false-fail the gate).compilestays in the PMD/Checkstyle invocation (PMD's type-resolving rules need Tycho's aux-classpath; the-am-pulled unchanged dependencies compile but are not analysed). When no reactor module changed at all (e.g. a docs-only PR) both lint invocations and the SARIF upload are skipped entirely (LINT_KEPT=0) — the upload guard mirrors the spotbugs lane's, since upload-sarif fails on an empty merge directory. Fail-safes unchanged (the full-scan path exportsKEPT=allso the workflow guard can never mistake it for the empty case), and changes underddk-configuration/(rulesets, filters) now trigger the full-scan path in both lanes.Measurements
Local (one changed module): PMD+Checkstyle invocation 11.7s vs 33.8s full, CPD 3.9s vs 20.1s, only the changed module reports, zero violations; CPD outputs identical at both token thresholds.
Code Scanning semantics: repo-wide alert state reflects the default branch, which receives no lint/spotbugs analyses (verify uploads on PR refs only), so a scoped upload that omits unchanged modules affects PR-context annotations only — the same property the spotbugs category has had since the per-module skip landed.
Merge chain:
#1399(merged) → #1396 → #1400 → #1456 → #1457 (this PR) → #1397🤖 Generated with Claude Code