ci: add fast syntactic-only scalafix gate#4581
Open
andygrove wants to merge 1 commit into
Open
Conversation
The lint-java job only catches scalafix violations after a full compile (needed for the semantic rules) and runs as a 4-way Spark matrix, so a trivial syntactic issue such as a redundant string interpolator surfaces only after roughly 3.5 minutes of building. Add a standalone scalafix-syntactic job that runs the syntactic-only rule subset via the scalafix CLI with no compilation, giving feedback in seconds. Because syntactic rules are profile-independent, it scans every Scala source once, including the spark-4.1 and spark-4.2 trees that lint-java skips (those profiles cannot run -Psemanticdb yet), so it is the only gate covering them. Fix the one pre-existing RedundantSyntax violation this surfaces, a redundant s interpolator in CometDecimalArithmeticViewSuite, so the new job passes on a clean tree.
mbutrovich
approved these changes
Jun 3, 2026
Contributor
mbutrovich
left a comment
mbutrovich
left a comment
There was a problem hiding this comment.
Awesome! Thanks @andygrove!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #4545
Rationale for this change
.scalafix.confmixes two kinds of rules: semantic rules (ExplicitResultTypes,NoAutoTupling,RemoveUnused) that require a full compile, and syntactic rules (DisableSyntax,LeakingImplicitClassVal,NoValInForComprehension,ProcedureSyntax,RedundantSyntax) that only parse source.The
lint-javajob runs the whole set behind./mvnw package -Psemanticdbas a 4-way Spark matrix, so a trivial syntactic issue (for example the redundantsstring interpolator that broke #4579) only surfaces after roughly 3.5 minutes of building, repeated across four matrix legs.The syntactic rules need no compilation and are independent of the Spark/Scala profile, so they can be checked once, in seconds.
A second benefit:
lint-javadeliberately skips Spark 4.1 / 4.2 (semanticdb-scalac for those Scala patch versions is not published, so it cannot run-Psemanticdb). The new check scans every Scala source, so it is the only gate covering thespark-4.1/spark-4.2trees. It immediately surfaced one pre-existingRedundantSyntaxviolation inCometDecimalArithmeticViewSuite(aspark-4.1test), which is fixed here.What changes are included in this PR?
.scalafix-syntactic.confcontaining the syntactic-only rule subset..scalafix.confis unchanged and still enforces the full set inlint-java.scalafix-syntacticjob inpr_build_linux.ymlthat runs the scalafix CLI (--check --syntactic) via coursier with no compile, no Maven, and no Spark matrix. It runs as a peer of the existing fastlintjob and does not gate or reorder the semantic matrix.sinterpolator inCometDecimalArithmeticViewSuiteso the new job passes on a clean tree.How are these changes tested?
Validated locally with the scalafix CLI against the full
spark/source tree:scalafix --check --syntactic --config .scalafix-syntactic.conf --exclude '**/target/**' sparkexits 0.sinterpolator makes the same command fail (exit 32, one violation reported), confirming the gate catches the fix: codegen dispatcher returns NULL for invalid try_make_timestamp inputs (#4554) #4579 class of issue.