Skip to content

ci: run RuboCop and YARD once instead of on every matrix job - #1672

Merged
jcouball merged 1 commit into
mainfrom
ci/split-lint-job-from-matrix
Aug 7, 2026
Merged

ci: run RuboCop and YARD once instead of on every matrix job#1672
jcouball merged 1 commit into
mainfrom
ci/split-lint-job-from-matrix

Conversation

@jcouball

@jcouball jcouball commented Aug 7, 2026

Copy link
Copy Markdown
Member

Motivation

Reduce the workload of PR CI runs. Profiling an actual run rather than guessing showed the cost was not in what the matrix tests, but in five jobs redundantly re-running the same lint.

From run 31204960715:

Job Total spec:unit spec:integration rubocop
3.2 ubuntu 71s 5.1s 13s 31s
3.4 ubuntu 82s 5.3s 13s 32s
4.0 ubuntu 84s 5.6s 13s 30s
truffleruby 347s 27.4s 75s 194s
jruby 403s 26.7s 80s 205s
3.2 windows 326s 4.4s 172s 72s

RuboCop was 51% of the JRuby job and 56% of the TruffleRuby job. It is pure duplicated work: RuboCop analyzes source text, and its verdict is fixed by TargetRubyVersion (pinned to 3.2), not by the host Ruby. The same is true of YARD.

Changes

  • New Lint and Docs job on ubuntu / Ruby 3.4 runs rake rubocop yard once. Matrix jobs run rake spec instead of rake default.
  • Concurrency groups on both PR workflows, so a new push cancels the in-progress run for the same PR instead of leaving a full matrix running against a commit nobody will merge. This matters most during rebases and review-feedback cycles.
  • Ruby 3.4 dropped from the spec matrix. It is bracketed by the 3.2 and 4.0 runs. It stays in CI as the lint host.
  • TargetRubyVersion comment corrected. It said 3.1 while the value was 3.2, and described guarding against main_branch_shared_rubocop_config being raised, when that config actually sets a lower value (3.1). The explicit pin is still required, because a value inherited via inherit_gem takes precedence over inference from the gemspec.

Why Ruby 3.4 hosts the lint job

yard-lint is only installed on Ruby 3.3+ (per the gemspec), and YARD cannot build on JRuby or TruffleRuby, so the host must be 3.3+. Choosing 3.4 over 4.0 keeps a merge gate insulated from churn in a brand-new major; Ruby 4.0 deprecations still surface through the matrix, which runs 4.0.

Side benefit: yard:lint now always runs. The old 3.2 job silently skipped it.

Measured result

Both figures below are warm-bundler-cache runs, so they are directly comparable: run 31204960715 had cache hits on all six jobs, and the numbers below come from attempt 2 of the run on this branch, which had cache hits too.

Before After
Total job-seconds 1,313s 731s (-44%)
Critical path 403s (JRuby) 238s (Windows) (-41%)

Per job, after: Lint and Docs 62s, 3.2 ubuntu 35s, 4.0 ubuntu 33s, truffleruby 163s, jruby 200s, 3.2 windows 238s.

The critical path is now Windows, bounded by its integration suite -- inherent process-spawn cost, not waste.

What was deliberately not done

  • Unit specs stay on every runtime. Dropping them from Windows/JRuby/TruffleRuby would save only 1.3%-7.9% of those jobs while giving up 5,816 examples of cross-engine coverage. That is the worst trade in the matrix: unit specs are where the parsers meet fixture output, which is exactly where the regexp and encoding behavior of MRI, JRuby, and TruffleRuby diverges.
  • No reduced PR matrix behind a merge queue. This change gets a larger win with no coverage loss and no ruleset changes. If faster PR feedback is still wanted after re-measuring, a merge queue is the right tool, but the release-please-- guards would need rewriting first, since merge_group events carry no github.event.pull_request.head.ref.

Verification

Locally: rake rubocop yard passes (RuboCop clean, plus yard:build, yard:lint, yard:example-test); rake spec passes with 5,816 unit and 569 integration examples at 100% line and branch coverage. Coverage of the old rake default is complete across the two job types, with build still running on every platform as a prerequisite of test:gem.

On this PR, confirm that the Lint and Docs log contains a rake yard:lint box and that no matrix job log contains a rake rubocop box.

Note on required checks

Unrelated to this diff, but surfaced while verifying it: the Release Branch ruleset requires only Verify Conventional Commits, so CI is not currently a merge gate and a PR with a red matrix is mergeable. That job's name is unchanged here, so nothing breaks. If CI should gate merges, the new names to add are Lint and Docs, Ruby 3.2 on ubuntu-latest, Ruby 4.0 on ubuntu-latest, Ruby truffleruby-24.2.1 on ubuntu-latest, Ruby jruby-10.0.0.1 on ubuntu-latest, and Ruby 3.2 on windows-latest.

Follow-up found while verifying (not addressed here)

This workflow triggers only on pull_request, so bundler-cache writes land under refs/pull/<n>/merge and are never shared between PRs. Only PRs whose base branch has a cache at default-branch scope get a hit -- currently just Ruby 4.0, populated incidentally by release.yml (ruby-version: ruby, which resolves to 4.0.6) during the v5.0.4 release.

The first run on this branch showed the cost: five of six jobs cold-installed gems, Setup Ruby took 293s of 925s total, and Windows alone spent 109s there versus 9s for the cache-hitting 4.0 job. Populating caches at default-branch scope looks worth roughly 240s of total time and 100s off the critical path -- comparable to this PR. Left for a separate change.

🤖 Generated with Claude Code

RuboCop and YARD analyze source text, so their results are identical on every
runtime: RuboCop's verdict is fixed by TargetRubyVersion, not the host Ruby.
Repeating them across all six matrix jobs was the largest cost in the build.

Measured from run 31204960715, RuboCop alone took 205s of the 403s JRuby job
and 194s of the 347s TruffleRuby job. Moving it and YARD into a single
ubuntu/Ruby 3.4 job cuts total matrix time from about 1313s to about 690s and
the critical path from 403s to about 245s, now bounded by the Windows
integration suite.

Also:

- Add concurrency groups so a new push cancels the in-progress run for the
  same PR, rather than leaving a full matrix running against a dead commit.
- Drop Ruby 3.4 from the spec matrix. It is bracketed by the 3.2 and 4.0 runs,
  and it remains in CI as the host of the new lint job.
- Host the lint job on Ruby 3.4, not 4.0. yard-lint needs 3.3 or later, and a
  merge gate is better insulated from churn in a brand-new major.
- Correct the TargetRubyVersion comment, which said 3.1 while the value was
  3.2 and described guarding against the shared config being raised when that
  config actually sets a lower value.

Unit specs stay on every runtime. They cost 4-27s per job, and they are where
the parsers get exercised against fixture output, which is exactly where the
regexp and encoding behavior of MRI, JRuby, and TruffleRuby diverges.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 7, 2026 20:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces duplicated CI work by moving linting and documentation checks out of the runtime matrix and into a single dedicated job, while also adding workflow-level concurrency cancellation to avoid running full matrices for outdated commits.

Changes:

  • Add a dedicated “Lint and Docs” job (Ubuntu, Ruby 3.4) to run rake rubocop yard once per CI run, and switch matrix jobs to run rake spec.
  • Add workflow-level concurrency to cancel in-progress runs when a PR receives new pushes.
  • Update .rubocop.yml documentation around TargetRubyVersion to correctly explain the explicit pin and its relationship to the gem’s Ruby floor.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
.rubocop.yml Clarifies why TargetRubyVersion is explicitly pinned to the gem’s minimum supported Ruby.
.github/workflows/enforce_conventional_commits.yml Adds concurrency cancellation so only the latest PR state is linted.
.github/workflows/continuous_integration.yml Introduces a single “Lint and Docs” job and trims the matrix to run specs only (dropping Ruby 3.4 from the spec matrix).

@jcouball
jcouball merged commit 0726964 into main Aug 7, 2026
14 checks passed
@jcouball
jcouball deleted the ci/split-lint-job-from-matrix branch August 7, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants