Skip to content

ci(rust): skip unnecessary setup steps - #3749

Merged
hubcio merged 3 commits into
apache:masterfrom
Standing-Man:ci-opt
Aug 3, 2026
Merged

ci(rust): skip unnecessary setup steps#3749
hubcio merged 3 commits into
apache:masterfrom
Standing-Man:ci-opt

Conversation

@Standing-Man

@Standing-Man Standing-Man commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Rationale

Rust CI was doing redundant work in two areas:

  1. Lightweight checks such as formatting, dependency sorting, and
    cargo machete installed nextest and system dependencies that they
    did not use.

This increased both CI setup time and total compute usage.

What changed?

  • Added configurable Rust setup options for installing nextest and
    platform-specific system dependencies.
  • Skipped unnecessary setup for formatting, sorting, and machete jobs.
  • Explicitly enabled nextest only for jobs that require it.

Results

The setup optimization was measured over three runs:

Measurement Baseline median Optimized median Improvement
Rust setup step 44 s 13 s 31 s / 70.5%
Benchmark job 52 s 20 s 32 s / 61.5%

A pre-merge comparison showed:

Job Baseline Optimized Improvement
fmt 47 s 22 s 25 s / 53%
sort 42 s 19 s 23 s / 55%
machete 79 s 61 s 18 s / 23%
Combined 168 s 102 s 66 s / 39%

The benchmark link.

Local Execution

  • Passed / not passed
    Passed
  • Pre-commit hooks ran / not ran
    Ran

AI Usage

@github-actions

Copy link
Copy Markdown

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

  • /ready - back to S-waiting-on-review after addressing feedback
  • /author - flip to S-waiting-on-author while you finish changes
  • /request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Jul 25, 2026
@codecov

codecov Bot commented Jul 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.80%. Comparing base (ddead57) to head (321150f).

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3749      +/-   ##
============================================
- Coverage     75.86%   75.80%   -0.07%     
  Complexity      969      969              
============================================
  Files          1323     1323              
  Lines        160111   160111              
  Branches     133501   133576      +75     
============================================
- Hits         121476   121376     -100     
- Misses        34993    35005      +12     
- Partials       3642     3730      +88     
Components Coverage Δ
Rust Core 75.75% <ø> (-0.03%) ⬇️
Java SDK 62.71% <ø> (ø)
C# SDK 71.16% <ø> (-1.11%) ⬇️
Python SDK 93.10% <ø> (ø)
PHP SDK 84.52% <ø> (ø)
Node SDK 96.30% <ø> (+0.08%) ⬆️
Go SDK 43.08% <ø> (ø)
see 51 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Standing-Man
Standing-Man force-pushed the ci-opt branch 2 times, most recently from b089984 to 4526ce8 Compare July 25, 2026 03:26
@Standing-Man
Standing-Man marked this pull request as draft July 25, 2026 03:32
@github-actions github-actions Bot removed the S-waiting-on-review PR is waiting on a reviewer label Jul 25, 2026
@slbotbm

slbotbm commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

The cpp bdd failure looks like a transient network failure

@Standing-Man
Standing-Man marked this pull request as ready for review July 28, 2026 04:46
@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Jul 28, 2026
@hubcio

hubcio commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

very nice! i'll review this today

@hubcio hubcio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the direction here is good, and the PR closes a real pre-existing hole worth naming: previously test-1 and test-2 each ran their own git fetch origin master and cargo rail plan, so two racing fetches could resolve different origin/master, produce different -E filters, and let a test fall out of both partitions. one build shipping a single nextest-filter.txt makes the partitions complementary by construction.

one blocker, on the artifact name - see the comment at line 323 of the composite action. everything else is should-fix or polish.

one finding lands outside the diff so it has no line to attach to: Install cargo-nextest in setup-rust-with-cache still carries continue-on-error: true, but this PR deletes the cargo test fallback that made a missing nextest survivable. a curl flake now shows a green setup step and dies several steps later at cargo nextest archive with "no such command: nextest". the flag has outlived its reason - worth dropping while you're in there.

Comment thread .github/actions/rust/pre-merge/action.yml Outdated
Comment thread .github/actions/rust/pre-merge/action.yml
Comment thread .github/workflows/_test.yml Outdated
Comment thread .github/workflows/_test.yml Outdated
Comment thread .github/actions/rust/pre-merge/action.yml Outdated
Comment thread .github/actions/utils/setup-rust-with-cache/action.yml
Comment thread .github/actions/rust/pre-merge/action.yml
Comment thread .github/actions/rust/pre-merge/action.yml Outdated
Comment thread .github/actions/rust/pre-merge/action.yml Outdated
Comment thread scripts/ci/sync-python-interpreter-version.sh
@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jul 29, 2026
@Standing-Man

Copy link
Copy Markdown
Contributor Author

/ready

@github-actions github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jul 31, 2026
@hubcio

hubcio commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

@Standing-Man something is still wrong:
image

it's building *-rust-tests tasks for all SDKs. it doesnt look good.

/author

@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jul 31, 2026
Comment thread .github/actions/rust/pre-merge/action.yml Outdated
Comment thread .github/actions/rust/pre-merge/action.yml Outdated
Comment thread .github/workflows/_test_rust.yml Outdated
Comment thread .github/workflows/_test_rust.yml Outdated
hubcio
hubcio previously approved these changes Aug 3, 2026
spetz
spetz previously approved these changes Aug 3, 2026
mmodzelewski
mmodzelewski previously approved these changes Aug 3, 2026
numinnex
numinnex previously approved these changes Aug 3, 2026
@hubcio

hubcio commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

overall i'm happy with the speedup itself, but i still don't like this skipped jobs noise:
image

@Standing-Man could you please think of restructuring the code so that we would cut this noise? can be done in in next PR / future.

@hubcio

hubcio commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

I took a peek at CI logs. why are we uploading something to codecov in job build-rust-tests? the tests itself were not yet executed.
image
https://github.com/apache/iggy/actions/runs/30797637786/job/91636081622?pr=3749

@hubcio hubcio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hubcio
hubcio merged commit e2bbaea into apache:master Aug 3, 2026
101 checks passed
@github-actions github-actions Bot removed the S-waiting-on-author PR is waiting on author response label Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants