Skip to content

ci: split fast tests from coverage tracing - #411

Merged
immrsd merged 3 commits into
mainfrom
ci/raise-test-timeout
Jun 26, 2026
Merged

ci: split fast tests from coverage tracing#411
immrsd merged 3 commits into
mainfrom
ci/raise-test-timeout

Conversation

@immrsd

@immrsd immrsd commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

What

Splits the single test job into two, per @0xNeshi's review:

  • testsui move test (no tracing): the fast regression gate (~5s even for math/core). Gates merges.
  • coveragesui move test --trace + sui move coverage lcov + Codecov: isolated, continue-on-error: true (informational, never blocks a merge), runs only after test is green.

Why the 30-minute bump wasn't enough

Profiled math/core locally (Sui 1.71.1):

  • sui move test (no trace): ~5.5s.
  • sui move test --trace (what CI runs): the single random test median_matches_sorted_reference_u256 alone takes 7m21s and writes 4.1 GB of traces. It quick_sort!s a Sui-generated random vector<u256> that can be very large, and --trace (full mode) records every VM instruction. The whole suite is many GB and never finishes inside 30 min.

--trace (full) is mandatory for sui move coverage lcov — the lighter --trace function-only / instruction-only modes produce no line records — so the cost can't simply be switched off.

The fix

@0xNeshi's split keeps the gate fast and moves the slow trace off the critical path. The coverage job also passes --rand-num-iters 1: coverage only needs each line executed once, and the full randomized property iterations still run in the fast test job, so this cuts trace volume without losing coverage.

Deferred follow-ups (math/core's domain)

  • The durable cure for the trace time is bounding the random vector length in math/core's ~9 heavy median/quick_sort random tests (one of them produces the 4.1 GB above). Left out here since it changes those property tests.
  • traces/ is not gitignored — worth adding so a local --trace run can't accidentally commit gigabytes.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores
    • Split automated checks into two stages for a faster primary test run and a separate coverage run.
    • Kept the main test job on a shorter timeout while allowing coverage collection to run independently.
    • Adjusted coverage execution to reduce trace output and improve reliability of test reporting.

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7225e0c8-ba84-4eb7-b295-7a9871d0dc85

📥 Commits

Reviewing files that changed from the base of the PR and between 093afa9 and 1aae159.

📒 Files selected for processing (1)
  • .github/workflows/test.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/test.yml

📝 Walkthrough

Walkthrough

The workflow splits CI testing into separate test and coverage jobs, updates job-level permissions and timeout settings, and changes the coverage test invocation to reduce trace volume.

Changes

CI workflow test and coverage

Layer / File(s) Summary
Test and coverage job setup
.github/workflows/test.yml
The test job drops its job-level permissions block, and a separate coverage job is added with its own timeout, permissions, error handling, and strategy settings.
Coverage trace command
.github/workflows/test.yml
The coverage job runs sui move test --trace --rand-num-iters 1 instead of the traced default iteration command.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

A bunny hopped through CI light,
Two jobs now split the testing flight.
🐇 One traces less, one runs with care,
And coverage blooms with tidy flair.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main CI change: splitting fast tests from coverage tracing.
Description check ✅ Passed The description thoroughly explains what changed, why, and the fix, but it omits the issue reference and checklist items from the template.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/raise-test-timeout

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.26%. Comparing base (23af906) to head (1aae159).

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #411       +/-   ##
===========================================
+ Coverage   83.26%   96.26%   +12.99%     
===========================================
  Files          29       29               
  Lines        2307     2968      +661     
  Branches      732      732               
===========================================
+ Hits         1921     2857      +936     
+ Misses        348       66      -282     
- Partials       38       45        +7     
Flag Coverage Δ
contracts/access 65.46% <ø> (ø)
contracts/allowance 52.00% <ø> (ø)
contracts/finance 25.57% <ø> (ø)
contracts/utils 44.09% <ø> (ø)
math/core 86.71% <ø> (?)
math/fixed_point 60.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread .github/workflows/test.yml
@immrsd immrsd changed the title ci: raise Test job timeout to 30 minutes ci: split fast tests from coverage tracing Jun 26, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/test.yml:
- Around line 134-144: The coverage job is missing the setup dependency, so its
matrix cannot read needs.setup.outputs.packages. Update the coverage job’s needs
सूची to include setup alongside test, and verify the matrix expression in the
coverage job still references needs.setup.outputs.packages so the job can
instantiate correctly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 06d0bcc3-250c-4c84-a949-00336abb7bec

📥 Commits

Reviewing files that changed from the base of the PR and between 7325eec and 093afa9.

📒 Files selected for processing (1)
  • .github/workflows/test.yml

Comment thread .github/workflows/test.yml Outdated
@immrsd
immrsd merged commit acbd3e7 into main Jun 26, 2026
29 checks passed
@immrsd
immrsd deleted the ci/raise-test-timeout branch June 26, 2026 13:28
@bidzyyys bidzyyys linked an issue Jun 26, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: math/core tests CI job takes too long to run

3 participants