feat: parallel CI integration tests #605
Conversation
Manual Deploy AvailableYou can trigger a manual deploy of this PR branch to testnet: Alternative: Comment
Comment updated automatically when the PR is synchronized. |
WalkthroughThe CI workflow is restructured to use a matrix strategy for executing 11 integration test batches in parallel, separating build and test jobs. The Makefile's Changes
Sequence Diagram(s)sequenceDiagram
participant GH as GitHub
participant build as build job
participant matrix as run_integration_tests<br/>(matrix 11×)
GH->>build: Trigger (non-PR or not draft)
build->>build: Checkout repo (v5)
build->>build: Build project & test programs
build->>matrix: Job complete
matrix->>matrix: Checkout repo (v5)
matrix->>matrix: Setup build env & Solana
matrix->>matrix: RUN_TESTS=batch_X<br/>make ci-test-integration
matrix->>matrix: Execute test suite
Note over matrix: All 11 batches<br/>run in parallel
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20–25 minutes
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/ci-test-integration.yml(2 hunks)
🧰 Additional context used
🪛 actionlint (1.7.8)
.github/workflows/ci-test-integration.yml
26-26: got unexpected character '+' while lexing expression, expecting 'a'..'z', 'A'..'Z', '_', '0'..'9', ''', '}', '(', ')', '[', ']', '.', '!', '<', '>', '=', '&', '|', '*', ',', ' '
(expression)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: run_make_ci_lint
- GitHub Check: run_make_ci_test
🔇 Additional comments (1)
.github/workflows/ci-test-integration.yml (1)
17-25: Confirm matrix batch labels map to actual test discovery logic.The matrix defines 5 batch labels (e.g., "schedulecommit", "chainlink,cloning", etc.) that are passed via
RUN_TESTS. Verify that the test discovery/filtering logic in the build system correctly parses comma-separated values and only runs the intended test suites per batch.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/ci-test-integration.yml (1)
14-47: Update deprecated GitHub Actions versions and verify runner configuration.The workflow uses deprecated action versions that will fail after their retirement dates:
- Line 20:
actions/checkout@v2is outdated; upgrade tov4(or v5)- Line 41:
actions/upload-artifact@v3is deprecated (retired Jan 30, 2025); upgrade tov4(or v5)The
extra-largerunner label on line 16 is valid as a custom self-hosted runner label but should be documented in your repository's runner configuration or CI documentation to clarify this is intentional.Apply this diff to update the checkout action:
- name: Checkout this magicblock-validator - uses: actions/checkout@v2 + uses: actions/checkout@v4Apply this diff to update the upload-artifact action:
- name: Upload build artifacts - uses: actions/upload-artifact@v3 + uses: actions/upload-artifact@v4
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/ci-test-integration.yml(2 hunks)Makefile(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.8)
.github/workflows/ci-test-integration.yml
16-16: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
41-41: the runner of "actions/upload-artifact@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
51-51: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
64-64: the runner of "actions/checkout@v2" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
78-78: the runner of "actions/download-artifact@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: run_make_ci_test
- GitHub Check: run_make_ci_lint
- GitHub Check: run_make_ci_format
🔇 Additional comments (3)
Makefile (1)
54-55: LGTM—RUN_TESTS is now properly propagated.Explicitly passing
RUN_TESTS=$(RUN_TESTS)to the recursive make invocation ensures the variable reliably reaches the downstream test target and aligns with project conventions. The removal ofcargo build --lockedfrom this target (now delegated to the CI workflow) is the correct separation of concerns..github/workflows/ci-test-integration.yml (2)
91-92: LGTM—RUN_TESTS environment variable correctly propagated.The
RUN_TESTSenvironment variable is correctly set from the matrix value (line 92), enabling themake ci-test-integrationtarget to filter tests by batch. This aligns with the Makefile change that explicitly forwards this variable downstream.
52-60: Matrix batch coverage is verified as complete and accurate.All 11 integration tests from the Makefile (schedulecommit, chainlink, cloning, restore_ledger, magicblock_api, table_mania, committor, pubsub, config, schedule_intents, task-scheduler) are included in the workflow matrix with no gaps or overlaps. Each batch name corresponds directly to a valid RUN_TESTS value.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/ci-test-integration.yml(2 hunks)
🧰 Additional context used
🪛 actionlint (1.7.8)
.github/workflows/ci-test-integration.yml
16-16: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
51-51: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: run_make_ci_format
- GitHub Check: run_make_ci_test
- GitHub Check: Build Project
🔇 Additional comments (4)
.github/workflows/ci-test-integration.yml (4)
16-16: Verify custom runner labelextra-large.actionlint flags
extra-largeas an unknown runner label. This may be a custom self-hosted runner label registered in your repository settings. If this is intentional, you can suppress the warning by adding the label toactionlint.yaml. If not intentional, replace with a standard GitHub-hosted runner label (e.g.,ubuntu-latest-16-coresor equivalent for your needs).Also applies to: 51-51
52-60: Batch strategy looks sound.The 5-batch partitioning appropriately parallelizes integration tests across distinct test categories, and
fail-fast: falseensures all batches run to surface any failures. Matrix logic is correct.
83-92: No action required—RUN_TESTS forwarding is correctly implemented.The Makefile target
ci-test-integration(line 55) explicitly forwards the RUN_TESTS variable usingRUN_TESTS=$(RUN_TESTS) $(MAKE)..., which aligns with the project convention and ensures proper variable propagation from the GitHub Actions workflow step to the recursive make invocation.
77-92: Artifact paths resolve correctly; no issue identified.The
DIRvariable in each Makefile correctly points to that file's directory. When artifacts are downloaded to the repo root and make runs withworking-directory: magicblock-validator, the relative paths in test-integration/Makefile resolve correctly:
$(DIR)target/deployfindsmagicblock-validator/test-integration/target/deploy$(DIR)../target/deployfindsmagicblock-validator/target/deployThe workflow artifact handling is sound.
lucacillario
force-pushed
the
feat/parallel-ci-integration-tests
branch
from
17472ee to
eaabcad
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/ci-test-integration.yml(2 hunks)Makefile(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.8)
.github/workflows/ci-test-integration.yml
16-16: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
51-51: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Build Project
- GitHub Check: run_make_ci_lint
- GitHub Check: run_make_ci_test
🔇 Additional comments (3)
.github/workflows/ci-test-integration.yml (2)
33-38: ✅ Build job structure and artifact upload are well-configured.The separation of build and test into distinct jobs enables parallel matrix-driven testing. The artifact upload correctly captures both
target/directories. The environment variable forwarding (RUN_TESTS) integrates properly with the Makefile.
49-92: ✅ Matrix-driven parallel integration test execution is well-structured.The new
run_integration_testsjob properly implements parallel test batching with five independent test runs. The job dependency onbuildensures artifacts are available before tests start. The RUN_TESTS environment variable is correctly set per batch and forwarded through make. The disabled fail-fast flag ensures all batches complete regardless of individual failures.Makefile (1)
54-55:⚠️ Original review comment is incorrect—no changes required.The removal of
cargo build --lockedfromci-test-integrationis intentional and correct. The GitHub Actions workflow has been refactored to use a build/test separation pattern:
- Build job runs
cargo build --locked, uploads artifacts- Test jobs (5 parallel batches) download artifacts and run
make ci-test-integrationwith differentRUN_TESTSvaluesThis avoids rebuilding the project 5 times and is a common CI optimization pattern. The
RUN_TESTSenvironment variable is properly passed through, enabling test batching. The Makefile change aligns with the intended workflow design.Local developers can run the
testtarget (which includes the build step) if they want a standalone build-and-test command.Likely an incorrect or invalid review comment.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
.github/actions/setup-build-env/action.yml(2 hunks).github/workflows/ci-test-integration.yml(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.8)
.github/workflows/ci-test-integration.yml
16-16: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
45-45: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: run_make_ci_test
- GitHub Check: Build Project
- GitHub Check: run_make_ci_lint
- GitHub Check: run_make_ci_format
🔇 Additional comments (3)
.github/actions/setup-build-env/action.yml (1)
17-19: Well-integrated cache configuration additions.The new
cache_restore_keysinput is properly declared as optional and correctly passed to the rust-cache step. The additional cache flags (cache-targets,cache-all-crates,cache-on-failure) enhance caching resilience across workspace targets.Also applies to: 41-41, 45-47
.github/workflows/ci-test-integration.yml (2)
14-41: Build job structure and caching strategy are well-configured.The job properly includes conditional execution, current action versions, and a robust cache strategy using both SHA-unique keys and fallback patterns. The addition of
make -C test-integration programsensures test infrastructure is compiled before dependent jobs consume the cache.
16-16: Verify actionlint configuration for custom runner label.The
extra-largerunner label on lines 16 and 45 continues to trigger actionlint warnings. Past review comments noted this is a custom self-hosted runner label that requires configuration. Ensure one of the following:
If
extra-largeis a self-hosted runner label, create or update.github/actionlint.yamlwith:self-hosted-runner-labels: - extra-largeIf this should use a GitHub-hosted runner, replace
extra-largewith a standard label likeubuntu-latest-8-coresorubuntu-latest-16-cores.Also applies to: 45-45
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/ci-test-integration.yml(2 hunks)
🧰 Additional context used
🪛 actionlint (1.7.8)
.github/workflows/ci-test-integration.yml
16-16: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
42-42: label "extra-large" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Build Project
- GitHub Check: run_make_ci_lint
- GitHub Check: run_make_ci_test
- GitHub Check: run_make_ci_format
🔇 Additional comments (2)
.github/workflows/ci-test-integration.yml (2)
33-38: Review comment is incorrect. Artifacts are handled via rust-cache shared-key, not explicit upload/download.The workflow correctly uses
Swatinem/rust-cache@v2with a shared cache key ("magicblock-validator-ci-test-integration-v000") across both thebuildandrun_integration_testsjobs. This GitHub Actions caching mechanism automatically shares the compiled target directories between jobs without requiring explicitupload-artifact/download-artifactsteps. Therun_integration_testsjob will restore cached artifacts on the second run, not rebuild from scratch. The workflow design is correct as-is.Likely an incorrect or invalid review comment.
24-29: The suggestedcache_restore_keysinput does not exist and should not be added.The
setup-build-envaction does not define or accept acache_restore_keysinput. The action's only defined inputs are:github_access_token,github_token,rust_toolchain_release, andbuild_cache_key_name. Caching is handled internally via theSwatinem/rust-cache@v2action, which is configured through thebuild_cache_key_nameparameter. Adding an unsupported input would have no effect and is unnecessary.Likely an incorrect or invalid review comment.
thlorenz
left a comment
thlorenz
left a comment
There was a problem hiding this comment.
Great, exactly what we need! LGTM.
* master: hotfix: unregistered metric COMMITTOR_INTENT_CU_USAGE (#612) feat: add metrics for intent execution time (#607) fix: don't increment subscription id for dups (#610) chore: broaden dependency range on the magic program api (#609) feat: parallel CI integration tests (#605) Handle uncleaned buffers + generalized tx send retries (#597) fix: support binary opcode (#604) fix: correct log imports (#606)
Summary by CodeRabbit