Skip to content

Move cmake-modules to integration tests#2649

Merged
sylvestre merged 5 commits intomozilla:mainfrom
Felixoid:cmake-modules-integration-tests
Mar 25, 2026
Merged

Move cmake-modules to integration tests#2649
sylvestre merged 5 commits intomozilla:mainfrom
Felixoid:cmake-modules-integration-tests

Conversation

@Felixoid
Copy link
Contributor

@Felixoid Felixoid commented Mar 23, 2026

A few improvements:

  • Use disk cache for the tests, avoid issues with GHA
  • Add a dedicated failing cmake 4.X test (example)
  • Move it to make test-tools to launch locally
  • Move files to tests/integration directory
  • Group logs in CI for easier reading

@codecov-commenter
Copy link

codecov-commenter commented Mar 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.38%. Comparing base (8d6cab9) to head (184b827).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2649   +/-   ##
=======================================
  Coverage   73.38%   73.38%           
=======================================
  Files          68       68           
  Lines       37338    37338           
=======================================
  Hits        27400    27400           
  Misses       9938     9938           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Felixoid
Copy link
Contributor Author

cc @TroyKomodo

@TroyKomodo
Copy link
Contributor

TroyKomodo commented Mar 24, 2026

This looks like a much better way to do the integration tests. More scalable!

Personally I would reach for snapshotting here for the json stats output since it shouldn't be changing run-to-run. Right now we just check that at least one file was cached.

This might be something for a broader change across all the integration tests.

Cc @sylvestre

@Felixoid
Copy link
Contributor Author

By snapshotting, I guess you mean the number of misses/hits. Because the full output looks redundant and can change at any time. Although the whole integration test checks whether the values change, not the exact numbers. Not sure if it's worth diverging now, or rather a small refactoring in the future.

Additionally, it was suspicious that tests work in docker and fail in https://github.com/mozilla/sccache/actions/runs/23459356336/job/68257326892?pr=2581#step:10:38. I added cmake 4.0 test with pip install, and now it catches the bug. I'll keep it here and create another issue.

@Felixoid Felixoid force-pushed the cmake-modules-integration-tests branch 2 times, most recently from 98d77fc to 42afcdd Compare March 24, 2026 08:44
@Felixoid Felixoid force-pushed the cmake-modules-integration-tests branch from 42afcdd to 184b827 Compare March 24, 2026 09:44
@sylvestre sylvestre merged commit f5edaf1 into mozilla:main Mar 25, 2026
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants