Skip to content

ci: release-only builds, remove benchmarks from CI - #5

Merged
godofecht merged 1 commit into
mainfrom
fix/release-ci-no-benchmarks
Mar 16, 2026
Merged

ci: release-only builds, remove benchmarks from CI#5
godofecht merged 1 commit into
mainfrom
fix/release-ci-no-benchmarks

Conversation

@godofecht

Copy link
Copy Markdown
Owner

Summary

  • CI only on PRs and release tags — no more builds on every push to main
  • Benchmarks removed from ctest — still compiled (catches build breaks) but not executed. Free CI runners give meaningless benchmark results and waste minutes.
  • Release pipeline — push a v* tag to build + upload binaries to GitHub Releases
  • TESTING.md — documents what runs, what's skipped, and why

What runs in CI now:

Test Time Status
TinyMLTests 20s runs
BasicTests <1s runs
Phase4FixedTest 2s runs
Phase5QuantizedTest 1s runs
Phase7AdvancedAttentionTest 5s runs
Phase10GraphNeuralTest 1s runs
LightweightAttentionTest 3s runs
Phase13TimeSeriesTest 8s runs
Phase11GenerativeTest <1s runs
Total ~40s

What's removed/disabled:

Item Reason
All benchmarks (5) Timing results meaningless on shared runners; some segfault
Phase8PhysicsInformedTest 16+ min — too slow
Phase8PhysicsComprehensiveTest Non-deterministic convergence
Phase12ReinforcementTest Segfault on Linux

Test plan

  • CI passes on this PR in ~2 min
  • TESTING.md is clear and accurate

🤖 Generated with Claude Code

**CI workflow overhaul:**
- PR checks: build + fast unit tests only (~2 min)
- Release builds: triggered by version tags (v*), uploads binaries
  to GitHub Releases as tarball
- Upgraded actions/checkout to v4

**Benchmarks removed from ctest:**
- SIMDBenchmark, AttentionBenchmark, SimpleAttentionBenchmark,
  ReinforcementLearningBenchmark, GenerativeModelsBenchmark
- Reason: benchmark results on shared CI runners are meaningless
  (hardware varies per run), waste CI minutes, and some segfault
- Benchmarks are still compiled (catches build breaks) but not run
- Run locally: cd build && ./bin/<BenchmarkName>

**TESTING.md added** documenting:
- What runs in CI and what doesn't
- Why each test/benchmark is skipped (with specific reasons)
- How to run the full suite locally
- Release process

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@godofecht
godofecht merged commit 7f5d772 into main Mar 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant