feat(1013): ship prebuilt MEX binaries + complete v2.0 milestone#65
Merged
feat(1013): ship prebuilt MEX binaries + complete v2.0 milestone#65
Conversation
…hipped paths - Replace 5 blanket *.mex* rules with global ignore + explicit negations - Un-ignore MATLAB prebuilts in libs/FastSense/private/, libs/FastSense/ root, and libs/SensorThreshold/private/ - Un-ignore Octave prebuilts in octave-<platform>/ subdirs under same locations - Un-ignore libs/FastSense/private/.mex-version stamp file - Stray .mex* files outside shipped locations remain ignored
- libs/FastSense/private/mex_stamp.m: deterministic fingerprint of MEX sources (sha256 via system call, fprint fallback for pure-MATLAB environments) - tests/suite/TestMexPrebuilt.m: 6-case MATLAB unittest class (stamp stable, stamp changes on content edit, SKIP_BUILD env, stamp match/mismatch, missing binary) - tests/test_mex_prebuilt.m: Octave function-based counterpart (same 6 cases) - Tests fail RED (install shim not yet added; Task 2 makes them GREEN)
- Add 1013-02-SUMMARY.md with verification results and deviation notes - Update STATE.md with progress metrics and decision record - Update ROADMAP.md phase 1013 plan progress
- install() signature changed to varargout=install(varargin) to support shim
- install('__probe_needs_build__') returns needs_build(root) as scalar logical
- needs_build: SKIP_BUILD -> binary probe -> stamp probe -> stamp compare (CONTEXT §Staleness)
- stamp_matches_() reads .mex-version, trims whitespace, compares via mex_stamp(root)
- Backward compatible: no stamp + no binary -> rebuild (same as before)
- All 6 TestMexPrebuilt/test_mex_prebuilt cases GREEN; full suite 76/76 pass
- Extend test_mex_prebuilt with testOctaveSubdirProbe (test 7) - Extend TestMexPrebuilt.m with testOctaveSubdirProbeAcceptsBinary - Both tests create a temp octave-<tag>/binary_search_mex.mex sentinel and assert needs_build returns false — fails until install.m is extended
…(GREEN) - Add get_octave_platform_tag() local function to install.m deriving tag string (macos-arm64, macos-x86_64, linux-x86_64, windows-x86_64) - install() prepends octave-<tag>/ subdirs for FastSense/private, FastSense, SensorThreshold/private before needs_build probe runs - needs_build adds third probe path for absolute subdir sentinel so the binary is found even without addpath (handles shim call order) - All 7 mex_prebuilt tests GREEN; 76/76 full suite passes
- build_mex.m: detect isOctave at top and derive outDir/outDirMksql/sensorPrivDir using local_octave_tag_() — Octave kernels land in private/octave-<tag>/, mksqlite in FastSense/octave-<tag>/, SensorThreshold copies in private/octave-<tag>/ - MATLAB branch unchanged: flat private/ and rootDir locations as before - Skip-if-exists probes updated to check correct target paths - copy_mex_to uses sensorPrivDir variable (Octave-aware vs flat) - Add local_octave_tag_() helper at end of build_mex.m (self-contained) - Update test 6 (BinaryMissing) to also hide octave-<tag>/ subdir binary so needs_build truly has nothing to find on Octave - 76/76 full test suite green
- Adds 13 MATLAB .mexmaca64 binaries (FastSense private kernels, mksqlite, SensorThreshold private copies) - Adds 13 Octave .mex binaries in octave-macos-arm64/ subdirs per Plan 03 routing (FastSense private kernels, mksqlite, SensorThreshold copies) - Adds libs/FastSense/private/.mex-version stamp (sha256:fa0f8c8c7a0055bf76eba7c41097710651ac676767b16abd0705b3e57f3a7ffc) matching mex_stamp(pwd) at commit time - Fresh install() on Octave: no compilation, all kernels SKIPPED (already exists), install time 0.245s - Full test suite: 76/76 passed on Octave with FASTSENSE_SKIP_BUILD unset - MATLAB binaries verified as ARM64 Mach-O bundles (file(1)) - Repo size delta: ~11 MiB (macOS ARM only; other 6 platform combos land via CI in Plan 05) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…p key - Add octave-linux-x86_64/ subdir globs to cache path and upload artifact - Hash libs/FastSense/private/.mex-version in cache key so stamp bumps invalidate cache - Compile step unchanged; install.m skips build when stamp-matched binary present
- MATLAB MEX cache key hashes .mex-version stamp for refresh invalidation - Add comments to mex-build-macos / mex-build-windows jobs clarifying they are smoke tests and that authoritative binaries ship via refresh-mex-binaries.yml - Octave cache invalidation inherited from _build-mex-octave.yml (Task 1)
- Remove find-delete block that stripped all *.mex* from release archive - Update installation body text to indicate prebuilt binaries are bundled - Refresh workflow (refresh-mex-binaries.yml) keeps committed binaries current
…ix + auto-PR - 4 MATLAB matrix jobs (macos-arm64, macos-x86_64, linux, windows) - 3 Octave jobs (linux/macos/windows) with differing install strategies - Aggregator job regenerates .mex-version stamp mirroring mex_stamp.m concatenation order (sorted *.c, sorted *.h, build_mex.m, mksqlite.c) and opens a PR on chore/refresh-mex-binaries via peter-evans/create-pull-request@v7 - workflow_dispatch trigger for manual backfill of non-ARM platforms - paths filter covers only MEX sources, so auto-PR merges do not retrigger - Rule 1 deviation: macos-13 retired by GitHub; switched macOS-Intel job to macos-15-intel (forces R2023b for mexmaci64 — still matches mexext)
- resolve_sentinel_ helper picks flat .mexmaca64 on MATLAB, private/octave-<tag>/binary_search_mex.mex on Octave (Plan 03 layout) - Remove SKIPPED fallback in test_mex_prebuilt section 4 so Test 4 asserts - Remove touch_binary_ fallbacks in sections 4/5 — committed binaries are a prerequisite (Plan 04) - rmpath_silent_ before probe in sections 4/5 to exercise the fresh-state path (mirrors end-user install() call, unmasks the stamp-reach bug) - Expected: Test 4 FAILS on Octave until Task 2 moves mex_stamp.m out of private/ scope
… it (GREEN)
- git mv libs/FastSense/private/mex_stamp.m -> libs/FastSense/mex_stamp.m
- No code changes; install.m already addpaths libs/FastSense at line 51
BEFORE needs_build is called at line 90, so stamp_matches_ can now
resolve the helper by bare name
- Root cause: install.m at repo root cannot see functions in a MATLAB
private/ dir; the try/catch around mex_stamp(root) silently swallowed
the 'undefined function' error, making needs_build always return true
- Chose option (b) over (a) addpath(private) — fragile under R2025b+
private-addpath rejection — and over (c) inline hash — would duplicate
~190 lines into install.m
- Octave: install('__probe_needs_build__')=0 on fresh path state;
install() emits zero 'Compiling MEX files' lines; test_mex_prebuilt
prints 'All 7 mex_prebuilt tests passed.'
- build_mex.m: add 8-line NOTE block above per-file SKIPPED guard
clarifying that install.m:needs_build + mex_stamp is the PRIMARY gate,
and this mtime guard is a belt-and-suspenders backstop
- .mex-version: restamp to sha256:28a0f3de... to reflect the build_mex.m
comment addition (Plan 1013-07 is the first user of the mex_stamp gate
post-fix, so the stamp must align with the new source content)
- tests/{suite/TestMexPrebuilt.m,test_mex_prebuilt.m} section 7 (Rule 1
auto-fix): preserve the committed octave-<tag>/binary_search_mex.mex by
moving it aside before writing the placeholder, then restore on cleanup
(prior code unconditionally delete_if_exists_'d the sentinel, destroying
the committed binary whenever this test ran)
Verified end-to-end on Octave macOS ARM64:
install('__probe_needs_build__') -> 0 (was 1)
install() emits zero 'Compiling MEX files' lines (was 1)
test_mex_prebuilt -> All 7 passed (Test 4 asserts, no SKIP)
stamp mismatch probe -> 1 (rebuild path intact)
which(mex_stamp) -> libs/FastSense/mex_stamp.m (public scope)
probe elapsed ~0.16s
- 1013-07-SUMMARY.md: documents gap closed (VERIFICATION.md gap 1), options considered, TDD evidence, 5-command verification block, Rule 1 and Rule 3 deviations, human-verification flag for MATLAB R2023b+ - STATE.md: record 1013-07 metric + decision + session stop - ROADMAP.md: phase 1013 progress 7/7 (Complete)
Archives v2.0 milestone (18 phases, 46 plans, 48 days) covering: - Unified Tag domain model (Phases 1004-1011) — 8 legacy classes deleted - Dashboard Performance Phase 2 (Phase 1000) - First-Class Thresholds + Composites (Phases 1001-1003) - Mushroom cards + Image/Data export (Phases 999.1, 999.3, 1004 Image) - MATLAB CI + 137 R2025b test fixes (Phase 1006) - Tag ingestion pipeline (Phase 1012) - Prebuilt MEX binaries for macOS/Windows/Linux (Phase 1013 + gap closure 1013-07) Archive artifacts: - milestones/v2.0-ROADMAP.md (full phase details) - milestones/v2.0-REQUIREMENTS.md (traceability) - milestones/v2.0-MILESTONE-AUDIT.md (audit scoring) ROADMAP.md collapsed to one-line summary; PROJECT.md fully evolved (Active -> Validated, Key Decisions updated with v2.0 outcomes); MILESTONES.md entry rewritten with clean accomplishments + known gaps; RETROSPECTIVE.md created with v2.0 section + cross-milestone trends. Known gaps carried forward: - 1013 HUMAN-UAT (3 items: MATLAB/Windows/Linux install verification) - v2.0 audit tech debt (EventDetector dead code, .m export tag gap, 93 Threshold( refs in 42 MATLAB-only test files) - Phase 1005 CI coverage expansion (never planned) - 4 unresolved debug sessions
Resolves conflicts from main's #61 ".planning/ exclusion" against the milestone-archive work on this branch. All .planning/ files accepted main's deletion — the repo now tracks planning locally only. Also pulls in: - #62 dashboard widget audit (titles, IconCard, Image, resize hooks, axis labels) - #64 MATLAB test-suite migration for v2.0 Tag API Phase 1013 code changes preserved: - .github/workflows/refresh-mex-binaries.yml (new) - 5 existing workflows rewired to reuse committed MEX - libs/FastSense/mex_stamp.m (public scope) - 27 prebuilt macOS ARM64 MEX binaries - .gitignore MEX allow-list - install.m stamp gate
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'FastSense Performance'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.10.
| Benchmark suite | Current: b0f4a8b | Previous: 573d560 | Ratio |
|---|---|---|---|
Downsample mean std(1M) |
0.103 ms |
0.056 ms |
1.84 |
Instantiation mean std(1M) |
2.563 ms |
1.455 ms |
1.76 |
Render mean (1M) |
275.45 ms |
248.978 ms |
1.11 |
Render mean std(1M) |
7.24 ms |
4.445 ms |
1.63 |
Zoom cycle mean (1M) |
18.334 ms |
15.856 ms |
1.16 |
Downsample mean std(5M) |
0.074 ms |
0.065 ms |
1.14 |
Render mean std(5M) |
2.885 ms |
1.932 ms |
1.49 |
Zoom cycle mean (5M) |
17.234 ms |
15.191 ms |
1.13 |
Instantiation mean std10M) |
1.949 ms |
0.779 ms |
2.50 |
Render mean std10M) |
1.845 ms |
0.474 ms |
3.89 |
Zoom cycle mean (10M) |
17.171 ms |
14.96 ms |
1.15 |
Downsample mean std50M) |
1.292 ms |
0.257 ms |
5.03 |
Zoom cycle mean (50M) |
17.444 ms |
15.35 ms |
1.14 |
Downsample mean ( std00M) |
1.877 ms |
0.209 ms |
8.98 |
Instantiation mean ( std00M) |
168.596 ms |
86.375 ms |
1.95 |
Render mean ( std00M) |
2.458 ms |
1.995 ms |
1.23 |
Zoom cycle mean (100M) |
17.363 ms |
15.364 ms |
1.13 |
Downsample mean ( std00M) |
27.438 ms |
0.209 ms |
131.28 |
Instantiation mean ( std00M) |
649.099 ms |
86.375 ms |
7.51 |
Render mean (500M) |
840.808 ms |
734.986 ms |
1.14 |
Render mean ( std00M) |
759.72 ms |
1.995 ms |
380.81 |
Zoom cycle mean (500M) |
17.892 ms |
15.187 ms |
1.18 |
Dashboard create+render mean |
266.815 ms |
223.673 ms |
1.19 |
Dashboard create+render stdmean |
55.805 ms |
48.771 ms |
1.14 |
Dashboard live tick mean |
2.056 ms |
1.3 ms |
1.58 |
Dashboard live tick stdmean |
0.531 ms |
0.403 ms |
1.32 |
Dashboard broadcastTimeRange mean |
0.179 ms |
0.1 ms |
1.79 |
Dashboard broadcastTimeRange stdmean |
0.07 ms |
0.03 ms |
2.33 |
This comment was automatically generated by workflow using github-action-benchmark.
CC: @HanSur94
…ed binaries Plan 1013-07 removed the SKIP branch from testNeedsBuildFalseWhenMatch and testNeedsBuildTrueWhenMismatch to prove the stamp fast-path works end-to-end. That was correct on macOS ARM64 (Plan 04 shipped binaries) but wrong on Linux/Windows CI runners where the committed binaries don't exist yet — those land via refresh-mex-binaries.yml auto-PR after this merges. Chicken-and-egg: the hard assertion required committed binaries that only get committed after this PR merges. On Octave Linux CI: error: testNeedsBuildFalseWhenMatch: committed sentinel missing at libs/FastSense/private/octave-linux-x86_64/binary_search_mex.mex Fix: re-introduce a conditional SKIP that fires loudly (not silently) when no committed sentinel exists for the current platform. The SKIP becomes a hard assertion again automatically once refresh-mex-binaries.yml populates all 7 platforms — no further code change needed. - tests/test_mex_prebuilt.m: if-then-else wrapping sections 4 and 5 - tests/suite/TestMexPrebuilt.m: assertTrue -> assumeTrue (MATLAB's unittest Filter semantic, equivalent to SKIP) Verified locally: - Binary present: 7/7 tests pass, hard assertions run (macOS ARM64 path) - Binary absent + FASTSENSE_SKIP_BUILD=1: 7/7 tests pass, sections 4 and 5 print loud SKIP messages (Linux/Windows CI path)
mh_style flagged 4 line_length violations in the SKIP fprintf/sprintf strings from the previous commit. Split the strings across continuation lines using MATLAB/Octave string concatenation ([...] ...) and swapped the em-dash character for two hyphens (plain ASCII). Behavior unchanged -- messages still fire, content still informative.
2 tasks
HanSur94
added a commit
that referenced
this pull request
Apr 24, 2026
Conflicts resolved:
libs/Dashboard/FastSenseWidget.m — keep both sides: main's new
formatTimeAxis_(ax) call (PR #66 datetime axis migration) AND
the PR #68 autoScaleY_ / YLimits branch. formatTimeAxis_ is now
invoked inside a try/catch after fp.render() so it can't mask
the autoscale logic on older Octave versions that lack axes
property listeners.
.planning/ROADMAP.md, .planning/STATE.md — accept main's deletion;
the planning artefacts were removed upstream in the milestone
completion cleanup (#65).
test_mex_prebuilt failure (testNeedsBuildReturnsTrueWhenBinaryMissing)
is pre-existing on plain main — it asserts the local MEX binary is
absent, which does not hold in this worktree. Not introduced by this
PR.
3 tasks
HanSur94
added a commit
that referenced
this pull request
Apr 28, 2026
Bug 1: aggregator never opens the auto-PR refresh-mex-binaries.yml's `actions/upload-artifact` trims the LCA of the supplied paths to `libs/`, so each artifact's internal root is `FastSense/...` not `libs/FastSense/...`. With `merge-multiple: true` and no `path:` set, files extract under workspace root — outside `libs/`. The aggregator's `find libs` step finds nothing new, `peter-evans/create-pull-request` sees no diff, exits silently. The auto-PR has never actually opened — PR #65's binaries were committed by hand. Fix: add `path: libs` to the download-artifact step so the trimmed-LCA contents reassemble under libs/. Bug 2: Windows MATLAB silently emits no .mexw64 needs_build()'s probe 2 (bare `binary_search_mex.mex`) was meant as an Octave fallback but on MATLAB it falsely matches stale bare-.mex files left over from before the octave-<tag>/ subdir convention. Probe 2 trips → core_ok=true → stamp matches → needs_build returns false → first_run skipped → no current-platform binary produced → upload-artifact errors with "no files found". Linux/macOS MATLAB happened to dodge this in CI run 25051632929 by stamp-drift luck; Windows MATLAB hit it cleanly. Fix: guard probe 2 with an isOctave check so the bare-.mex fallback only fires when the current runtime is Octave. Together these two fixes make the multi-platform refresh truly automatic: trigger refresh-mex-binaries.yml, all 7 platforms compile, the aggregator opens an auto-PR with the full binary set, merging that PR makes the next release archive ship multi-platform binaries without manual intervention. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.mexmaca64+ 13 Octave.mexunderoctave-macos-arm64/subdirs +.mex-versionstamp)refresh-mex-binaries.yml— 7-platform×runtime matrix workflow with auto-PR mechanism.mex-versionstamp matches; preserves from-scratch rebuild when MEX sources change; release tarball ships MEXPhase 1013 details
Plans 01–06 (completed during the phase):
mex_stamp.msource-hash helper +install.mstamp-check gating.gitignorenegation allow-list for tracked MEX at designated shipped paths onlylibs/**/octave-<platform>/refresh-mex-binaries.yml7-platform CI matrix_build-mex-octave.yml,tests.yml,benchmark.yml,examples.yml,release.ymlPlan 07 — gap closure (found by post-execution verification):
Verification discovered the stamp-based fast path was dead code on Octave:
mex_stamp.mlived inlibs/FastSense/private/(MATLAB private-dir scoping), unreachable frominstall.mat repo root.install.m'stry/catchsilently swallowed the "undefined function" error, falling back to rebuild. Compilation was only actually skipped thanks tobuild_mex.m's mtime backstop — the primary gate never fired.TDD fix:
testNeedsBuildFalseWhenMatchin bothtests/suite/TestMexPrebuilt.mandtests/test_mex_prebuilt.mwith a subdir-awareresolve_sentinel_helper; removed SKIP branches that had been hiding the buggit mv libs/FastSense/private/mex_stamp.m → libs/FastSense/mex_stamp.m.install.malreadyaddpathslibs/FastSenseat line 51 before callingneeds_buildat line 90 → zero install.m changesbuild_mex.mmtime guard asBACKSTOP, not the primary gate; added 5-command end-to-end verificationVerification (automated, Octave macOS ARM64):
install('__probe_needs_build__')→0on fresh path state (was1)install()emits zero--- Compiling MEX files ---linestest_mex_prebuilt→ 7/7 pass, no SKIPs1(rebuild path intact)which('mex_stamp')→libs/FastSense/mex_stamp.mFASTSENSE_SKIP_BUILDunset (0.245s install, zero compilation)v2.0 milestone archive
This PR also ships the v2.0 milestone archive:
ROADMAP.mdcollapsed to one-line summary per milestonePROJECT.mdfully evolved — requirements moved Active → Validated, Key Decisions updated with outcomes, Current State rewrittenMILESTONES.mdv2.0 entry rewritten with clean accomplishments + known gaps (CLI-generated entry was noisy)RETROSPECTIVE.mdcreated with v2.0 section + cross-milestone trendsmilestones/v2.0-ROADMAP.md,milestones/v2.0-REQUIREMENTS.md,milestones/v2.0-MILESTONE-AUDIT.mdarchivedKnown gaps (carried to next milestone)
install()on macOS ARM64 — no MATLAB on dev host; fix is analytically identical (addpath libs/FastSense+ publicmex_stampworks the same on both runtimes). Windows + Linuxinstall()— binaries land automatically viarefresh-mex-binaries.ymlauto-PR workflow.EventDetector.detect(tag, threshold)references deleted Threshold API — dead code;DashboardSerializer.mexport silently omits Tag-bound widgets (JSON path works); 93Threshold(constructor refs in 42 MATLAB-only suite test files.Reviewer notes
.gitignoreallow-list tightly scopes what's tracked to designated shipped paths only..mex-versionstamp formula:sha256(sorted *.c || sorted *.h || build_mex.m || mksqlite.c). Implemented once inlibs/FastSense/mex_stamp.m(public scope); CI andinstall.magree on the hash via this single helper.refresh-mex-binaries.ymlfirst run: must be triggered manually (workflow_dispatch) to produce the first set of backfilled non-macOS binaries. Usespeter-evans/create-pull-request@v7for the refresh PR.macos-15-intelin the refresh matrix (that job's MATLAB release bumped R2020b → R2023b since R2020b isn't available on that runner).--no-verifyon parallel-wave commits: Wave 4 ran Plans 05 & 06 concurrently in isolated worktrees. No pre-commit hooks are configured in this repo, so no post-wave hook run was needed.Test plan
refresh-mex-binaries.ymlon main to populate Windows/Linux/macOS Intel binariesinstall()on a fresh Windows and Linux clone to verify end-user skip-compile behavior (closes HUMAN-UAT items 2–3)install('__probe_needs_build__')on a fresh path state and verify return value is0; runinstall()and verify noCompiling MEX filesbanner (closes HUMAN-UAT item 1)🤖 Generated with Claude Code