Add NDR symmetry testing framework for cross-language validation#100
Merged
stevevanhooser merged 5 commits intomainfrom Apr 11, 2026
Merged
Add NDR symmetry testing framework for cross-language validation#100stevevanhooser merged 5 commits intomainfrom
stevevanhooser merged 5 commits intomainfrom
Conversation
Mirrors the symmetry test framework introduced in NDI-matlab / NDI-python so that NDR-matlab and NDR-python can verify that they read and interpret example data identically. - Add +makeArtifacts and +readArtifacts sub-packages under tools/tests/+ndr/+symmetry with INSTRUCTIONS.md for each phase. - Add an example reader.readData pair that dumps channel metadata and a short sample of Intan RHD data to JSON and verifies it against artifacts produced by either the MATLAB or Python suites.
Runs the MATLAB +makeArtifacts and +readArtifacts test packages on pushes to main, pull requests targeting main, and manual dispatch. This workflow is separate from the regular run_tests.yml CI so that the cross-language symmetry suite can evolve independently.
Contributor
Two issues were causing the workflow to fail before any tests ran:
1. `addpath(genpath("+ndr"))` is invalid because MATLAB namespace
directories (anything starting with `+`) must not themselves
appear on the path -- only their parent directories. Replace
with `addpath(pwd)` plus `addpath(tools/tests)` so both the
repo-root `+ndr` and the test `+ndr/+symmetry` packages are
discoverable.
2. `ndr_Init` tried to `mkdir` a filecache path under `userpath`
which isn't writable on the CI runner. The symmetry tests only
need `ndr.fun.ndrpath()` (self-contained), so skip `ndr_Init`
entirely.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #100 +/- ##
==========================================
+ Coverage 31.95% 32.76% +0.81%
==========================================
Files 92 92
Lines 4810 4810
==========================================
+ Hits 1537 1576 +39
+ Misses 3273 3234 -39 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
The readData make-artifact test calls ndr.reader.intan_rhd methods that depend on vlt.string.strcmp_substitution from vhlab-toolbox-matlab. That dependency is declared in tools/requirements.txt but nothing was installing it in the symmetry CI. Call matbox.installRequirements(tools) to fetch and add it to the path, matching how NDI-matlab's symmetry CI handles requirements.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a comprehensive symmetry testing framework for NDR that enables validation of data reading and artifact generation across MATLAB and Python implementations. The framework ensures numerical parity between language ports by generating standardized artifacts and verifying them against expected values.
Key Changes
GitHub Actions Workflow (
test-symmetry.yml): Added automated CI/CD pipeline that:makeArtifactsandreadArtifactstest suitesArtifact Generation Tests (
+makeArtifacts/+reader/readData.m):ndr.reader.intan_rhdmetadata.jsonwith channel info, sample rates, time boundaries, and epoch clock typesreadData.jsonwith a deterministic 100-sample chunk for numerical verificationArtifact Verification Tests (
+readArtifacts/+reader/readData.m):matlabArtifactsandpythonArtifactsDocumentation: Added comprehensive
INSTRUCTIONS.mdfiles explaining:Implementation Details
tempdir()/NDR/symmetryTest/<sourceType>/<namespace>/<class>/<test>/following a consistent conventionjsonencode/jsondecodefor language-agnostic data exchangeexample_data/example.rhdin the NDR-matlab repository for deterministic inputshttps://claude.ai/code/session_01BWkrpUgzWSQx48s8Tk6fnT