Add cross-language symmetry test framework for NDI-python ↔ NDI-matlab#37
Merged
Merged
Conversation
Introduce tests/symmetry/ with make_artifacts and read_artifacts suites that mirror the MATLAB tests/+ndi/+symmetry/ structure. The makeArtifacts tests build NDI sessions and export documents, probes, and the session database to a shared temp directory; the readArtifacts tests verify those artifacts from either language. Symmetry tests are excluded from the default pytest run (--ignore=tests/symmetry) since they require prior MATLAB execution to be fully exercised. https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
DirSession.__init__ had two bugs preventing it from reading existing session data (e.g. MATLAB artifacts): 1. Used self.database_search() which appends a session_id filter, creating a circular dependency — can't find the session_id without searching, can't search without the session_id. Fixed by using self._database.search() directly to bypass the filter. 2. Used hasattr/getattr on document_properties (a dict) as if it were an object with attributes. Fixed to use dict access (in/get). https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
DID-matlab and NDI-matlab have always used branch "a" as the default DID database branch. NDI-python was using "main", which meant it could never see documents stored by MATLAB — the SQL query filters by branch_id, so documents under branch "a" were invisible. https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
Separates test_build_session_artifacts into two independent tests: - test_build_session_documents: verifies JSON document round-trip (runs first) - test_build_session_probes: verifies probe reconstruction (depends on more) This makes it easier to isolate failures since getprobes depends on many other things working correctly. https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
…ction
Three fixes for cross-language parity:
1. Ido: Generate 16_16 hex UIDs matching MATLAB DID format.
Python was generating 13_12 hex UIDs which MATLAB's DID database
rejects with DID:Database:ValidationFieldUID.
2. Element._load_from_document: Use dict access instead of attribute
access on document_properties. The old code used props.element.name
which silently failed (dicts don't have attribute access), leaving
all loaded elements with blank name/reference/type.
3. Session._document_to_object: Check element.ndi_element_class for
"probe" to identify probe documents. Probes use document_class
"element", so doc_isa("probe") never matched. MATLAB sets
ndi_element_class to values like "ndi.probe.timeseries.mfdaq".
https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
MATLAB's element.newdocument() sets element.ndi_element_class to class(ndi_element_obj), which returns the MATLAB class name (e.g. "ndi.element", "ndi.probe.timeseries.mfdaq"). Python was leaving this as the schema default "ndi_element". Changes: - Element: Add ndi_element_class() returning "ndi.element", set it in newdocument() - Probe: Override ndi_element_class() to use probetype2object.json mapping (e.g. "n-trode" -> "ndi.probe.timeseries.mfdaq"), remove redundant newdocument() override (inherit from Element like MATLAB) - Element.searchquery(): Match MATLAB's query which includes name, type, ndi_element_class, and reference (was only base.id) https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
…bject MATLAB's class(self) returns the actual subclass name, and ndi_document2ndi_object uses eval() to dispatch to the right constructor. Python now mirrors this exactly: - Each probe subclass overrides ndi_element_class() to return its MATLAB class name (ndi.probe, ndi.probe.timeseries, ndi.probe.timeseries.mfdaq, ndi.probe.timeseries.stimulator) - _document_to_object uses a class registry mapping MATLAB class names to Python classes, matching MATLAB's eval([class '(S,D)']) - Removed probetype2object.json lookup from Probe.ndi_element_class since the class hierarchy handles it directly, same as MATLAB https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
Port MATLAB ndi.util.sessionSummary and ndi.util.compareSessionSummary to Python. Update make_artifacts and read_artifacts symmetry tests to write and compare sessionSummary.json instead of probes.json, matching the updated MATLAB branch add-symmetry-make-artifacts-test-15791077857957030388. https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
…tem loading Two fixes for MATLAB/Python symmetry testing: 1. Populate doc_data/fields tables in SQLite when adding documents. DID-python's _do_add_doc skips field indexing, but DID-matlab's search relies on the doc_data table. Without these entries, MATLAB cannot find Python-generated documents via database_search. 2. Fix DAQSystem._load_from_document to use dict access instead of broken attribute access (doc_props.base.name → doc_props["base"]["name"]), add MATLAB class names to the reader lookup map, and gracefully handle missing reader/navigator dependencies instead of raising. https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
…ehavior Documents the architectural gaps between DID-python and DID-matlab's sqlitedb implementation: doc_data population, SQL-based search, query resolution, and field naming conventions. https://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a comprehensive cross-language symmetry test framework that verifies interoperability between NDI-python and NDI-matlab implementations. The framework enables both language stacks to generate and validate artifacts, ensuring that sessions, documents, and probes created by one language can be correctly read and interpreted by the other.
Key Changes
Framework Architecture: Implemented a two-phase artifact system:
makeArtifacts: Generates NDI sessions, documents, and probe metadata to a shared temporary directoryreadArtifacts: Loads and validates artifacts from either language implementationArtifact Storage: Centralized artifact location at
<tempdir>/NDI/symmetryTest/{pythonArtifacts,matlabArtifacts}/<namespace>/<className>/<testName>/with standardized directory structure including:.ndi/folder)jsonDocuments/directory)probes.json)Initial Test Suite: Added first symmetry test for basic session building:
tests/symmetry/make_artifacts/session/test_build_session.py: Creates a session with subject and measurement documentstests/symmetry/read_artifacts/session/test_build_session.py: Validates artifacts from both language sources with parameterized testingConfiguration & Documentation:
tests/symmetry/conftest.pywith shared fixtures and artifact path constantspyproject.tomlto exclude symmetry tests from default pytest runs (via--ignore=tests/symmetry)docs/developer_notes/symmetry_tests.mdINSTRUCTIONS.mdfiles for bothmake_artifacts/andread_artifacts/phasesREADME.mdwith symmetry test execution instructionsNotable Implementation Details
readArtifactstests skip (rather than fail) when artifacts from the other language are unavailable, allowing the suite to run on single-language environmentsreadArtifactstests use pytest fixtures to validate bothmatlabArtifactsandpythonArtifactsin a single test classhttps://claude.ai/code/session_01RBSaNF8SSrGqnpnP5vUT47