Add cross-language symmetry tests for MATLAB ↔ Python DID databases#19
Merged
stevevanhooser merged 5 commits intomainfrom Mar 16, 2026
Merged
Add cross-language symmetry tests for MATLAB ↔ Python DID databases#19stevevanhooser merged 5 commits intomainfrom
stevevanhooser merged 5 commits intomainfrom
Conversation
Add makeArtifact and readArtifact symmetry tests (mirroring DID-matlab's tests/+did/+symmetry/) that verify DID databases created in Python can be read by MATLAB and vice versa. Includes: - tests/symmetry/make_artifacts/ - generates database + JSON summary artifacts - tests/symmetry/read_artifacts/ - validates artifacts from either language - src/did/util/ - database_summary and compare_database_summary utilities - .github/workflows/symmetry.yml - 3-stage CI: MATLAB make -> Python make+read -> MATLAB read - Updated README with instructions for running test groups separately - Added pytest markers (symmetry, make_artifacts, read_artifacts) https://claude.ai/code/session_01JeFfZ9DvpQws6LUCk5a8E6
The MATLAB jobs need MatBox (ehennestad/matbox-actions/install-matbox@v1) to install project requirements (mksqlite, vhlab-toolbox-matlab) before running tests. Also adds Statistics_and_Machine_Learning_Toolbox product, enables MATLAB caching, and uses the proper TestSuite/TestRunner pattern matching DID-matlab's own test-symmetry.yml workflow. https://claude.ai/code/session_01JeFfZ9DvpQws6LUCk5a8E6
- Fix black formatting and ruff lint errors (unused imports, re-export aliases) - Combine all three CI stages (MATLAB make, Python make+read, MATLAB read) into a single job so artifacts in the runner's temp directory are shared across all steps https://claude.ai/code/session_01JeFfZ9DvpQws6LUCk5a8E6
- Document.__init__ now properly handles dot-notation kwargs (e.g., 'demoA.value=42') by navigating/creating nested dicts, matching MATLAB's document_properties structure where demoA.value is a nested field - Fix pre-existing bug in field_search OR operation that required params to be dicts (they're lists) - Update compare_database_summary to compare .value subfield, matching MATLAB's compareDatabaseSummary behavior https://claude.ai/code/session_01JeFfZ9DvpQws6LUCk5a8E6
- Transform flat classname/superclasses from schema files into nested document_class structure (class_name, superclasses, etc.) matching MATLAB's document definition format - Fix _normalize_loaded_props to re-wrap string superclasses back into lists after SQLite round-trip (was only handling dict→list case) https://claude.ai/code/session_01JeFfZ9DvpQws6LUCk5a8E6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a comprehensive cross-language symmetry testing framework that verifies DID databases created in Python can be correctly read by MATLAB and vice versa. The implementation includes CI/CD automation, utility functions for database comparison, and parameterized test suites.
Key Changes
GitHub Actions Workflow (
.github/workflows/symmetry.yml): Three-stage CI pipeline that:makeArtifacttests to generate database artifactsmakeArtifactandreadArtifacttests (reading MATLAB artifacts)readArtifacttests (reading Python artifacts)Database Utility Functions:
src/did/util/database_summary.py: Generates a comprehensive summary dict of a DID database including branch hierarchy, document counts, and propertiessrc/did/util/compare_database_summary.py: Compares two database summaries and returns detailed discrepancy messagessrc/did/util/__init__.py: Exports the new utility functionsSymmetry Test Suite (
tests/symmetry/):make_artifacts/database/test_build_database.py: Creates a multi-branch DID database with documents, exports it as JSON artifacts and a SQLite database fileread_artifacts/database/test_build_database.py: Reads artifacts from either Python or MATLAB sources, validates them against a live database summaryconftest.py: Shared fixtures and pytest markers for symmetry testsTest Configuration:
pyproject.toml: Added pytest markers (symmetry,make_artifacts,read_artifacts) and test path configurationREADME.md: Updated with instructions for running symmetry tests and explanation of the cross-language testing approachImplementation Details
<tempdir>/DID/symmetryTest/{pythonArtifacts,matlabArtifacts}/) for artifact exchange between languagesreadArtifacttests are parameterized to run against both Python and MATLAB artifact sources, gracefully skipping when artifacts are unavailablehttps://claude.ai/code/session_01JeFfZ9DvpQws6LUCk5a8E6