Skip to content

pysimlin: JSON schema-compliance tests silently skipped due to wrong path (doc/ vs docs/) #550

@bpowers

Description

@bpowers

Problem

The pysimlin JSON-types schema-compliance test suite is silently skipped because of a wrong path, so the Python JSON dataclasses are not actually validated against the project JSON schema.

src/pysimlin/tests/test_json_types.py:44 resolves the schema with:

SCHEMA_PATH = Path(__file__).parents[3] / "doc" / "simlin-project.schema.json"

parents[3] from src/pysimlin/tests/test_json_types.py is the repo root, so this resolves to <repo-root>/doc/simlin-project.schema.json. That directory does not exist -- the actual schema lives at <repo-root>/docs/simlin-project.schema.json (note docs/, not doc/). Verified: doc/ does not exist; docs/simlin-project.schema.json does.

Because the path does not exist, the if SCHEMA_PATH.exists() guard falls through and sets PROJECT_SCHEMA = None. The entire TestSchemaCompliance class (src/pysimlin/tests/test_json_types.py:348-389) is decorated with @pytest.mark.skipif(PROJECT_SCHEMA is None, reason="Schema file not found"), so all four schema-validation tests (test_stock_validates_against_schema, test_flow_validates_against_schema, test_auxiliary_validates_against_schema, test_module_validates_against_schema) are skipped with no failure.

Why it matters

  • Test gap / false confidence: the suite exists specifically to ensure pysimlin's JSON dataclasses validate against docs/simlin-project.schema.json, but it has never actually run. Drift between the Python dataclasses and the schema (which is shared with the Rust json::* types and the TS Json* types) is currently undetected.
  • Likely latent drift already present: the Python Model dataclass (src/pysimlin/simlin/json_types.py:309) appears to be missing a groups field that the Rust json::Model and TS JsonModel both have (groups does not appear anywhere in src/pysimlin/simlin/json_types.py). Once the tests actually run they may surface this and possibly other mismatches.
  • A skip-on-missing-file guard that silently passes is a footgun: the only signal is a skipped test in pytest output, easy to miss.

Component(s) affected

  • src/pysimlin/tests/test_json_types.py (line 44: the path; lines 348-389: the skipped TestSchemaCompliance class)
  • src/pysimlin/simlin/json_types.py (the dataclasses under test, e.g. Model and its possibly-missing groups field)
  • docs/simlin-project.schema.json (the schema the tests should validate against)

Possible approaches

  1. Change "doc" to "docs" in src/pysimlin/tests/test_json_types.py:44.
  2. Run the now-active TestSchemaCompliance tests and fix whatever mismatches they surface -- in particular, check whether the Python Model dataclass needs a groups field to match Rust json::Model / TS JsonModel.
  3. Consider hardening the guard so a missing schema file fails loudly (or assert SCHEMA_PATH.exists() at module load) rather than silently skipping, so a future path break is caught immediately instead of degrading to a no-op.

Discovery context

Identified during implementation-planning investigation for Vensim macro support; out of scope for that work. Distinct from existing tech-debt.md entry #19 (which is about flaky Hypothesis health-check failures in the same test file, not the wrong-path skip) and from GitHub issues #538/#539 (protobuf patch messages / AiInformation protobuf round-trip).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions