Skip to content

engine: MDL importer subscript/dimension classification is non-deterministic across process hash seeds #859

Description

@bpowers

Problem

The MDL importer/converter (src/simlin-engine/src/mdl/convert/ + src/simlin-engine/src/mdl/xmile_compat.rs) is non-deterministic across processes: parsing the same MDL fixture with mdl::parse_mdl in two separate processes (which get different HashMap random seeds) can produce a semantically different datamodel. Within a single process the parse is deterministic, so this is HashMap-iteration-order dependence in the dimension/subrange classification path.

Two observed manifestations:

  1. Bang-subscript wildcard classification varies. A Vensim bang subscript x[Dim!] sometimes lands in the datamodel equation as a plain wildcard (x[*], which the writer's Expr0 parser reads as IndexExpr0::Wildcard) and sometimes as a subrange wildcard (x[Dim.*] / *:Dim, read as IndexExpr0::StarRange) -- depending on the process's HashMap seed.

  2. Which dimension a wildcard binds to varies. For test/test-models/tests/subscript_transposition/test_subscript_transposition.mdl, the recovered bang was io matrix[sector1, sector!] on some seeds and io matrix[sector1, sector1!] on others.

The likely root cause is an unordered container iterated during classification -- e.g. the subrange_dims set construction in dimension building, or converter symbol iteration -- where iteration order determines the classification outcome. (Needs confirmation via instrumentation.)

Why it matters

Two runs of the importer on the same file can yield semantically different arrayed equations. This is a latent correctness problem in the importer, independent of the writer. It also contributes to the residual non-idempotence recorded for several arrayed/subscript fixtures in EXPECTED_NON_IDEMPOTENT (src/simlin-engine/tests/mdl_roundtrip.rs).

Currently masked, not fixed

The writer change in #847 renders both Wildcard (via declared-dimension recovery) and StarRange as the same Vensim bang Dim!, so the exported .mdl re-parses regardless of which way the importer classified the subscript. That masks the bug at the corpus re-parse ratchet, but the underlying importer still produces a seed-dependent datamodel.

Component

  • src/simlin-engine/src/mdl/convert/ (dimension/subrange classification, converter symbol iteration)
  • src/simlin-engine/src/mdl/xmile_compat.rs

Possible approach

Make the converter's dimension/subrange classification deterministic: use ordered containers (e.g. BTreeSet/IndexSet) or sort before iterating in the classification path. A test that parses a fixture under multiple forced hash seeds (or a deterministic hasher swap) and asserts identical datamodel output would pin it.

How it was discovered

Identified while hardening the Vensim MDL writer (branch mdl-writer-hardening, PR for #847). This is a pre-existing importer bug, out of scope for the writer-hardening work, so it is filed separately here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions