Skip to content

fix(nasim): reconcile SDL evidence requirements, simulator capture, and backend manifest #87

Description

@Brad-Edwards

Objective

Reconcile the authored NASim SDL/task evidence requirements, the data the pinned simulator can actually expose, the data needed for equivalence testing, and the NASim adapter's implemented capture and backend manifest.

Scope

  • Inventory every data-capture requirement in the NASim SDL and task.
  • Inventory the corresponding native NASim outputs and state whether each required datum is available, unavailable, lossy, redacted, or withheld.
  • Identify all action, outcome, observation, availability, termination, stochastic-control, evaluator, and lineage data needed to reproduce and evaluate equivalence.
  • Correct authored requirements that overclaim simulator capability and explicitly disclose any unavoidable loss.
  • Extend the adapter to capture every required datum that the simulator exposes without leaking native or hidden state into portable artifacts.
  • Make the backend manifest truthful at field/artifact granularity.
  • Remove any false or hardcoded satisfaction claim in the NASim path.

Acceptance criteria

  • Every SDL/task capture requirement maps to a native source, an implemented adapter capture path, a manifest declaration, and an emitted portable artifact field.
  • Every unavailable or lossy requirement is explicit and cannot be admitted as satisfied.
  • The captured evidence is sufficient to recompute the declared equivalence measures and trace each episode to its actions and outcome.
  • Tests exercise the complete requirement-to-capability-to-artifact chain and fail on missing or falsely claimed data.
  • The canonical verification graph passes.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedBlocked on an upstream gate or dependency

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions