Skip to content

[golden] Golden fixtures are weaker than their documented pass criteria (GS-4/5/6/7/9/10/11/13) #116

Description

@JArmandoAnaya

What

Several golden fixtures are weaker than the pass criteria docs/roadmap/golden-scenarios.md declares for them, so the acceptance suite does not test what the release doc says it tests:

GS Declared Fixture reality
GS-4 TimeToCollision trigger + min-gap/no-collision checkpoint distance keeping triggered by SimulationTimeCondition value="1"; only two speed checkpoints
GS-5 polyline pedestrian trajectory, TTC brake, ReachPosition resume no Trajectory element; constant-speed pedestrian; ego brakes on SimulationTimeCondition value="4"; no resume
GS-6 TTC trigger, StandStillCondition, non-zero ConditionDelay two SimulationTimeConditions, delay="0" throughout
GS-7 polyline + clothoid + NURBS chained with TimeReference Polyline only, <TimeReference><None/> (clothoid/NURBS live only in determinism_test.cpp)
GS-9 catalog + ParameterAssignments, second overridden run, malformed variant asserting a rule-ID diagnostic expressions only; tests/golden/catalogs/ declared in the layout but absent; harness has no override path
GS-10 scena-run --replay ego=<csv> host round-trip run_scenario() never passes --replay; no replay CSV committed; the Controller source=host property is consumed by no code — checkpoint comes from an engine-side SpeedAction
GS-11 junction map + cross-vehicle-during-orthogonal-phase checkpoint signals only, no map, no cross-vehicle checkpoint
GS-13 "--select brake changes what runs and nothing else" golden.py Scenario has no select field; only the default alternative ever runs in CI

Harness gaps: golden.py has no compare-external subcommand though golden-scenarios.md:39 documents --compare-external <trace> --tol <spec>.

Done when

  • Each fixture upgraded to its declared criteria (or the doc criteria amended where a deliberate simplification is kept — decision recorded per scenario)
  • golden.py Scenario gains replay and select fields; GS-10 exercises --replay, GS-13's --select arm runs in CI with its own reference trace
  • Decide whether an XML Controller property source=host should map to host control ownership; today it is decorative
  • compare-external implemented or the doc downgraded to "maintainer-run, ad hoc"

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions