Skip to content

Fail validation when required frameworks or scenarios do not run #48

Description

@stefan-jansen

Findings: BR-014.

Make the full validation command distinguish pass, fail, unsupported, and unavailable outcomes and
return failure whenever a required framework or mandatory scenario did not execute.

Primary surfaces: validation CLI, adapters, result aggregation, exit codes, CI invocation.

Acceptance:

  • The retained unavailable-VectorBT-Pro case returns nonzero and cannot be counted as 16 passes.
  • Adapter import failure, subprocess failure, timeout, skip, malformed output, and missing scenario
    each produce a distinct retained failure record.
  • Optional explicitly unsupported scenarios remain visible and cannot satisfy required counts.

Dependencies: Milestone 1.


Plan reference: /home/stefan/ml4t/libraries/ml4t-backtest-dev/.workspace/work/backtest-stable-release/milestone-2/plan.md - Issue 8 in milestone 0.1.0-m2 - Lifecycle and exact framework parity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions