Skip to content

Make public CLI and example configs truthful #30

@bordeauxred

Description

@bordeauxred

Why

Some public commands/configs imply support that is not currently wired, which can confuse first-time users exploring the examples.

What

  • Decide whether clawloop run should delegate to the unified TrainConfig runner or remain a legacy benchmark-only command.
  • If clawloop run remains documented, make sure documented benches work or remove them from public docs.
  • Fix or move examples/configs/taubench_harness.json if it is not intended to be a valid TrainConfig.
  • If TauBench is advertised as supported through examples/train_runner.py, register it in the train environment registry and add validation coverage.
  • Add tests for documented CLI/config paths.

Acceptance

  • No documented public command fails with Unknown benchmark.
  • Public example configs either validate or are clearly marked as experimental/non-TrainConfig.
  • README/examples distinguish stable demos from heavier experimental adapters.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai_tinker_launchAI Tinkerers newsletter launch readinessenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions