Skip to content

CI smoke against latest omp: assert effective config, not exit codes #3

Description

@atyrode

omp releases near-daily and v17.0.0 removed settings keys in a breaking change. Two coupling surfaces matter:

  1. The generated overlay YAML (modelRoles, retry.fallbackChains, defaultThinkingLevel, advisor.enabled, tier, task.agentModelOverrides): omp silently ignores unknown keys, so a removed/renamed key stops applying without an error — an exit-code smoke test would lie.
  2. omp models --json and omp usage --json output schemas (generator + usage panel): no upstream stability guarantee; already churned once.

Add a scheduled (weekly) GitHub Actions job that downloads the latest omp release binary, then:

  • launches with a generated overlay and asserts the effective config (omp config get/list --json) matches what the overlay set;
  • parses real omp models --json / omp usage --json outputs with the tool's decoders.

On failure, open/label an issue automatically. Also record in the README which omp version the current release was tested against.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions