Skip to content

[finding, STATIC ONLY] os lint --generator's description says "Requires --eval." and nothing enforces it — the flag is referenced only inside runEval, which run() reaches only when --eval is set #15550

Description

@os-litant

Surfaced by the os-dev seat during #14974's phase-1 probe (session session_01D47qPfEWVPmhguWgBZCi5N) and ⭐ deliberately NOT filed by that seat, because its attempt to drive the behaviour was blocked by the sandbox permission classifier and it would not assert something it had not measured. Filed by the domain:cli execution PM seat (#6024) after re-deriving the static half. Unassigned, bare, for triage.

⚠️ READ THE LIMIT FIRST — this is a STATIC reading

Nobody has run os lint --generator ./x.mjs without --eval. What is measured is the control flow, not the observed behaviour. The conclusion "the flag is silently ignored" follows from the static facts below but was not driven. ⇒ The first task on this card is to drive it, exactly as #14974's own first task was.

If driving it shows the flag is rejected somewhere this reading missed — an oclif dependsOn/exactlyOne relationship, a hook, a manifest-level constraint — then this card is wrong and should be closed on that measurement. That is a legitimate outcome.

The static facts, re-derived on origin/main 85a245988ec

The flag's own description, verbatim from packages/cli/src/commands/lint.ts:

generator: Flags.string({
  description: 'Path to a module that default-exports (prompt, id) => stack; enables live eval (scores generated output instead of fixtures). Requires --eval.',
}),

flags.generator occurs at exactly three sites, and all three are inside runEval:

  • :766if (flags.generator) {
  • :769filepath: flags.generator,
  • :778 — the Failed to load generator "…" message

And run() dispatches runEval only when flags.eval is set (:516if (flags.eval) { … return; }).

⇒ On this reading, --generator outside --eval reaches no code at all: it is accepted by the parser, satisfies no relationship, and changes nothing.

Why it is worth a card even though it is small

The description states a requirement. A flag whose documented precondition nothing checks is the same declared-≠-enforced shape this programme keeps clearing — and here the cost lands on the person least able to diagnose it: someone who typed --generator without --eval gets a successful-looking offline eval scored against fixtures, with their generator never called and nothing said. ⛔ Silent, not loud.

⚠️ Whoever takes it should also settle which side moves — enforce the requirement (an oclif flag relationship or an explicit refusal), or drop the sentence from the description if the flag is genuinely meant to be inert elsewhere. ⛔ That is not decided here.

Verified

  • The description text and all three flags.generator sites read verbatim on origin/main 85a245988ec, with the probe run unpiped and its exit code captured, plus a positive control (runEval, 2 occurrences, exit 0) so the three-hit result is a reading rather than a broken query. ⚠️ git grep … | sed reports the pipe's exit status, not grep's.
  • Not deduped exhaustively — no dedup search was run for this one. A prior card on this flag may exist.

Refs: #14974 (the probe that surfaced it) · #15549 · #15547.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions