Skip to content

fix: reject or warn on top-level param-file keys outside defaults:/grid:/params: (#530) - #537

Merged
scttfrdmn merged 1 commit into
mainfrom
fix/530-top-level-param-keys-dropped
Aug 19, 2026
Merged

fix: reject or warn on top-level param-file keys outside defaults:/grid:/params: (#530)#537
scttfrdmn merged 1 commit into
mainfrom
fix/530-top-level-param-keys-dropped

Conversation

@scttfrdmn

Copy link
Copy Markdown
Contributor

Closes #530.

The bug

pkg/params.ParamFileFormat has exactly three fields — defaults, grid,
params — and both parsers (pkg/params/parser.go's JSON and YAML paths)
unmarshal straight into it. Any top-level key that is none of those three is
discarded silently, before validation ever sees it — not passed through as a
PARAM_* env var (which at least leaves a tag/trace), just gone.

Two of spawn's own shipped examples were written wrong this way:

  • examples/simple-params.yaml had region:/instance_type:/ami: at the
    top level.
  • examples/schedule-params.yaml had sweep_name:/region:/
    max_concurrent:/launch_delay:/instance_type:/ami:/disk_size: at
    the top level — and cmd/schedule.go's getSweepName/getMaxConcurrent/
    getLaunchDelay only ever read those from defaults:.

The dangerous case: ttl:, idle_timeout:, cost_limit: at the top level —
an easy mistake, and the same shape one level up from the row-level keys
#526 already fixed — vanish completely, producing an unbounded instance with
no error, no warning, no tag.

The fix

Per the issue's suggested design, a blanket error is not right — most
unknown top-level keys are harmless metadata. So:

  1. Error when the key IS a recognized row/sweep key (ttl,
    idle_timeout, cost_limit, instance_type, ..., plus the reserved
    CLI-only/near-miss list — sweep_name, max_concurrent, launch_delay,
    ttl_hours, budget, etc). This reuses Unknown param-file keys silently become PARAM_* env vars — a mistyped or invented spend control caps nothing #526's recognizedRowKeys /
    reservedRowKeys registry rather than a second, independently-drifting
    list. Message: "ttl" at the top level is ignored; move it under defaults: to apply it to every row.
  2. Warn, not error, for anything else unrecognized — description:,
    version:, and other reasonable metadata. Printed to stderr the same way
    Unknown param-file keys silently become PARAM_* env vars — a mistyped or invented spend control caps nothing #526's passthrough-params list is, so it has somewhere to be noticed.
  3. Fixed examples/simple-params.yaml and examples/schedule-params.yaml in
    this PR, since the new check fires immediately on spawn's own examples
    otherwise.

Scope: pkg/params AND cmd/schedule.go

The issue flagged that spawn schedule uses its own separate
parseParamsFile in cmd/schedule.go, not pkg/params — a fix in
pkg/params alone would not cover it. Both paths turned out tractable to fix
with the same shared classification/error logic
(classifyTopLevelKey/validateTopLevelParamKeys in cmd/sweep_keys.go),
so both are fixed here:

  • pkg/params.ParamFileFormat gets a new UnknownTopLevelKeys []string
    field, populated by a second generic-map decode pass in parseJSON/
    parseYAML (decoding twice rather than DisallowUnknownFields, since
    unknown keys are not automatically an error here).
  • cmd/schedule.go's parseParamsFile/parsedParams gets the same
    unknownTopLevelKeys detection, independently, since it has its own struct
    and its own copy of this bug.
  • cmd/launch_sweep.go (the spawn launch --param-file sweep path) and
    cmd/resume.go (which reloads the file independently) both call
    validateTopLevelParamKeys before validateSweepParamKeys's existing Unknown param-file keys silently become PARAM_* env vars — a mistyped or invented spend control caps nothing #526
    check, so a top-level mistake is reported first — it means a setting the
    user wrote was never even read, the more dangerous of the two failures.
  • cmd/schedule.go's runScheduleCreate calls the same function.

Deliberately NOT unified: the launch path treats defaults: as a per-row
fallback, while the schedule path treats it as sweep-scope (reading
sweep_name/max_concurrent/launch_delay from there) — a real, pre-existing
divergence the issue called out as not necessarily in scope to fix here. Kept
as-is; examples/schedule-params.yaml's comments now document it.

Tests

  • pkg/params/parser_test.go: TestParseYAML_UnknownTopLevelKeys /
    TestParseJSON_UnknownTopLevelKeys (both formats populate the field
    correctly, and the dropped key does not leak into Defaults),
    TestParseYAML_NoUnknownTopLevelKeys (clean file → empty), and
    TestParseYAML_GridIsNotUnknown (grid: itself is not flagged).
  • cmd/sweep_keys_test.go: TestClassifyTopLevelKeyErrorsOnDangerousKeys /
    ...OnReservedKeys / ...WarnsOnHarmlessKeys, and
    TestValidateTopLevelParamKeys{Errors,WarnOnlyPasses,AcceptsCleanFile}.
  • test/e2e/tier0_sweep_top_level_keys_test.go (Substrate, zero spend), 3
    tests: a top-level ttl: 2h rejected with nothing launched
    (requireNothingLaunched, same style as tier0_sweep_param_keys_test.go
    from Unknown param-file keys silently become PARAM_* env vars — a mistyped or invented spend control caps nothing #526), a top-level cost_limit: 5 likewise, and a harmless top-level
    description: that only warns (named in stderr) while the sweep still
    launches and gets tagged normally.

The ttl/cost_limit rejection tests pass a real --ttl 1h on the command
line so the pre-existing --no-detach "every row needs a bound" guard
(#525) would otherwise let the launch through — without it a non-zero exit
would prove only that the row was unbounded, not that the top-level key
itself was caught. All 3 new e2e tests + the 2 new unit test files verified
failing (compile error, pre-fix) and passing (post-fix)
via a stash-based
before/after run. Full test/e2e -tags=e2e_tier0 suite and make check both
green.

@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 69.69697% with 20 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/schedule.go 0.00% 11 Missing ⚠️
cmd/launch_sweep.go 0.00% 3 Missing ⚠️
cmd/resume.go 0.00% 3 Missing ⚠️
cmd/sweep.go 0.00% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

…id:/params: (#530)

pkg/params.ParamFileFormat has exactly three fields, and both json.Unmarshal
and yaml.Unmarshal silently drop any top-level key that is none of them — a
ttl:/idle_timeout:/cost_limit: written one indentation level too high used to
vanish with no trace at all, not even a PARAM_* env var. Two of spawn's own
shipped examples had this bug: examples/simple-params.yaml and
examples/schedule-params.yaml, both fixed here.

A top-level key that is a recognized spawn setting (reusing #526's
recognizedRowKeys/reservedRowKeys registry) is now a hard error before
anything is launched or priced. Anything else unrecognized is a warning
printed to stderr, so harmless metadata (description:, version:) does not
break. Covers spawn launch --param-file (pkg/params + cmd/sweep_keys.go),
spawn resume (which reloads the file independently), and spawn schedule
create (its own separate parseParamsFile in cmd/schedule.go).
@scttfrdmn
scttfrdmn force-pushed the fix/530-top-level-param-keys-dropped branch from 0009820 to d486bfe Compare August 19, 2026 21:55
@scttfrdmn
scttfrdmn merged commit fb6f5dc into main Aug 19, 2026
7 checks passed
@scttfrdmn
scttfrdmn deleted the fix/530-top-level-param-keys-dropped branch August 19, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Top-level param-file keys are dropped in silence — spawn's own examples put settings there

1 participant