fix(harbor): reject invalid aggregate_attempts values at construction

[shehabyasser-scale]

Stacked on #21.

Only the exact string mean switches collation to mean-of-k; any other value silently fell through to best-of-k. A config typo (Mean, avg) would therefore run an experiment with inflated pass@k scores while looking de-noised, and nothing would ever error. This is the same silent-config-failure class that invalidated a full TB2 trial in our campaign (bare task names, #16/#17), so it fails loudly now.

HarborConfig.__post_init__ raises ValueError for values outside {best, mean}. The only dynamic construction site is HarborConfig(**config.harbor) at sidecar startup (serve.py), so a bad baked config now kills the sidecar at boot with a clear message instead of after the trial's budget is spent.

Tests: invalid value raises; valid values round-trip.

🤖 Generated with Claude Code

Greptile Summary

This PR adds a __post_init__ guard to HarborConfig that raises ValueError for any aggregate_attempts value outside the allowed set {"best", "mean"}, closing a silent-failure path where a typo would cause an experiment to run best-of-k scoring while appearing de-noised.

• config.py: __post_init__ validates aggregate_attempts at dataclass construction time with a clear error message, so a bad baked config kills the sidecar at boot rather than silently producing inflated scores.
• test_harbor_runner.py: New TestAggregateAttemptsValidation class covers invalid value rejection and round-trip acceptance of both valid values.

Confidence Score: 5/5

Safe to merge — the change is a targeted input guard on a single dataclass field with no side effects on existing valid configurations.

The default value 'best' passes the new guard, so all existing callers are unaffected. The only dynamic construction site (HarborConfig(**config.harbor) in serve.py) will now fail at sidecar boot with a clear message if a baked config contains an invalid value, which is exactly the intended behavior. The validation logic, error message, and tests are all correct and complete.

No files require special attention.

Important Files Changed

Filename	Overview
vero/src/vero/harbor/config.py	Adds post_init to HarborConfig that validates aggregate_attempts is exactly 'best' or 'mean', raising ValueError with a descriptive message otherwise; logic is correct and the default 'best' passes validation.
vero/tests/test_harbor_runner.py	New TestAggregateAttemptsValidation class tests invalid value rejection and valid value round-trips; pytest.raises match pattern correctly anchors to the field name in the error message.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["HarborConfig(**config.harbor)"] --> B["__post_init__"]
    B --> C{aggregate_attempts\nin 'best', 'mean'?}
    C -- Yes --> D["Config object ready\n(sidecar boot continues)"]
    C -- No --> E["ValueError raised\n'aggregate_attempts must be best or mean, got ...'"]
    E --> F["Sidecar exits at boot\n(loud failure, clear message)"]
    D --> G["runner uses aggregate_attempts\nto select scoring mode"]
    G --> H{"== 'mean'?"}
    H -- Yes --> I["De-noised mean-of-k scoring"]
    H -- No --> J["Best-of-k scoring"]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A["HarborConfig(**config.harbor)"] --> B["__post_init__"]
    B --> C{aggregate_attempts\nin 'best', 'mean'?}
    C -- Yes --> D["Config object ready\n(sidecar boot continues)"]
    C -- No --> E["ValueError raised\n'aggregate_attempts must be best or mean, got ...'"]
    E --> F["Sidecar exits at boot\n(loud failure, clear message)"]
    D --> G["runner uses aggregate_attempts\nto select scoring mode"]
    G --> H{"== 'mean'?"}
    H -- Yes --> I["De-noised mean-of-k scoring"]
    H -- No --> J["Best-of-k scoring"]

Loading

_{Reviews (2): Last reviewed commit: "fix(harbor): reject invalid aggregate_at..." | Re-trigger Greptile}