Task Description
oo learn frequently generates invalid TOML. Research shows the most reliable cross-provider fix is a validation + retry loop: if the LLM returns invalid TOML, send the parse error back to the LLM and ask it to fix only the broken part.
Root cause: LLMs occasionally generate almost-valid TOML but fail at specific lines (trailing commentary, malformed sections). This is a known problem with all LLM-based structured output generation.
Research findings (2026-03-05):
- Anthropic structured outputs (JSON schema constraint) is 100% reliable but JSON-only — requires JSON→TOML conversion
- Retry with error feedback works across ALL providers (Anthropic, OpenAI, Cerebras)
- Temperature 0 reduces variability
- Stop sequences prevent trailing commentary
Proposed Solution
In src/learn.rs, in run_learn(), after calling the LLM:
- Validate the TOML response
- If invalid, send a follow-up message: "Your previous TOML was invalid: {error}. Here is what you returned: {toml}. Output ONLY the corrected TOML, nothing else."
- Retry up to 2 times
- If all retries fail, write to learn-status.log and return Err
Also add temperature: 0 to API calls for consistency.
Quality Gates (Non-Negotiable)
Acceptance Criteria
Task Description
oo learnfrequently generates invalid TOML. Research shows the most reliable cross-provider fix is a validation + retry loop: if the LLM returns invalid TOML, send the parse error back to the LLM and ask it to fix only the broken part.Root cause: LLMs occasionally generate almost-valid TOML but fail at specific lines (trailing commentary, malformed sections). This is a known problem with all LLM-based structured output generation.
Research findings (2026-03-05):
Proposed Solution
In
src/learn.rs, inrun_learn(), after calling the LLM:Also add
temperature: 0to API calls for consistency.Quality Gates (Non-Negotiable)
oo learn cargo testproduces a valid pattern fileAcceptance Criteria
run_learnretries up to 2 times on invalid TOML with error feedbackoo learn cargo testreliably produces a validcargo-test.toml