Skip to content

fix: simplify lever prompt to restore content quality#297

Merged
neoneye merged 1 commit intomainfrom
fix/simplify-lever-prompt
Mar 16, 2026
Merged

fix: simplify lever prompt to restore content quality#297
neoneye merged 1 commit intomainfrom
fix/simplify-lever-prompt

Conversation

@neoneye
Copy link
Member

@neoneye neoneye commented Mar 16, 2026

Summary

  • Remove mandatory Immediate → Systemic → Strategic consequence chain format
  • Remove mandatory quantification (% change, cost delta) — models were fabricating numbers
  • Remove conservative → moderate → radical option progression template
  • Remove "Radical option must include emerging tech/business model"
  • Add explicit prohibitions: no fabricated statistics, no marketing language
  • Shorten target consequence length from 3-5 to 2-4 sentences
  • Simplify Pydantic field descriptions to match

Motivation

External review of full PlanExe reports (hong_kong_game) rated the baseline report 6.5/10 and the report built on optimized levers 5.8/10: "Version 2 improved specificity, but regressed in credibility."

The optimization loop improved structural compliance (97.1% success rate) but degraded content quality. Baseline consequences averaged 269 chars; iteration 17 averaged 980 chars (3.6x) with fabricated percentages and marketing-copy tone.

Test plan

  • 225 existing tests pass
  • Run experiments across 7 models × 5 plans
  • Compare field lengths against baseline (target: within 2x of baseline)
  • Check for fabricated percentage claims (target: fewer than before)
  • Run analysis pipeline with updated content quality metrics

External review rated baseline report 6.5/10 vs optimized 5.8/10:
"Improved specificity, regressed in credibility." The system prompt
was forcing verbose, overconfident output with fabricated numbers.

Changes:
- Remove mandatory Immediate→Systemic→Strategic chain format
- Remove mandatory % change / cost delta quantification
- Remove conservative→moderate→radical progression template
- Remove "Radical option must include emerging tech/business model"
- Add anti-fabrication: "Do not fabricate percentages or cost estimates"
- Add anti-marketing: "NO marketing language"
- Shorten target consequence length from 3-5 to 2-4 sentences
- Simplify Pydantic field descriptions to match
@neoneye neoneye merged commit dd31c1f into main Mar 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant