Prompt Optimizer 1.2.0
This release is about knowing whether the prompts we generate are actually
any good — and about the prompts themselves getting measurably better.
Added — conformance checking
Every generated variant now carries a badge showing which of its format's own
rules it satisfies, with the evidence for each violation.
The rules are not generic advice: they are the ones each format declares in the
app's own system prompts. A Claude Code prompt is checked for backticked paths,
a plan-and-stop section, a separate git branch; a GEMINI.md for Markdown-only
structure and concrete, runnable commands; a System+User pair for a non-empty
User field.
Only rules a parser can decide are checked. Anything needing judgement is left
unchecked rather than guessed — an approximate check produces false positives,
which is exactly what made the previous linter untrustworthy.
This replaces a linter that applied 3 generic rules to all five formats while
the formats declare around forty, and that contradicted them in two places.
Added — a regression harness for the prompts
npm run eval runs a fixed corpus of 66 cases through the real system
prompts and reports a per-rule conformance rate. Pinned model, three repetitions
per case, evidence for every failure. It is a development tool: it never runs in
the shipped app and never spends the end user's quota.
Until now the prompts were the one part of the project we changed without any
regression test — we edited them, they read better, we shipped them.
Changed — the system prompts, rewritten and measured
Not by taste: by measurement, before and after.
| Rule | Before | After |
|---|---|---|
| Chat: XML tags properly closed | 76% | 100% |
Claude Code: reads CLAUDE.md |
93% | 100% |
| Gemini: Markdown structure kept | 92% | 100% |
Two of those failures existed because the prompt had never actually said what
it wanted — "close the tags" and "no XML tags" were both left implicit, and
emphatic capitals had been quietly compensating for the omission.
Changed — model selection
The default is now the current Flash-Lite alias: cheapest and fastest tier,
and a moving alias, so the app follows Google's current model without an update.
Sampling parameters are left at the model default, as Google recommends for the
Gemini 3.x family.
Fixed
- Few-shot examples are sent once instead of being repeated for every
selected format — up to five copies of the same text in a single request. - Two teaching cards in the app were out of date. One of them recommended
prefilling, a technique Anthropic has since removed: following it today
returns a 400 error, on the very models this app uses. - Scaffold generation no longer bakes Windows line endings into its output.
Full details in the CHANGELOG.
The measurements behind every claim above are in
docs/eval-baseline.md,
including the experiments that failed.
Download
- Installer (recommended):
Prompt.optimizer_1.2.0_x64-setup.exe— installs
into your user profile, no admin rights required. - Portable:
pop_app.exe— a single executable (requires WebView2,
usually already present on Windows 10/11).
⚠️ The app is unsigned: on first launch Windows SmartScreen may warn you →
"More info" → "Run anyway".