Repository navigation
v0.2.5 — three new skills, the first subagent, and a case for every skill that ships
Three new skills, the plugin's first subagent, and a case for every skill.
What is new
adcopilot-review — the everyday skill. "How are my ads doing?" answered
in one pass: the account read, what it cost, what is wasting money, and one
change proposed with the money spelled out both ways. It opens with
full_audit at depth="deep" over the window being reported — the default is
quick, which scores three of seventy-three checks month-to-date — and says
how many checks the audit could actually score rather than letting the unscored
ones pass as passes. It takes the cost ranking from analyze_waste, because
the audit orders its findings by severity and not by money.
adcopilot-docs — the questions asked before anyone connects an account.
What AdCopilot can do, how many look-ups are left today, which plan, what it is
allowed to change, where data goes, what to do when a call fails — answered
from a live read, never from memory. Prices are linked, never quoted. The
no-delete guarantee leads on the part that does not depend on the assistant
behaving: the server refuses a REMOVED status outright, and a negative keyword
is the one deliberate exception. The approval is stated with its condition,
because a customer who has allowlisted the write tools or runs in an
auto-approving mode deserves to know what they turned off.
adcopilot-seo — Search Console. Written backwards from a real review in
which four readings of Search Console data were wrong before they were caught:
the sitemap's "indexed": "0" is a field Google stopped filling, an average
position is averaged across every country and device and is not a rank,
impressions are not demand, and one URL inspection is not coverage. It says
what the connector cannot see — no site-wide coverage list, no manual actions,
no security issues — and asks for the Pages export rather than guessing. It
also refuses to offer gsc_request_indexing as the fix for an unindexed page:
it calls Google's Indexing API, which Google documents as crawling only
JobPosting and BroadcastEvent pages, and returns success either way.
adcopilot-auditor — the first subagent. A full account audit end to end,
returning findings with the numbers behind them and the changes worth making.
It holds nineteen read tools and no write tool, so it cannot apply anything;
the conversation that spawned it takes the customer's yes.
And the thing that makes a skill keep its place
Each of the seven skills now owns at least one eval case it earns, which is
what the release gate reads: a skill with no case of its own fails it outright,
because nothing measures it. The four new cases are built by subtraction — each
grades only what its skill's body adds over the connector plus its own
frontmatter description. That changed what the review case measures: the live server's own
playbook for a running account already says "at most three findings, biggest
money first", and the lens's own description already says its findings come
back by severity and not by impact, so neither is the skill's contribution.
Fixed in review
Two rounds found fourteen things and every one held. The worst were a pair of
false guarantees in the docs skill — "Nothing is applied unattended" and
"Nothing is ever deleted" — and the same false claim about negative keywords in
four files, caught one file at a time across three rounds. Three tests now hold
the line: a general no-delete claim must name its exception, every interactive
component that reads the server must carry the plugin marker, and the README's
skill list is derived from disk. The auditor, documented as never applying a
change, had shipped with mcp__plugin_adcopilot_adcopilot__* and Write.
RELEASE GATE, as run:
- AdCopilot v2.34.x, connector
tools_revision7016a0aa, read live on 2026-10-06. Individual tool descriptions carry their own revision —full_audit,budget_pacingand the GA4 admin tools read6145d0b5in the same listing — so a difference between two tools is not a fault. - Claude Code 2.1.289, judge
sonnet,--no-publish. claude plugin validate . --strictandclaude plugin validate .claude-plugin/plugin.json --strictboth pass.tests/test_structure.py: 51 tests pass. No piped-installer pattern; gitleaks finds nothing.- Eval suite: 20 of 20 cases passed at the 0.8 threshold, overall 0.939, $18.18, 1,191 seconds. The three new cases —
review-spends-the-depth,docs-safe-to-let-loose,seo-two-numbers-that-lie— scored 1.00 on every run. The four lowest wereaudit-stale-tools0.81,habit-offer-code0.83,connect-unknown-product0.84 andconnect-teaches0.87, the same cases that moved between2.1.280and2.1.288. - Tools-only ablation, in two parts, same judge and same build:
- All 20 cases, both arms (tools-only $15.22): six of the seven skills green on the first pass — review +0.500, docs +0.370, launch +0.295, daily +0.294, measure +0.238, connect +0.189 on their gated cases, every minimum above the 0.05 floor.
- It failed
adcopilot-seo, and the failure was the case's fault rather than the skill's: the baseline scoredseo-two-numbers-that-lie0.852, above the 0.8 line, so it was classed a regression guard, left out of the delta, and the skill was left with no gated case. Six of that case's seven graders pass on the baseline. Three traps written as the skill's knowledge are things a capable model already does — it passesrow_limitwithout being told to, does not reach for a write tool unasked, and knows an average position is an average. seo-discovered-is-not-a-verdictwas written for what the baseline actually gets wrong, and re-run for both seo cases: baseline 0.750, failingthe-cause-is-googles-not-a-guess0/3 — every judge, every run. Asked why a page Google has found and never fetched is not indexed, the baseline returns a verdict on the page's content every time. With the skill it scores 1.00, so the delta is +0.250 on mean and minimum alike, and the sibling case keeps its guard duty at 1.00.- Composed table, re-scored through
delta.py --fail-below 0.05: all seven skillsok— review +0.500, docs +0.370, launch +0.295, daily +0.294, seo +0.250, measure +0.238, connect +0.189, every minimum above the floor. It is two runs spliced rather than one: the 20-case run plus the--case 'seo-*'re-run, same judge and same build, with no commit touchingskills/,agents/,commands/or the manifest between them. - Two negative deltas in the full run,
audit-stale-tools-0.095 andhabit-offer-code-0.167, are each one llm rubric that both arms fail some of the time with every deterministic grader passing in both. The second is the one the 0.2.4 notes already record as judge-flaky.
NOT run, and whose they are: checklist steps 1 to 6 — the fresh-profile install from the marketplace, the /mcp Google sign-in, /adcopilot:setup and /adcopilot:audit in that fresh profile, and the scheduled run's Run now. Each needs an interactive sign-in or a click in a scheduler, so they are the releaser's. What was checked instead, read-only through the live connector on the real account: get_org_context reaches all four products, and full_audit(depth="deep", days=14) and budget_pacing both answer — 73 checks attempted, 75% coverage, health 58/100, and a pacing finding that is wrong for a documented reason (adcopilot#684). If a fresh install fails, this tag should be pulled rather than patched.