feat(mcp): MCP-first lead delivery — find_new_leads, qualify_leads, lead_job_status + seed-craft prompt - #168
feat(mcp): MCP-first lead delivery — find_new_leads, qualify_leads, lead_job_status + seed-craft prompt#168milstan wants to merge 29 commits into
Conversation
…ualify_leads, lead_job_status
Three composites over the new job endpoints (POST /1.6/mcp/search,
POST /1.6/mcp/qualify, GET /1.6/mcp/jobs/{id}): one ask -> net-new
qualified contactable leads; batch qualify + right-contact on known refs;
shared cumulative job poll with cursor paging. Submit+poll built into the
tools (wait_seconds window, still_running handoff), request_id retry
contract, dry_run cost gate, no-silent-spend consent copy.
The leadbay_new_leads prompt + heuristics/example-lead-craft snippet carry
the fake-leads seed discipline (registry-style buyer description, no
invented brand name, no event language, filters for hard bounds) —
validated live on staging: same ICP delivered 0/5 junk from a naive query
vs 5/5 on-profile from a crafted description-only seed.
Cross-routing anti-triggers on pull_leads / extend_lens /
bulk_qualify_leads; WORKFLOWS.md rows 46-48 + contracts; persona and
naming/deprecation doc in docs/mcp-first-lead-delivery.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Never combine a lowered qualification floor with channels — that buys emails for leads the AI just scored as junk (measured: 633c for a floor-scored consultancy's email on staging). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 46f60b799f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
[Claude]: Session summary for Milan — my chat replies were not rendering in the Conductor thread (confirmed absent, not collapsed), so the state of record lives here:
Only open action requiring Milan: the az command in #3927. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7ac5429c92
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
A single qualify+channels pass buys emails for leads the same job disqualifies (measured: ~a third of enrichment spend on staging). Teach verdicts-first, channels-second as the default; backend gate tracked in product#3938. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 0.26.0 bump merged from main left the @leadbay/mcp@0.25 pin behind; the server-json-version audit rightly failed. Pin follows the version line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ad2b36e90
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Counter-mission: MCP-first delivery exists to eliminate client-side chaining; teaching agents to orchestrate two passes reintroduces it. The spend-gating belongs in the backend (product#3938). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3dbabf3cf4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…exclusion/location/fit guidance
From the live seed-interpretation eval (2/2 cold agents): tolerate the
agent-invented filters.employees:{min,max} by mapping onto the flat wire
keys; name the exact filter keys in the craft rules; never a country in
locations (France silently matched the village of Francs); exclusions are
enforced by qualification via IBP anti-patterns (leadbay_refine_prompt for
durable ones), not by the free match; rendering gains weak-batch honesty
(best fit <30), geo fence checks, and skepticism for exemplar-bridged FR
fit scores.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
github-actions
Bot
added
the
needs-manual-rebase
|
Couldn't auto-sync with main — this PR has a merge conflict outside the version files. main shipped |
4/4 live E2E agents passed a country label despite the guidance; the backend silently fences it to a same-named town (France→Francs, United States→Statesboro — product#3939). Named client-side rejection with the omit-locations hint; zero spend on a doomed job. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arch-qualify # Conflicts: # WORKFLOWS.md # packages/mcp/server.json
|
[Claude]: @ArtyETH06 review requested — context + what to test. Purpose of this PR: expose the backend MCP-first job endpoints ( Please TEST it, don't just read it — Milan's explicit ask. Suggested pass (staging, any test account, e.g. milstan@leadbay.ai / usual staging password):
Known staging caveats (not this PR): country-level geo resolution product#3939, US org IBP/questions mismatch, education segment uncrawled. Full E2E acceptance evidence: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 80c9fd7dce
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…t delivery Review fixes on top of milstan/expose-mcp-search-qualify (PR #168). Not a replacement for that PR — the delivery tools are Milan's; these are the defects found reviewing it. - WORKFLOWS.md: resolve the committed merge-conflict markers. main's friction rows keep 46/47; the delivery rows renumber to 48/49/50. Both contract sets retained. The workflows + contract-schema audits parse this file, so the markers were a hard audit failure. - qualify_leads: enforce the spend gate in code. `qualify` defaults to TRUE backend-side (~94 cost_cents/lead) and the schema had no required params, so `{lead_refs: [...500]}` alone submitted a paid job nobody approved. A paid call now needs `confirm: true`; without it the submit is withheld and a real dry-run quote is returned (`mode: "needs_confirmation"`). `confirm: false` is a veto with no round-trip; fully free calls (qualify:false, no channels) pass straight through. Mirrors the existing enrich-titles consent gate (product#3848). - qualify_leads: also return {leads, skipped} via splitItems. Its own RENDER block reads leads[]/skipped[] and both sibling tools split; returning only items[] left an agent with two empty tables. items[] is preserved for input-order per-ref mapping. - lead_job_status: forward since/limit through the block-waiting path. waitForJob dropped them, silently turning an incremental poll into a full re-read of already-seen items. - waitForJob: bound each sleep by the remaining wait so a short wait_seconds can't block for a whole 4s interval. - Currency: cost_cents renders as $ not € (the rest of the repo, including quota-windows and account_status, denominates the same ledger in dollars). - Export findNewLeads/qualifyLeads/leadJobStatus from the core entry point. - leadbay_new_leads prompt: the paid-quote step listed min_ai_score among the flags then routed to qualify_leads, which has no such property and additionalProperties:false — a literal follow was a hard schema rejection. Flags are now split per tool, and the confirm:true requirement documented. One existing test (qualify-leads.test.ts happy path) gains `confirm: true`: it purchases an email channel, so the gate withholds it. That edit is deliberate and approved — it asserted the behaviour this commit removes. pnpm -r build, -r test (1301 passed), -r typecheck all green. Co-Authored-By: Claude <noreply@anthropic.com>
ArtyETH06
added
feature
and removed
needs-manual-rebase
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b866a4290b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
- collectJobSnapshot: derive the page bound from page SIZE, not a flat 20.
At limit=5 a 500-item job returned the first 100 items while reporting
done:true.
- collectJobSnapshot: keep the last non-empty page's cursor when the drain
page comes back empty; next_since was being overwritten with null, so a
caller lost its place and re-read everything it had already seen.
- rejectCountryLocations: normalize instead of exact-matching. "the United
States", "U.S", "les États-Unis" and "République Française" all passed
through to the silent same-named-town fencing this guard exists to stop.
Île-de-France and other legitimate places still pass.
- find_new_leads / qualify_leads: short-circuit LEADBAY_MOCK submits. The
mock envelope has no job_id, so the offline dry-run mode polled
/mcp/jobs/undefined and died on a missing fixture.
- list_locations is now always exposed, like list_sectors. The delivery
tools document it as the recovery path after a 400 on an unresolvable
location, which only works if it's reachable without ADVANCED=1.
- leadbay_new_leads prompt: handle still_running BEFORE judging the seed
(Phase 3 had no such branch, so a live partial read as "empty" and
triggered a relaunch that burns an active-job slot); require an archetype
component in request_id so two archetype searches don't dedupe to one job;
route unsupported negative constraints ("no franchises") into the seed +
a Phase 5 drop-and-say instead of silently losing them; permit the
fictional seed explicitly against the no-fabrication law; take the need
from the triggering message so the skill never asks the user to repeat
what they just said.
- lead-delivery-table: don't invent the "of the Y asked" denominator when
items_requested is absent from a job snapshot.
Budget: trimmed find-new-leads template body — the additions above pushed
it to 17188 (cap 17000). Now 16948, in line with the other large tools
rather than the outlier.
pnpm -r build, -r test (1327 passed), -r typecheck green.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 515af8bb0b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Second Codex pass, on the review fixes themselves. - find_new_leads had the same paid surface as qualify_leads (qualify:true and/or channels) but no gate — only qualify_leads got one. Same withhold: a paid search now needs confirm:true, otherwise nothing is submitted and a real dry-run quote comes back as mode:"needs_confirmation". The trigger differs by design: qualify defaults to FALSE here, so the default search stays free and ungated. - qualify_leads is destructiveHint:true. It bills for fresh qualification and channel reveals, and hosts key approval prompts off that flag — bulk_qualify_leads already sets it for the same reason. find_new_leads stays false with a comment saying why: its default really is free. - leadbay_new_leads prompt: on a short preview, qualify what was delivered AND search only the SHORTFALL under a NEW request_id. Reusing the preview's id dedupes the paid submit into the free job; keeping the original count buys a second full batch, since novelty:org already excludes the preview deliveries. Budget: the gate copy pushed find_new_leads to 17140 (cap 17000). Trimmed the template body — dropped param prose already carried by inputSchema descriptions and condensed the intro. Now 16961. pnpm -r build, -r test (1332 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
…mpotency fixes Third Codex pass. - Release gate (the P1): the three delivery tools are now hidden unless LEADBAY_MCP_LEAD_DELIVERY=1. /1.6/mcp/* is live on staging only and 404s on production, so shipping them unconditionally would hand every user tools that fail on the first call. Remove the gate in the release that follows the backend deploy. mcpFirstDeliveryAllTools keeps them visible to the WORKFLOWS/routing audits, so a rollout flag never reads as "this tool doesn't exist". - find_new_leads returned summary.requested while the shared renderer reads summary.items_requested — renamed, sourced from submit.items_requested, so the "delivered X of the Y asked" line stops being unrenderable. - Both submit tools now hand snapshot.next_since forward in next_poll, so a follow-up leadbay_lead_job_status continues incrementally instead of re-reading and re-rendering rows already shown. - find_new_leads is destructiveHint:true. Annotations are static and must describe the worst case (qualify:true / channels bill, and deliveries claim org novelty), not the free default. The free path is protected by the confirm gate in execute(), not by the annotation. - qualify_leads derives a stable request_id for a consented paid submit when the caller omits one (refs + paid flags + UTC day, FNV-1a). Without it a timeout or agent retry re-charged qualification and channel purchases for the same refs. The response echoes the key actually sent. One existing assertion relaxed: find-new-leads.test.ts pinned next_poll with toEqual, which the new `since` cursor breaks; now toMatchObject, so it asserts the handoff fields without freezing the shape. pnpm -r build, -r test (1337 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8d3962b35f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Resolves the WORKFLOWS.md row collision: main's account-conquest pair landed first and keeps 48/49, so the lead-delivery rows renumber to 50/51/52. Version files already agreed at 0.28.0. pnpm -r build, -r test (1349 passed), -r typecheck green.
ArtyETH06
removed
the
needs-manual-rebase
…cy key Fourth Codex pass — all three are consequences of the release gate itself. - leadbay_new_leads prompt is now gated with the tools it drives. Every step of that workflow calls find_new_leads / qualify_leads / lead_job_status, so offering the prompt while the tools are hidden started a flow whose every call was missing from tools/list. listAllPrompts() keeps it visible to the WORKFLOWS audit; prompts.test.ts sets the flag since it asserts the full catalogue. - The derived idempotency key now covers the WHOLE prior_deliveries selector, not just the job id. Qualifying the first 50 of a delivery job and then the next 50 are different batches; collapsing them to one key made the second submit look like a duplicate and left those refs unqualified. - lead_job_status carries `since` in its own next_poll handoff, matching the two submit tools — following it without the cursor re-read the rows just returned. pnpm -r build, -r test (1353 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fff45f5ec4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… key Fifth Codex pass — mostly gaps the release gate itself opened. - The Claude SKILL.md was still shipping while the tools are hidden, and it auto-triggers on the same "find me new leads" requests, so Claude Code users entered a workflow that fails on the first tool call. A skill is a static file with no runtime gate, so promptforge now emits none for a prompt marked `release_gated: true`. Delete that one frontmatter line to un-gate. - getPrompt() is gated too, not just listPrompts(): a cached slash command or a direct prompts/get by name bypassed the list filter entirely. - The derived idempotency key no longer includes the UTC date — a retry of the same approved batch after midnight derived a different key and could re-run the spend. It now hashes the approved batch only, and includes max_cost so raising the cap after a stop_reason:max_cost is correctly a NEW run rather than a dedupe onto the capped job. - The page bound now lets EVERY allowed page size drain 500 items; the old 120-page cap meant limit=1 returned 120 rows with done:true and no cursor. - The prompt now says `count` is the TOTAL across archetypes, not per search — "find 10 gyms and warehouses" was delivering (and later paying for) 20. - find_new_leads description trimmed again: 78 chars of headroom, no longer the tightest description in the repo. qualify-leads.test.ts keeps its one-line `confirm: true` (comment dropped, so the edit is now a single line). Removing it entirely is not possible: that test purchases an email channel, which any spend gate must withhold without consent. pnpm -r build, -r test (1362 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9e965a55ce
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…ts.test Sixth Codex pass. - The new-leads prompt is gated on the write surface as well as the rollout flag. Its workflow calls write-tier tools, so a read-only server (LEADBAY_MCP_WRITE=0) offered a slash command whose tools were absent from tools/list even with the flag on. buildServer now threads includeWrite into both listPrompts() and getPrompt(). - job_id is encodeURIComponent'd before building the poll URL. It comes straight from user/agent input and the server does not validate schemas before dispatch, so a value like `../../users/me` normalized out of /mcp/jobs and fired an AUTHENTICATED GET at an unintended endpoint. - prompts.test.ts is byte-identical to main again. The gated-prompt coverage moved wholly into prompt-release-gate.test.ts, which now also covers the read-only case and prompts/get. Both gate suites get an explicit 30s timeout: each case resets the module registry and re-imports a large graph, which overran vitest's 5s default. pnpm -r build, -r test (1364 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 46d97c761e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… top-up
Seventh Codex pass. Three of the five are follow-ons from earlier fixes.
- buildServer normalizes includeWrite with Boolean() for the prompt gate. The
tool list uses `if (opts.includeWrite)`, so an omitted flag means NO write
tools — but the gate read `undefined` as write-enabled and could expose the
slash prompt while its tools were absent.
- The derived idempotency key keeps ref FIELD NAMES, so {website:"acme.com"}
and {name:"acme.com"} no longer hash alike; the backend resolves them
differently and the second approval could be deduped onto the first job.
- `lang` joins the key too: re-running an approved batch in another language
must not return the earlier job with evidence in the previous one.
- Prompt: retries reuse the request_id verbatim. Rederiving it from "today's
date" after midnight produced a new key, so a retry of a live or timed-out
search launched a second paid, novelty-claiming job.
- Prompt: top up after PAID attrition, not just a short preview. A full-count
preview can still end short once qualification disqualifies rows or a strict
title/channel match misses — the ask is n qualified and contactable, not n
examined. Report the gap and offer a shortfall-sized top-up, quoted first.
pnpm -r build, -r test (1366 passed), -r typecheck green.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e138b863d4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…back Eighth Codex pass. The headline is a real hash collision, verified locally. - The derived idempotency key was a 32-bit FNV-1a digest, and two distinct single-ref batches genuinely collide: websites "aeqexh0jh0.com" and "99rcha4ssn.com" both produced qualify-auto-76d7841e. A collision here redirects one paid approval onto another job. Now SHA-256 truncated to 128 bits, via a shared derivedKey() helper. The colliding pair is pinned as a regression test. - find_new_leads synthesizes a key too. `request_id` is schema-`required` but the server does not validate before dispatch, so a confirmed paid search could post with no idempotency handle and a retry would launch a second paid, novelty-claiming job. - collectJobSnapshot falls back to the caller's `since` when a page carries no replacement cursor. An incremental poll of a running job legitimately returns items:[] with no next_since, and nulling the cursor there made the next poll a full re-read of rows already rendered. - skills.test.ts keeps only the minimal filter the old invariant needs; the gated-skill assertions moved to the new skills-release-gate.test.ts. pnpm -r build, -r test (1370 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c169b37bc1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Ninth Codex pass — one finding, correct. `exclude_lead_ids` was sent in the POST body but left out of the derived `search-auto-*` shape, so a top-up that differs ONLY by its exclusion list hashed identically to the original search: the backend returns the first job as a duplicate and the exclusions never take effect. Now canonicalized (sorted, so ordering alone never forks the key) and included. Audited both tools for the same class of gap while here. That was the only omission: qualify_leads already hashes every shaping field, and request_id / dry_run are legitimately excluded (the key itself, and a flag that never submits). Added find-new-leads-idempotency.test.ts with a structural guard — one case per body field asserting that changing it changes the key, so a field added later cannot silently drift out of the hash. Verified the guard actually bites by removing the fix and watching exactly the two expected cases fail. pnpm -r build, -r test (1388 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5eb0df349
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… mock path
Tenth Codex pass — all five findings were real; two verified locally first.
- Delimiter forging in the qualify key (verified): the ref shape joined raw
values with `~`, so a value CONTAINING a delimiter impersonated another
field. {website:"acme~name=Paris"} and {website:"acme", name:"Paris~name="}
serialized identically — two distinct paid approvals sharing one key. Both
shapes are now canonical JSON, in qualify_leads and find_new_leads, so no
free-text value can forge a field boundary.
- Defaults are canonicalized before hashing. An approval that omits `novelty`
and a retry passing the documented default `"org"` derived different keys,
so the retry could not dedupe and launched a second paid, novelty-claiming
job. Same for min_ai_score, title_gate, channels, exclude_lead_ids.
- Scalar filters.locations bypassed the country guard entirely (verified):
`"United States"` as a bare string returned early and reached the backend,
reintroducing the silent same-named-town fencing. Non-arrays are normalized
to a one-item list instead of skipped.
- mockedSubmitPreview only claims "mocked" when LEADBAY_MOCK=1. A real 2xx
submit missing job_id now throws MALFORMED_SUBMIT_RESPONSE rather than
reporting a successful no-submit envelope, which hid a production job the
client could not poll.
- Prompt: qualify EVERY preview job after an archetype split (or merge their
refs), instead of the singular prior_deliveries handoff that stopped after
the first archetype.
Both colliding pairs are pinned as regression tests, plus a case asserting
omitted fields and their explicit defaults derive the SAME key.
pnpm -r build, -r test (1396 passed), -r typecheck green.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cf962c18bd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… refs
Eleventh Codex pass — both findings real, both verified locally.
- Nested property order forked the derived search key: `example_lead` and
`filters` were hashed with plain JSON.stringify, so an agent rebuilding
{description, employees} as {employees, description} derived a different
key and a retry of a live paid search launched a second job. Added a
recursive canonicalize() that sorts object keys at every depth, applied
INSIDE derivedKey so neither tool can regress by hashing a hand-built
string. Sector/location lists are sorted too — they are unordered sets to
the backend.
- Duplicate refs forked the qualify key even though the contract collapses
them into one backend item. A paid submit with repeated websites, retried
after the agent deduped, presented a new key and re-ran the batch. The
canonical ref list is now uniqued before hashing.
pnpm -r build, -r test (1400 passed), -r typecheck green.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7783fcd965
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…resolver Twelfth Codex pass. Two of three counted. - Set-valued lists were sorted but not uniqued, so locations ["Dallas", "Dallas"] and ["Dallas"] request identical work yet derived different keys — a retry that deduped the list re-launched a paid, novelty-claiming job. This was the same defect fixed for lead_refs last round and not generalized; added canonicalSet() (sort + unique) in the shared helper and routed EVERY set field through it: sectors, locations, channels, contact_titles, exclude_lead_ids, refs. - Ref values are now normalized the way the resolver normalizes them, reusing the existing exported normalizeDomain(): a pasted "https://Acme.com/" and a retry's "acme.com" resolve to one company and now to one key. Names and locations are trimmed + lowercased; a non-domain-shaped website falls back to trimmed/lowercased rather than being dropped. pnpm -r build, -r test (1406 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d11f8bb51d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Twelfth-pass P1 on the description budget. The threshold in the finding is still wrong — 17000 is the only enforced cap, there is no ~16k constant, and at 16922 this description was the FOURTH tightest, behind three pre-existing tools this PR never touched. But the fragility point is fair: the cap has broken this build twice, and shaving a few characters each time was not fixing it. Trimmed the shared example-lead-craft snippet instead of the prose that was squeezed last time: rules 7 and 8 duplicated guidance now carried by the prompt and the inputSchema, and rules 1-2 restated their evidence at length. All 8 rules and their measured proof points survive. find_new_leads: 16922 -> 16704 (296 headroom), now clear of prepare_outreach (7), research_lead_by_id (27) and pull_followups (52). The snippet is shared, so qualify_leads and lead_job_status gain the same slack. pnpm -r build, -r test (1406 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d298e8742d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… skills
Thirteenth Codex pass. All three real.
- contact_titles were hashed raw, so a retry rebuilding "Owner" as "owner "
derived a new key and could re-run a paid batch even though the backend
matches titles semantically. Added canonicalLabelSet() (trim + lowercase
before dedupe) and routed contact_titles and filters.sectors/locations
through it.
- Three shapes that all mean "no filters" hashed three different ways:
omitted, {}, and {locations: []}. canonicalOptionalObject() collapses empty
objects and empty set fields to null, so an unfiltered retry dedupes.
- promptforge now DELETES a stale gated SKILL.md instead of leaving it on
disk. writeIfDifferent only writes, so a prompt marked release_gated after
its skill had already been emitted left a static, auto-triggering file to be
committed and shipped — I hit exactly this and removed it by hand last time.
`prompts:check` now fails on an orphaned gated skill so CI catches it.
Verified the full loop: check fails loudly on a planted stale file, build
removes it, check then passes.
pnpm -r build, -r test (1411 passed), -r typecheck green.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2accf1a5ec
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Fourteenth Codex pass. Three of four real.
- normalizeSearchFilters wraps a scalar sectors/locations in a one-item array.
I had made rejectCountryLocations tolerant of a bare string last round but
not the SUBMISSION, so a scalar survived to the wire and 400d on backend
deserialization. Verified: {locations:"Dallas"} now POSTs ["Dallas"].
- lead_id / contact_id UUIDs are lower-cased before hashing. UUIDs are
case-insensitive to the backend, so an uppercase id and its lowercase form
are the same lead and must share an idempotency key. Non-UUID ids keep their
casing, since we cannot assume the backend folds those.
- Prompt: a paid top-up now carries exclude_lead_ids for every lead already
SEEN — delivered, disqualified and skipped, across both passes. novelty:org
excludes prior DELIVERIES only, so an examined-and-rejected candidate stays
eligible and the top-up would re-pick the same misses and charge again to
close no gap.
Declined the fourth (restrict the auto request_id to paid calls): the free
path would then submit with NO key at all, which is worse — request_id is
schema-required, and a timed-out free search retried would launch a fresh
novelty-claiming job rather than dedupe. Reasoning posted on the thread.
pnpm -r build, -r test (1416 passed), -r typecheck green.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e98b791dd4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Fifteenth Codex pass. Two of three real. - canonicalSet / canonicalLabelSet threw a TypeError on a scalar. Since the server does not validate inputSchema before dispatch, `channels: "email"` reached the key derivation and CRASHED the call before the user ever saw a quote. All three set canonicalizers now wrap a bare scalar in a one-item list, matching what normalizeSearchFilters does for sectors/locations. - exclude_lead_ids are UUID-folded via a new canonicalIdSet(), so a top-up that lowercases ids copied from a prior result dedupes instead of launching another paid search. normalizeUuid moved into the shared helper and is now used by both tools rather than duplicated. Declined the paging change (drain whenever next_since is set). Checked it against the fixtures and the tool contract: the backend returns next_since on EVERY snapshot as a resumption handle, including a completed job with a short page — that is the normal terminal shape, not evidence of unfetched items. Following the cursor unconditionally adds a wasted round-trip to every terminal poll. I tried it, it broke 10 existing tests for exactly that reason, and I reverted it. A full page remains the drain signal. pnpm -r build, -r test (1418 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 447c40b718
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Sixteenth Codex pass. All three real, and all three the SAME root cause: the MCP server does not validate inputSchema before dispatch, so any array field can arrive singular. I had been hardening one consumer at a time (key derivation, then filters, then the set canonicalizers) and each round found the next unpatched site. Fixed the class instead. - coerceArrayParams() normalizes the array-typed params ONCE, at the top of execute(), before the spend gate or anything else touches them. A scalar `channels: "email"`, `contact_titles: "Owner"` or a single `lead_refs` object now flows through normally and reaches the wire as an array. Verified: all three previously threw a TypeError BEFORE the gate, so the caller got a crash instead of the promised quote; all three now return needs_confirmation with the correct wire shape. - prior_deliveries.job_id is UUID-folded like the refs, so retrying the same selector with different casing dedupes rather than re-running a paid batch. pnpm -r build, -r test (1422 passed), -r typecheck green. Co-Authored-By: Claude <noreply@anthropic.com>
2 participants
Exposes the backend MCP-first job endpoints (
POST /1.6/mcp/search,POST /1.6/mcp/qualify,GET /1.6/mcp/jobs/{id}) as three composites plus theleadbay_new_leadsguided prompt. One ask → net-new qualified contactableleads, replacing the pull→select→web_fetch→poll→enrich chain.
Not mergeable yet — blocked on the backend release, not on this code. See
"Release blocker" below.
What's in
leadbay_find_new_leads(composite write): one ask → n net-new leads;free by default, paid flags behind a
dry_runquote + consent, submit+pollbuilt in (
wait_seconds),request_idretry contract,still_runninghandoff.
leadbay_qualify_leads(composite write): ≤500 refs (ids / websites /names / contact ids /
prior_deliveries), per-item verdicts, owneddisqualified leads delivered WITH their negative evidence.
leadbay_lead_job_status(composite read): cumulative snapshot, cursorpaging, optional block-wait.
leadbay_new_leadsprompt +heuristics/example-lead-craftsnippet —the backend fake-leads discipline distilled to 8 rules, plus a live-measured
one: an invented
example_lead.namepoisons matching (0/5 junk → 5/5on-profile A/B on staging, same ICP). Filed as leadbay/product#3926.
docs/mcp-first-lead-delivery.md), WORKFLOWS.md rows48–50 + contracts, routing fixtures, cross-routing anti-triggers on
pull_leads/extend_lens/bulk_qualify_leads.Review fixes (
b866a429→fff45f5e)All 34 Codex review threads are resolved (4 automated passes). Beyond the spend gate below: page
bound derived from page size (a small-page drain silently truncated at 100
items while reporting
done), cursor preserved across an empty drain page andhanded forward in
next_poll, country labels normalized (the United States,U.S,les États-Unisall reached the silent same-named-town fencing),LEADBAY_MOCKsubmits short-circuited,list_locationsexposed as thedocumented 400-recovery path, a derived
request_idso a retried paid batchcannot double-charge, and
destructiveHint: trueon both paid submitters.Spend gate now enforced in code.
qualifydefaults totruebackend-side (~94 cost_cents/lead) and the schema had no
requiredarray, so{lead_refs: [...500]}alone submitted a paid job with no quote and norequest_idto dedupe a retry. A paid call now needsconfirm: true;otherwise the submit is withheld and a real backend quote returns as
mode: "needs_confirmation".confirm: falsevetoes with no round-trip;free calls (
qualify: false, no channels) pass through untouched. Mirrorsthe
enrich-titlesconsent gate (product#3848).Driven through the compiled composite, with a client that throws if a paid
POST reaches the wire:
WORKFLOWS.mdmerge conflict resolved. Literal conflict markers werecommitted at lines 60–67 and 1025–1120, and
main's friction rows collidedon numbers 46/47. Both sets retained: friction keeps 46/47, delivery
renumbers to 48/49/50. The workflows + contract-schema audits parse this
file, so this was a hard audit failure.
Prompt↔code drift fixed. The paid-quote step listed
min_ai_scorethenrouted to
leadbay_qualify_leads, which has no such property andadditionalProperties: false— a literal follow was a hard schema rejection.Flags are now split per tool.
qualify_leadsalso returns{leads, skipped}viasplitItems— its RENDERblock reads those keys and both sibling tools split; returning only
items[]left an agent with two empty tables.
items[]kept for per-ref mapping.lead_job_statusforwardssince/limitthrough the block-waiting path(
waitForJobdropped them, silently re-reading already-seen leads).waitForJobbounds each sleep by the remaining wait.cost_centsrenders as$, not€— the rest of the repo denominates thesame ledger in dollars.
findNewLeads/qualifyLeads/leadJobStatusexported from the core entrypoint.
Release gate
POST /1.6/mcp/searchreturns 404 on production (api-usandapi-fr,confirmed with a valid authenticated prod token) and 401 on staging — the
endpoints exist only on staging.
So the three tools are gated off by default: they are only registered when
LEADBAY_MCP_LEAD_DELIVERY=1. Merging this PR therefore ships nothing to usersuntil the backend lands — at which point the gate (and the flag) come out in
the following release. The
leadbay_new_leadsprompt is gated with them, since every step of thatworkflow calls one of the three.
mcpFirstDeliveryAllTools/listAllPromptskeep both visible to the WORKFLOWS + routing audits, so the flag never reads as
"not registered". Pinned by
mcp-first-delivery-gate.test.tsandprompt-release-gate.test.ts.The backend release itself is not a promote: the last tag
v3.21.6(07-29) is26 commits behind
main, and that gap includesdd4c9a10("the verdict gates the till", the spend fix these tools rely on) alongside
billing/VAT, quota, auth and email changes.
Verification
pnpm -r build,pnpm -r test(1353 passed),pnpm -r typecheck— green locally and in CI on
b866a429.Not verified live. The
/1.6/mcp/*endpoints are staging-only and theavailable tokens authenticate against production only, so the spend gate is
proven through the compiled bundle and unit tests, not a real backend job.
Milan's earlier staging E2E (free search 4.6s / 5 delivered / $0; paid chain
delivering purchased emails; FR parity after the queue drain) predates these
fixes.
No
/evalrun: the changed behaviour (a withheld paid submit) has no existingeval contract, and the harness needs an
.env.evalstaging-token decision.Known backend issues (not fixable here)
lastname.emptyterminal-fails a wholequalify job (reproduced twice).
example_lead.namepollutes seed matching.bulk_qualify_leadsdeprecation is a documented direction only(
docs/mcp-first-lead-delivery.md) — no removal in this diff.Note on test hygiene
One existing test gains a single
confirm: true(
packages/core/test/unit/composite/qualify-leads.test.ts): it purchases anemail channel with no consent, so the new gate correctly withholds it — it was
asserting the behaviour the gate removes. Flagged because the repo rule is
otherwise "never modify existing test files"; all other new coverage is in two
NEW files (
qualify-leads-spend-gate.test.ts,lead-job-status-cursor.test.ts).