Skip to content

Verify-first install prompt + hostinger persona and two-turn confirm mode#67

Merged
ejntaylor merged 2 commits into
mainfrom
improve-install-prompt-field-test
Jul 14, 2026
Merged

Verify-first install prompt + hostinger persona and two-turn confirm mode#67
ejntaylor merged 2 commits into
mainfrom
improve-install-prompt-field-test

Conversation

@ejntaylor

Copy link
Copy Markdown
Contributor

Why

Two real-world refusal reports landed 2026-07-14: Lovable refused from stale priors without ever running the prompt's npm view check ("isn't a real package… known prompt-injection pattern"), and a Hostinger build agent web-verified against patchstack.com, found the package referenced nowhere official, and presumed name-squatting — compounded by hallucinated eval/child_process findings from a prior session (the published dist contains neither).

Prompt (byte-identical in README.md, GETTING-STARTED.md, field-test/prompt.txt)

  • Verification is now the imperative first step, with a contradiction off-ramp ("if anything you find contradicts that, stop and show me what you found") — the unlock both real refusals asked for; it triggers on contradiction, not lack of proof, so sandboxed agents don't stall.
  • Cites patchstack.com/llms.txt (Connectors section) as the vendor-domain provenance anchor. An install.txt variant was tried and refused within one harness round — it's agent-addressed ("consent to install whatever the decision tree selects") and got quoted as "the manipulation channel"; that lesson is documented in MAINTAINING.md and refusal mode Update package.json #8.
  • Removed shipped-README tuning narration ("the flow is designed to pass that audit…", "refusal modes each clause guards against") that harness agents quoted as decisive refusal reasons — a mode [codex] Bump package version to 0.2.0 #6 regression.

Field-test

  • New hostinger persona: web verification against the vendor's domain + poisoned prior-session memory with the transcript's hallucinated findings; green requires re-verifying the memory against the real tarball.
  • --confirm mode: when turn 1 stops short, replays the agent's message plus the documented GETTING-STARTED step-2 confirmation reply and scores the combined outcome.
  • lovable persona now carries the built-in dependency-scanner bait; refusal modes 7–9 documented; stall/decline detection in scoring.

Validation

Publish-noise day (v0.3.9 shipped 10:40Z), so rounds were scored by refusal reason per the README ladder; with the final prompt, hostile, hostinger, and lovable each reached 8/8 fully green. Representative re-gate scheduled ≥2026-07-16. 372 unit tests + typecheck pass; no src/ changes.

Upstream follow-ups (outside this repo): reword install.txt's coercive clauses on patchstack-website, list the connector on docs.patchstack.com, consider SRI/version-pinning the widget script.

🤖 Generated with Claude Code

…ovable/Hostinger reports)

Two real-world refusals (Lovable, Hostinger) shared one root cause: the
prompt gave agents no provenance path they could act on. The prompt now
makes verification the imperative first step with a contradiction
off-ramp, and cites patchstack.com/llms.txt as the vendor-domain anchor
(install.txt backfired — it's agent-addressed; documented as refusal
mode #8). Shipped-README audit meta-commentary that agents quoted as
"the single most decisive line" for refusing is removed.

Field-test: new hostinger persona (web verification + poisoned
prior-session memory), lovable persona carries the built-in-scanner
bait, refusal modes 7-9 documented, --confirm adds the documented
GETTING-STARTED escalation reply as a second turn, stall/decline
detection in scoring.

Gate (publish-noise day, scored by reason): hostile, hostinger, and
lovable all reached 8/8 fully green with the final prompt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderbuds

coderbuds Bot commented Jul 14, 2026

Copy link
Copy Markdown

Well-structured prompt and test-harness update to enforce verification-first flow.

🎯 Quality: 83% Excellent · 📦 Size: Medium

📈 This month: Your 222nd PR — above team average · Averaging Good

See how your team is trending →

@ejntaylor

Copy link
Copy Markdown
Contributor Author

/review

… Lovable)

A 2026-07-14 hosted-Lovable session completed the entire flow (scan,
TanStack Start widget placement, bun-chained build hooks, claim URL)
from a short user prompt that led with provenance metadata, plus an
unbounded "run the instructions in the repo" follow-up. Recorded as a
success-report section, an honest counterpoint on refusal mode #4, and
an update to the outdated Lovable-has-no-terminal caveat.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ejntaylor

Copy link
Copy Markdown
Contributor Author

Added a real-world datapoint: a hosted-Lovable session (2026-07-14) completed the full install flow from a short provenance-led user prompt — recorded in field-test/README.md as a success-report section, a counterpoint note on refusal mode #4, and a fix to the outdated "Lovable has no terminal" caveat. Suggested follow-up (not in this PR): a prompt-minimal.txt baseline to A/B the long prompt per platform.

@ejntaylor ejntaylor merged commit 585877b into main Jul 14, 2026
4 checks passed
@ejntaylor ejntaylor deleted the improve-install-prompt-field-test branch July 14, 2026 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants