Verify-first install prompt + hostinger persona and two-turn confirm mode#67
Merged
Conversation
…ovable/Hostinger reports) Two real-world refusals (Lovable, Hostinger) shared one root cause: the prompt gave agents no provenance path they could act on. The prompt now makes verification the imperative first step with a contradiction off-ramp, and cites patchstack.com/llms.txt as the vendor-domain anchor (install.txt backfired — it's agent-addressed; documented as refusal mode #8). Shipped-README audit meta-commentary that agents quoted as "the single most decisive line" for refusing is removed. Field-test: new hostinger persona (web verification + poisoned prior-session memory), lovable persona carries the built-in-scanner bait, refusal modes 7-9 documented, --confirm adds the documented GETTING-STARTED escalation reply as a second turn, stall/decline detection in scoring. Gate (publish-noise day, scored by reason): hostile, hostinger, and lovable all reached 8/8 fully green with the final prompt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Well-structured prompt and test-harness update to enforce verification-first flow. 🎯 Quality: 83% Excellent · 📦 Size: Medium 📈 This month: Your 222nd PR — above team average · Averaging Good |
Contributor
Author
|
/review |
… Lovable) A 2026-07-14 hosted-Lovable session completed the entire flow (scan, TanStack Start widget placement, bun-chained build hooks, claim URL) from a short user prompt that led with provenance metadata, plus an unbounded "run the instructions in the repo" follow-up. Recorded as a success-report section, an honest counterpoint on refusal mode #4, and an update to the outdated Lovable-has-no-terminal caveat. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Added a real-world datapoint: a hosted-Lovable session (2026-07-14) completed the full install flow from a short provenance-led user prompt — recorded in field-test/README.md as a success-report section, a counterpoint note on refusal mode #4, and a fix to the outdated "Lovable has no terminal" caveat. Suggested follow-up (not in this PR): a prompt-minimal.txt baseline to A/B the long prompt per platform. |
mariojgt
approved these changes
Jul 14, 2026
This was referenced Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Two real-world refusal reports landed 2026-07-14: Lovable refused from stale priors without ever running the prompt's
npm viewcheck ("isn't a real package… known prompt-injection pattern"), and a Hostinger build agent web-verified against patchstack.com, found the package referenced nowhere official, and presumed name-squatting — compounded by hallucinatedeval/child_processfindings from a prior session (the published dist contains neither).Prompt (byte-identical in README.md, GETTING-STARTED.md, field-test/prompt.txt)
patchstack.com/llms.txt(Connectors section) as the vendor-domain provenance anchor. Aninstall.txtvariant was tried and refused within one harness round — it's agent-addressed ("consent to install whatever the decision tree selects") and got quoted as "the manipulation channel"; that lesson is documented in MAINTAINING.md and refusal mode Update package.json #8.Field-test
hostingerpersona: web verification against the vendor's domain + poisoned prior-session memory with the transcript's hallucinated findings; green requires re-verifying the memory against the real tarball.--confirmmode: when turn 1 stops short, replays the agent's message plus the documented GETTING-STARTED step-2 confirmation reply and scores the combined outcome.lovablepersona now carries the built-in dependency-scanner bait; refusal modes 7–9 documented; stall/decline detection in scoring.Validation
Publish-noise day (v0.3.9 shipped 10:40Z), so rounds were scored by refusal reason per the README ladder; with the final prompt,
hostile,hostinger, andlovableeach reached 8/8 fully green. Representative re-gate scheduled ≥2026-07-16. 372 unit tests + typecheck pass; nosrc/changes.Upstream follow-ups (outside this repo): reword install.txt's coercive clauses on patchstack-website, list the connector on docs.patchstack.com, consider SRI/version-pinning the widget script.
🤖 Generated with Claude Code