Add the Factory — supervision for weak-model app builds - #397
Merged
Conversation
Weak models write code well but manage themselves badly: 26 logged builds, and every failure was self-management — churning on one dead end, fabricating causes where evidence was blank, shrinking "email me" into "logs it", announcing success after failed verification, or quitting unnoticed. The model keeps the hands; this takes away the clipboard. - app/factory: stdlib-pure Machine (persisted arc, 3×/12 retry caps, escalation, redispatch-on-surrender, honest stuck reports), defect cards (cause = quoted evidence or "unknown" — theories unrepresentable), build graph as a pure transition (only a parsed PASS reaches done), distiller, cookbooks (one generic callAction pattern + dry-run param discovery, email as the proven worked example), CraftBot host adapter; layering lint + 3 test suites incl. replays of two real incidents. - Fix missions are fresh runs carrying cards; ready/stuck messages are machine-composed — agent-authored status is retired. - Evidence everywhere the platform was silent: gate stderr, response bodies, refused-request URLs, boot-log excerpts, handler exceptions, migrate-hang kills, pre-boot migration-rename check, CLI error.cause unwrapping. - Integrations: bridge action endpoint runs CraftBot's own implementations (send_gmail, semantic params), grants derived from code by the gate, irreversible-confirm + dry-run; omitted `to` = account owner — apps never hold identity. Wizard offers marketplace matches; capability map injected. - Verifier: full spec coverage required, value plausibility, fail-closed verdict parsing.
… into livingui-third-party-fix
ahmad-ajmal
changed the title
Collaborator
|
Despite having hard-coded filters and pattern-matching patterns, those are only activated when with bad model. However, a solution like this will miss many edge cases and unforesee situation. We will have to revise in the future. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cheap AI models write good code but manage themselves badly. Across 26
recorded builds, every failure was bad self-management: retrying the same
dead end for half an hour, inventing explanations when error details were
missing, quietly downgrading "email me" into "log it", claiming success
right after its own test failed, or just stopping — with nothing noticing.
So the AI keeps doing all the coding, and a small plain program now does
all the managing.
file. It doesn't direct the work — on a clean build it only confirms the
checkpoints and announces the result. It steps in only when something
fails or stalls: it allows tries per problem, then stops
with an honest report. Failures are turned into "defect cards": what
broke, the exact error line as proof, and a command to reproduce it.
A card can never contain a guess — proof or "unknown".
short file of known-good code patterns to copy. All "ready" and "failed"
messages to the user are written by the program from its records — the
AI no longer reports on its own work.
which URL failed and why, crash logs. The AI only invented explanations
where the platform gave it nothing — so nothing gives it nothing anymore.
runs CraftBot's own tested code. Permissions are read from the app's
code automatically; sending needs an explicit confirm; a "dry run" mode
checks everything without sending. Apps never see the user's email
address or passwords. The setup wizard now also offers ready-made
marketplace apps before building from scratch.
looks fake (all zeros), and its verdict only counts if it parses cleanly.