feat: agent harness can run arbitrary workflows#385
Conversation
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
This stack of pull requests is managed by Graphite. Learn more about stacking. |
gewenyu99
changed the title
sarahxsanders
left a comment
sarahxsanders
left a comment
There was a problem hiding this comment.
worked locally!! and other wizard runs are working as well :)
a few things:
- didn't realized I needed the feat/revenue-analytics-setup-skillbranch in context-mill checked out to test, might want to add to description!
- wizard remarks flagged this which should be fixed: the skill's SKILL.MD has no greenfield path so the wizard actually just used it's claude brain and improvised to create stripe.js. there's only guidance in the skill for when code exists in the first place, not when it doesn't exist at all
OOoo I think this should work with latest main? It is merged 👀
What do you mean by this? Like when there is no stripe integration or no |
|
it didn't work in main :hmmmm I'll retry though for the wizard remarks: I ran it on a tmp app claude made that didn't have stripe set up correctly (didn't realize until after the fact lol) but it detected the presence of the folder it should have been in. so it set Stripe up for me before correctly setting up rev analytics too 👀 wasn't sure if this was intended |
yikes we gotta tell it not to do that omg |
|
replicated it: detection checks only package.json for stripe not whether stripe is actually used in any code. so any project with stripe installed will trigger it and the agent sets up from scratch. we should def limit this. fix would be in |
|
@sarahxsanders Try with this workbench branch and main on context mill! I'm gonna address the other concern through the context mill |
Added an abort signal when we find invalid posthog or stripe integrations we can use, gonna push it up @sarahxsanders |
|
Abort on wizard: https://app.graphite.com/github/pr/PostHog/wizard/389?mode=tour Abort logic just added to context mill guardrails |
|
@gewenyu99 big PR, but it's mostly organization and wrapper abstractions |
## Problem chasing the 🐇 down the 🕳️ [graphite link](https://app.graphite.com/github/pr/PostHog/wizard/388?mode=tour) tl;dr - **One runner** (`agent-runner.ts`) with a configurable pipeline - **One config type** (`WorkflowRun`) that controls what varies between workflows - **Extracted detection modules** that any workflow can call - **A workflow registry** that auto-wires CLI subcommands and TUI flows - **Prompt assembly** that can run a custom prompt, a skill, or both ```bash wizard // core integration workflow wizard integrate // core integration workflow wizard revenue // revenue analytics workflow wizard skill={skill-id} // runs any skill from context mill ``` <img width="3046" height="2303" alt="image" src="https://github.com/user-attachments/assets/ba0000d6-ec70-46da-99b9-7a3774eba9a9" />
edwinyjlim
changed the title
sarahxsanders
left a comment
sarahxsanders
left a comment
There was a problem hiding this comment.
looks like the stripe detection thing is all fixed!
definitely add the abort boundaries for the core wizard run, but lgtm
|
big question: how does this interact with the mainline integration? everyone new should be getting this treatment too right? |
Mainline is using workflows, but on the outside will be functionally identical |
Merge activity
|
4 participants
Once again I suggest looking at this in graphite
Run the workflow locally with: