fix(integration): known-good insight payloads in the dashboard step + soften event-plan cleanup#214
Merged
Conversation
… soften event-plan cleanup Two field fixes from the wizard's 2026-07-07 triage (items 6 and 8): 1. The dashboard step now includes two known-good insight-create payloads, verified against the live MCP: a trends insight with a breakdown (breakdowns live in breakdownFilter.breakdowns — agents kept sending a top-level TrendsQuery.breakdown and getting rejected) and a conversion funnel (funnelWindowInterval/funnelWindowIntervalUnit are camelCase INSIDE funnelsFilter — agents put them at the FunnelsQuery top level or used snake_case). Also lists the valid trendsFilter.display enum, the other recurring rejection (~5 remarks/week guessed ActionsBarChart). Upstream ask to make the MCP errors self-describing: PostHog/posthog#68944. 2. The conclude step told the agent to "remove .posthog-events.json" unconditionally — on the pi harness rm is fence-blocked, making this the #1 bash denial (~31/week) and a retry loop. Now: update the file to match implemented events, try removal with file tools, and if removal is blocked leave it — the wizard host cleans it up after the run. Generated-By: PostHog Code Task-Id: bcb96bcb-e37f-421d-841d-738647c73b8e
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
edwinyjlim
approved these changes
Jul 7, 2026
Collaborator
Author
|
Live validation against us.posthog.com project 2, 2026-07-07 ~15:01 UTC. Test insights soft-deleted after creation.
{
"name": "Signups by plan (wizard) — schema validation, will delete",
"query": {
"kind": "InsightVizNode",
"source": {
"kind": "TrendsQuery",
"series": [{ "kind": "EventsNode", "event": "user_signed_up", "math": "total" }],
"interval": "day",
"dateRange": { "date_from": "-30d" },
"breakdownFilter": { "breakdowns": [{ "type": "event", "property": "plan" }] },
"trendsFilter": { "display": "ActionsBar" }
}
}
}Server-normalized query echoed back: {
"kind": "InsightVizNode",
"source": {
"kind": "TrendsQuery",
"series": [{ "kind": "EventsNode", "math": "total", "event": "user_signed_up" }],
"interval": "day",
"dateRange": { "date_from": "-30d", "explicitDate": false },
"breakdownFilter": { "breakdowns": [{ "type": "event", "property": "plan" }], "breakdown_type": "event" },
"trendsFilter": { "display": "ActionsBar", "...defaults": "..." },
"filterTestAccounts": false,
"version": 2
}
}
{
"name": "Signup funnel (wizard) — schema validation, will delete",
"query": {
"kind": "InsightVizNode",
"source": {
"kind": "FunnelsQuery",
"series": [
{ "kind": "EventsNode", "event": "page_viewed" },
{ "kind": "EventsNode", "event": "user_signed_up" }
],
"dateRange": { "date_from": "-30d" },
"funnelsFilter": {
"funnelVizType": "steps",
"funnelOrderType": "ordered",
"funnelWindowInterval": 14,
"funnelWindowIntervalUnit": "day"
}
}
}
}Server-normalized query echoed back: {
"kind": "InsightVizNode",
"source": {
"kind": "FunnelsQuery",
"series": [
{ "kind": "EventsNode", "event": "page_viewed" },
{ "kind": "EventsNode", "event": "user_signed_up" }
],
"dateRange": { "date_from": "-30d", "explicitDate": false },
"funnelsFilter": {
"funnelVizType": "steps",
"funnelOrderType": "ordered",
"funnelStepReference": "total",
"funnelWindowInterval": 14,
"funnelWindowIntervalUnit": "day",
"breakdownAttributionType": "first_touch",
"layout": "vertical",
"exclusions": []
},
"filterTestAccounts": false
}
}
{
"series": [
{ "kind": "EventsNode", "event": "$pageview" },
{ "kind": "EventsNode", "event": "insight analyzed" }
],
"dateRange": { "date_from": "-7d" },
"funnelsFilter": {
"funnelVizType": "steps",
"funnelOrderType": "ordered",
"funnelWindowInterval": 14,
"funnelWindowIntervalUnit": "day"
}
}{ "results": [
{ "name": "$pageview", "order": 0, "count": 273310 },
{ "name": "insight analyzed", "order": 1, "count": 28964, "median_conversion_time": 10.62 }
] }
{
"series": [{ "kind": "EventsNode", "event": "$pageview", "math": "total" }],
"interval": "day",
"dateRange": { "date_from": "-7d" },
"breakdownFilter": { "breakdowns": [{ "type": "event", "property": "$browser" }] },
"trendsFilter": { "display": "ActionsBar" }
}{ "results": [
{ "label": "Chrome", "count": 7303852 },
{ "label": "Safari", "count": 579330 },
{ "label": "Brave", "count": 565727 },
{ "label": "Firefox", "count": 557183 },
{ "...": "..." }
] } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Items 6 (partial) and 8 from the wizard's 2026-07-07 triage (PostHog/wizard@eebc04a). Both changes are in
integration/references/4-conclude.md, the single source shared by every framework variant.1. Known-good insight payloads in the dashboard step
Agents build the first dashboard by trial and error (~7 remarks/week across both harnesses):
TrendsQuery.breakdownrejected (the accepted shape isbreakdownFilter.breakdowns),funnelWindowInterval/funnelWindowIntervalUnitrejected at theFunnelsQuerytop level or as snake_case (they're camelCase insidefunnelsFilter), and guessedtrendsFilter.displayvalues (ActionsBarChart→ invalid).The dashboard step now pastes two complete
insight-createpayloads — both verified against the live MCP (query-trendsandquery-funnelaccepted and computed them as-is) — plus the validdisplayenum, and tells the agent to fix a rejected payload against the examples instead of retrying variations.Upstream ask to make the MCP validation errors self-describing: PostHog/posthog#68944.
2.
.posthog-events.jsoncleanup no longer demands removalThe conclude step said "Upon completion, remove .posthog-events.json." unconditionally. On the wizard's pi harness
rmis fence-blocked, making this the #1 bash denial (~31/week) and a retry loop; the wizard host already deletes the file after the run. Now: update the file to match the implemented events, try removal with file tools, and if removal is blocked leave it and move on — explicitly no shell fallbacks and no retries. The anthropic path (where removal works) keeps working.Complements PostHog/wizard#816, which adds the same guidance to the pi runtime notes wizard-side.
pnpm test(95) andpnpm build(114 skills) green.🤖 Generated with Claude Code