fix(subscriptions): freeze post-fix HogQL in the AI query plan

[vdekrijger]

Problem

AI subscription reports freeze a query plan (Subscription.ai_query_plan) on the first delivery so later runs skip the planner LLM and produce consistent results. But the frozen plan stored the planner's original per-step HogQL — and a step can succeed only after the fix LLM (_arequest_hogql_fix) rewrites its query mid-run.

Freezing the pre-fix form means every reused delivery replays the known-broken query, fails, and re-bills the fix LLM for the same repair — the exact repetition the frozen plan exists to avoid. Determinism is also only partial: the fixer re-rolls each run and may not produce the same rewrite twice.

Found during review of the (now closed) #66824.

Changes

• run_step writes each succeeding step's final window-agnostic HogQL (post any fix-LLM rewrite) back onto its plan step, so _plan_to_freeze freezes what actually ran. A failed step never reaches the write-back and keeps the planner's original — a broken rewrite is never kept. Synthesis reads only the plan's intent, so the in-place update has no other readers. (An earlier revision threaded this through widened return tuples; a review pass simplified it to the write-back.)
• The existing fail-safe now applies to the post-fix text: if the fixer stripped the {{date_range}}-style window placeholder, nothing is frozen (freezing it would cement an unbounded scan) and the next run re-plans — same degradation semantics as today.

flowchart LR
    P[planner HogQL] --> E{executes?}
    E -- yes --> F[freeze as-is]
    E -- no --> X[fix LLM rewrite] --> R{rerun succeeds?}
    R -- yes --> G[freeze POST-fix HogQL]
    R -- no --> O[keep original, step degrades]
    G --> H{placeholder still present?}
    H -- no --> N[do not freeze - re-plan next run]

Loading

How did you test this code?

I'm an agent (Claude Code via PostHog Code). Automated tests only — no manual testing claimed.

• New parameterized test test_freeze_carries_post_fix_hogql (through the public generate_ai_report): (a) a step fixed by the fix LLM freezes the post-fix HogQL, not the original — catches the regression where reused plans replay broken queries; (b) a fixer output without the window placeholder freezes nothing — pins the fail-safe to the post-fix text.
• Extended test_run_steps_retries_then_succeeds to assert the freezable form is the fixed query.
• Full touched suite green locally: test_report_pipeline.py 30/30. Existing tests updated only mechanically for the widened _run_steps/_execute_plan return tuple; none encoded the old pre-fix freezing behavior.
• ruff check / ruff format clean. (Local DB-fixture errors in unrelated test files reproduce identically on clean master — pre-existing environment staleness, not this change.)

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

No user-facing docs describe the freeze internals; behavior change is cost/consistency only.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

• Built with Claude Code (PostHog Code) in an isolated worktree, directed by @vdekrijger.
• Origin: review finding on feat(subscriptions): edit and regenerate the frozen AI query plan #66824 (closed in favor of this upstream fix — improving planner/frozen-plan quality beats maintaining owner-facing plan-editing escape hatches).
• Key decision: failed steps keep the planner's original HogQL in the frozen plan rather than a possibly-broken rewrite; the placeholder fail-safe intentionally moved to the post-fix text.

---

Created with PostHog Code

[greptile-apps]

_{Reviews (1): Last reviewed commit: "refactor(subscriptions): write post-fix ..." | Re-trigger Greptile}

[github-actions]

Contained fix to internal AI subscription plan-freezing logic with added parameterized tests covering both the fix-path and fail-safe; author is on the owning team with strong familiarity, and Greptile reviewed with no concerns. No risky territory (no schema/API/auth/billing/CI/deps touched).

• Author wrote 100% of the modified lines and has 16 merged PRs in these paths (familiarity STRONG).
• 👍 on the PR from greptile-apps[bot], hex-security-app[bot].

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	5L, 1F substantive, 56L/2F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1b-small (56L, 2F, single-area, fix)
stamphog 2.0.0b3		.stamphog/policy.yml @ 1c274d3 · reviewed head 96b169f

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[stamphog]

Retaining stamphog approval — delta since last review classified as trivial_paths.

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-09 15:35 UTC	Run
prod-us	✅ Deployed	2026-07-09 16:06 UTC	Run
prod-eu	✅ Deployed	2026-07-09 16:06 UTC	Run