Skip to content

chore(workflows): add cyclotron index for the reschedule sweep predicate - #71473

Merged
mayteio merged 2 commits into
masterfrom
posthog-code/workflows-reschedule-sweep-index
Jul 16, 2026
Merged

chore(workflows): add cyclotron index for the reschedule sweep predicate#71473
mayteio merged 2 commits into
masterfrom
posthog-code/workflows-reschedule-sweep-index

Conversation

@mayteio

@mayteio mayteio commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Problem

#71293 adds a reschedule sweep whose queries filter cyclotron_jobs by function_id, action_id, status, and scheduled. action_id has never been indexed, so at the scale the sweep is built for (parked backlogs into the hundreds of thousands) each call would repeat a heap-filtered scan up to ~22 times against the live queue table. Raised as a must-fix by ReviewHog on #71293; the migrations-separation check (correctly) refuses to let the migration ship in the same PR as the service code, hence this standalone PR.

Changes

One sqlx migration for the cyclotron-node database:

CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_cyclotron_jobs_action_reschedule
    ON cyclotron_jobs (function_id, action_id, scheduled)
    WHERE status = 'available';

Partial on status = 'available' (only parked rows are sweepable, and it keeps the index off the write path of terminal transitions), scheduled as a trailing key column so the sweep's range predicate lands in the Index Cond rather than a heap filter - same reasoning, and same CREATE INDEX CONCURRENTLY / -- no-transaction pattern, as the email fair-dequeue composite index (20260622000001). team_id is deliberately left out: function_id is a per-workflow UUID and already maximally selective, so team_id stays a cheap residual filter.

Ordering: should merge before #71293's feature is exercised, but nothing breaks without it - the sweep's queries fall back to the existing function_id index plus a heap filter, just slower.

How did you test this code?

Ran the migration against the local cyclotron-node test database, seeded 50k parked rows (mixed action_ids, one workflow), and confirmed via EXPLAIN that the planner uses a bitmap index scan with all three columns in the Index Cond and only team_id as a residual filter. The full cyclotron-v2 Jest suite (113 tests) passes against the migrated schema. No new tests: an index has no behavior of its own to pin, and the queries it serves are covered in #71293.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

Not needed - the module README note pointing at this index ships with #71293.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Authored with PostHog Code (Claude). Split out of #71293 after the migrations-separation check flagged the sqlx migration + nodejs service code combination; the index itself was ReviewHog's (valid) must-fix finding there.


Created with PostHog Code

Partial composite index (function_id, action_id, scheduled) WHERE
status = 'available' on cyclotron_jobs, supporting the timing-edit
reschedule sweep queries (#71293). Shipped separately per the
migration/service separation rule; the sweep's queries are safe without
it (they fall back to the function_id index plus a heap filter), so
ordering against the service deploy is not critical - but this should
merge first so the sweep never runs unindexed at scale.

Generated-By: PostHog Code
Task-Id: 7f5083cf-5d17-4f37-8bee-8c43db70cb79
@mayteio mayteio self-assigned this Jul 16, 2026
mayteio added a commit that referenced this pull request Jul 16, 2026
The migrations-separation check forbids sqlx migrations and nodejs
service changes in one PR; the index now ships in #71473.

Generated-By: PostHog Code
Task-Id: 7f5083cf-5d17-4f37-8bee-8c43db70cb79
@trunk-io

trunk-io Bot commented Jul 16, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@mayteio
mayteio marked this pull request as ready for review July 16, 2026 10:42
@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "chore(workflows): add cyclotron index fo..." | Re-trigger Greptile

@mayteio
mayteio merged commit 1a88285 into master Jul 16, 2026
193 checks passed
@mayteio
mayteio deleted the posthog-code/workflows-reschedule-sweep-index branch July 16, 2026 12:18
@deployment-status-posthog

deployment-status-posthog Bot commented Jul 16, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-07-16 12:48 UTC Run
prod-us ✅ Deployed 2026-07-16 13:08 UTC Run
prod-eu ✅ Deployed 2026-07-16 13:10 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants