fix(workflows): enable BullMQ retries for workflow-run.process [ENG-1228]#8404
fix(workflows): enable BullMQ retries for workflow-run.process [ENG-1228]#8404xernobyl wants to merge 1 commit into
Conversation
…228] Now that send_email execution is idempotent per step (claim-before-send + @@unique([runId, stepId]), #8395), drop the attempts:1 override on enqueueWorkflowRunJob so workflow-run.process inherits the shared retry policy (attempts + exponential backoff from the queue defaultJobOptions). A transient failure (SMTP blip, DB pool exhaustion, stalled redelivery) now retries and resumes instead of dying on the first attempt; the deterministic jobId keeps a re-enqueue idempotent. Must merge with/after #8395 — retries must never be enabled without the per-step send idempotency.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
WalkthroughThis change removes an explicit Changes
Sequence Diagram(s)Not applicable — this change is a configuration adjustment without new interaction flows. Related Issues: None provided. Related PRs: None provided. Suggested labels: jobs, bug-fix, testing Suggested reviewers: None specified. Poem 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
|
Folding this into #8403 — the retry-enablement and the orphan reconciler touch the same |



What does this PR do?
Pairs with #8395 (ENG-1228), stacked on
epic/workflows-v1. Now thatsend_emailexecution is idempotent per step (claim-before-send +@@unique([runId, stepId]), in #8395), this drops theattempts: 1override onenqueueWorkflowRunJobsoworkflow-run.processinherits the shared retry policy —attempts: 3+ exponential backoff from the queue'sdefaultJobOptions, same as every other job.Why: with
attempts: 1,maxAttemptswas always 1, so the executor's retry/resume machinery and its pool-exhaustion handling never actually fired (every attempt was "final") — a transient failure (SMTP blip, DB pool exhaustion, stalled redelivery) killed the run on the first try and could strand it. With retries enabled and the per-step idempotency from #8395, a transient failure now retries and resumes without re-sending. The deterministicjobId(the run id) keeps a re-enqueue idempotent (e.g. the ENG-1608 reconciler re-dispatching an orphan).How should this be tested?
One-line behaviour change (drop the
attempts: 1override). Theworkflow-runenqueue test now asserts no per-jobattemptsoverride, so the job inherits the sharedattempts: 3+ backoff policy — which itself is covered by the existing "creates the shared queue with the expected defaults" test. End-to-end (worker running): asend_emailstep that throws a transient error now retries up to 3× and resumes from the claimed-step log, instead of failing permanently on attempt 1.Checklist
Required
console.logs(none)epic/workflows-v1)Appreciated