Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/respect-backoff-max-elapsed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---

Stop `ExponentialBackoff.execute()` retries when callback execution time pushes the run past `maxElapsed`.
9 changes: 8 additions & 1 deletion packages/core/src/v3/apps/backoff.ts
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 maxElapsed wall-clock check is bypassed when callback throws AttemptTimeout

The new maxElapsed check at line 343 is placed after the try/catch/finally block. When the callback throws an AttemptTimeout (line 335-337), the continue statement skips all code after the try/catch/finally, including the new maxElapsed check. Although the finally block still executes and correctly updates elapsedMs, the actual check is never evaluated.

This means if a caller uses both attemptTimeoutMs and maxElapsed, and every attempt times out, the loop will never exit early due to maxElapsed. It will only stop when retryAsync's delay-based check (which doesn't account for callback execution time, packages/core/src/v3/apps/backoff.ts:127-130) or maxRetries terminates the generator. For example, with maxElapsed: 10 (seconds), attemptTimeoutMs: 3000, and maxRetries: 100, the loop could run for up to ~300 seconds instead of stopping after 10.

(Refers to lines 335-337)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@ export class ExponentialBackoff {
return this.#clone(undefined, { maxRetries });
}

// TODO: With .execute(), should this also include the time it takes to execute the callback?
maxElapsed(maxElapsed?: number) {
return this.#clone(undefined, { maxElapsed });
}
Expand Down Expand Up @@ -340,6 +339,14 @@ export class ExponentialBackoff {
elapsedMs += Date.now() - start;
clearTimeout(attemptTimeout);
}

if (elapsedMs / 1000 > this.#maxElapsed) {
return {
success: false,
cause: "Timeout",
error: finalError,
};
}
}

if (finalError instanceof AttemptTimeout) {
Expand Down
27 changes: 27 additions & 0 deletions packages/core/test/backoff.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { describe, expect, it } from "vitest";
import { ExponentialBackoff } from "../src/v3/apps/backoff.js";

describe("ExponentialBackoff", () => {
it("stops execute retries when callback time exceeds maxElapsed", async () => {
const backoff = new ExponentialBackoff("NoJitter", {
factor: 0,
maxElapsed: 0.01,
maxRetries: 5,
});
const error = new Error("retryable");
let attempts = 0;

const result = await backoff.execute(async () => {
attempts++;
await new Promise((resolve) => setTimeout(resolve, 25));
throw error;
});

expect(result).toEqual({
success: false,
cause: "Timeout",
error,
});
expect(attempts).toBe(1);
});
});