diff --git a/src/assets/images/changelog/d1/d1-auto-retry-success-ratio.png b/src/assets/images/changelog/d1/d1-auto-retry-success-ratio.png new file mode 100644 index 000000000000000..fddeb10e404fe17 Binary files /dev/null and b/src/assets/images/changelog/d1/d1-auto-retry-success-ratio.png differ diff --git a/src/content/changelog/d1/2025-09-11-d1-automatic-read-retries.mdx b/src/content/changelog/d1/2025-09-11-d1-automatic-read-retries.mdx new file mode 100644 index 000000000000000..18cf8bdf8ae0a18 --- /dev/null +++ b/src/content/changelog/d1/2025-09-11-d1-automatic-read-retries.mdx @@ -0,0 +1,22 @@ +--- +title: D1 automatically retries read-only queries +description: D1 now detects read-only queries and automatically attempts up to two retries to execute those queries in the event of failures with retryable errors. +products: + - d1 + - workers +date: 2025-09-11 +--- + +D1 now detects read-only queries and automatically attempts up to two retries to execute those queries in the event of failures with retryable errors. You can access the number of execution attempts in the returned [response metadata](/d1/worker-api/return-object/#d1result) property `total_attempts`. + +At the moment, only read-only queries are retried, that is, queries containing only the following SQLite keywords: `SELECT`, `EXPLAIN`, `WITH`. Queries containing any [SQLite keyword](https://sqlite.org/lang_keywords.html) that leads to database writes are not retried. + +The retry success ratio among read-only retryable errors varies from 5% all the way up to 95%, depending on the underlying error and its duration (like network errors or other internal errors). + +The retry success ratio among all retryable errors is lower, indicating that there are write-queries that could be retried. Therefore, we recommend D1 users to continue applying [retries in their own code](/d1/best-practices/retry-queries/) for queries that are not read-only but are idempotent according to the business logic of the application. + +![D1 automatically query retries success ratio](~/assets/images/changelog/d1/d1-auto-retry-success-ratio.png) + +D1 ensures that any retry attempt does not cause database writes, making the automatic retries safe from side-effects, even if a query causing changes slips through the read-only detection. D1 achieves this by checking for modifications after every query execution, and if any write occurred due to a retry attempt, the query is rolled back. + +The read-only query detection heuristics are simple for now, and there is room for improvement to capture more cases of queries that can be retried, so this is just the beginning. diff --git a/src/content/docs/d1/best-practices/retry-queries.mdx b/src/content/docs/d1/best-practices/retry-queries.mdx new file mode 100644 index 000000000000000..d2d46d84a6f0f1f --- /dev/null +++ b/src/content/docs/d1/best-practices/retry-queries.mdx @@ -0,0 +1,44 @@ +--- +title: Retry queries +pcx_content_type: concept +sidebar: + order: 3 + +--- + +import { GitHubCode } from "~/components"; + +It is useful to retry write queries from your application when you encounter a transient [error](/d1/observability/debug-d1/#error-list). From the list of `D1_ERROR`s, refer to the Recommended action column to determine if a query should be retried. + +:::note +D1 automatically retries read-only queries up to two more times when it encounters a retryable error. +::: + +## Example of retrying queries + +Consider the following example of a `shouldRetry(...)` function, taken from the [D1 read replication starter template](https://github.com/cloudflare/templates/blob/main/d1-starter-sessions-api-template/src/index.ts#L108). + +You should make sure your retries apply an exponential backoff with jitter strategy for more successful retries. +You can use libraries abstracting that already like [`@cloudflare/actors`](https://github.com/cloudflare/actors), or [copy the retry logic](https://github.com/cloudflare/actors/blob/9ba112503132ddf6b5cef37ff145e7a2dd5ffbfc/packages/core/src/retries.ts#L18) in your own code directly. + +```ts +import { tryWhile } from "@cloudflare/actors"; + +function queryD1Example(d1: D1Database, sql: string) { + return await tryWhile(async () => { + return await d1.prepare(sql).run(); + }, shouldRetry); +} + +function shouldRetry(err: unknown, nextAttempt: number) { + const errMsg = String(err); + const isRetryableError = + errMsg.includes("Network connection lost") || + errMsg.includes("storage caused object to be reset") || + errMsg.includes("reset because its code was updated"); + if (nextAttempt <= 5 && isRetryableError) { + return true; + } + return false; +} +``` \ No newline at end of file diff --git a/src/content/docs/d1/observability/debug-d1.mdx b/src/content/docs/d1/observability/debug-d1.mdx index 5d04ef5035000ea..7b70c0260d5ac81 100644 --- a/src/content/docs/d1/observability/debug-d1.mdx +++ b/src/content/docs/d1/observability/debug-d1.mdx @@ -59,7 +59,7 @@ While some D1 errors can be resolved by retrying the operation, retrying is only Before retrying any failed operation: - Verify your query is idempotent (for example, read-only operations, or queries such as `CREATE TABLE IF NOT EXISTS`). -- Consider implementing application-level checks to identify if the operation can be retried, and retrying only when it is safe and necessary. +- Consider [implementing application-level checks](/d1/best-practices/retry-queries/) to identify if the operation can be retried, and retrying only when it is safe and necessary. ::: | `D1_ERROR` type | Description | Recommended action | @@ -79,6 +79,16 @@ Before retrying any failed operation: +## Automatic retries + +D1 detects read-only queries and automatically attempts up to two retries to execute those queries in the event of failures with retryable errors. + +D1 ensures that any retry attempt does not cause database writes, making the automatic retries safe from side-effects, even if a query causing modifications slips through the read-only detection. D1 achieves this by checking for modifications after every query execution, and if any write occurred due to a retry attempt, the query is rolled back. + +:::note +Only read-only queries (queries containing only the following SQLite keywords: `SELECT`, `EXPLAIN`, `WITH`) are retried. Queries containing any [SQLite keyword](https://sqlite.org/lang_keywords.html) that leads to database writes are not retried. +::: + ## View logs View a stream of live logs from your Worker by using [`wrangler tail`](/workers/observability/logs/real-time-logs#view-logs-using-wrangler-tail) or via the [Cloudflare dashboard](/workers/observability/logs/real-time-logs#view-logs-from-the-dashboard). diff --git a/src/content/docs/d1/worker-api/return-object.mdx b/src/content/docs/d1/worker-api/return-object.mdx index a7ca9573b9da965..5fc5a22ff7c8e7f 100644 --- a/src/content/docs/d1/worker-api/return-object.mdx +++ b/src/content/docs/d1/worker-api/return-object.mdx @@ -37,6 +37,7 @@ The methods [`D1PreparedStatement::run`](/d1/worker-api/prepared-statements/#run size_after: number, // the size of the database after the query is successfully applied rows_read: number, // the number of rows read (scanned) by this query rows_written: number // the number of rows written by this query + total_attempts: number //the number of total attempts to successfully execute the query, including retries } results: array | null, // [] if empty, or null if it does not apply }