-
Notifications
You must be signed in to change notification settings - Fork 94
fix: add IDB healing mechanism for backing store corruption and connection lost errors #780
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
6569dc7
a2c8ea7
213480b
1bb21b2
7633f32
1f14374
8bc7cd4
fccdb4a
3fbb657
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,12 +2,48 @@ import * as IDB from 'idb-keyval'; | |
| import type {UseStore} from 'idb-keyval'; | ||
| import * as Logger from '../../../Logger'; | ||
|
|
||
| const HEAL_ATTEMPTS_MAX = 3; | ||
|
|
||
| /** | ||
| * Detects the Chromium-specific IDB backing store corruption error. | ||
| * Fires when LevelDB files backing IndexedDB are corrupted and Chrome's | ||
| * internal recovery (RepairDB -> delete -> recreate) also fails. | ||
| */ | ||
| function isBackingStoreError(error: unknown): boolean { | ||
| return error instanceof Error && error.message.includes('Internal error opening backing store'); | ||
| } | ||
|
|
||
| /** | ||
| * Detects Safari/WebKit IDB connection termination errors. | ||
| * Fires when Safari kills the IDB server process for backgrounded tabs. | ||
| * WebKit bugs: https://bugs.webkit.org/show_bug.cgi?id=197050, https://bugs.webkit.org/show_bug.cgi?id=201483 | ||
| */ | ||
| function isConnectionLostError(error: unknown): boolean { | ||
| if (!(error instanceof Error)) return false; | ||
| const msg = error.message.toLowerCase(); | ||
| return msg.includes('connection to indexed database server lost') || msg.includes('connection is closing'); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Using
It's intended, right?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes, it's intended - for all unexpected errors from this family |
||
| } | ||
|
|
||
| function isInvalidStateError(error: unknown): boolean { | ||
| return error instanceof Error && error.name === 'InvalidStateError'; | ||
| } | ||
|
Comment on lines
+27
to
+29
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Potential edge case: The old code checked: if (error instanceof DOMException && error.name === 'InvalidStateError') {The new code uses: function isInvalidStateError(error: unknown): boolean {
return error instanceof Error && error.name === 'InvalidStateError';
}This is more permissive ( Suggested fix - tighten the check: function isInvalidStateError(error: unknown): boolean {
return (error instanceof Error || error instanceof DOMException) && (error as Error).name === 'InvalidStateError';
} |
||
|
|
||
| /** Errors that trigger a budgeted heal-and-retry in store(). */ | ||
| function isBudgetedHealError(error: unknown): boolean { | ||
| return isBackingStoreError(error) || isConnectionLostError(error); | ||
| } | ||
|
|
||
| function getBudgetedHealErrorLabel(error: unknown): 'backing store' | 'connection lost' { | ||
| return isBackingStoreError(error) ? 'backing store' : 'connection lost'; | ||
| } | ||
|
Comment on lines
+36
to
+38
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Issue: Similar to here, this assumes that if it's not a backing store error, it MUST be a connection lost error. If a third budgeted heal error category is ever added to Suggested fix: function getBudgetedHealErrorLabel(error: unknown): 'backing store' | 'connection lost' | null {
if (isBackingStoreError(error)) return 'backing store';
if (isConnectionLostError(error)) return 'connection lost';
return null;
} And update the caller: const label = getBudgetedHealErrorLabel(error);
if (!label) {
throw new Error('Unexpected: isBudgetedHealError matched but no label found');
}
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree but we shouldn't throw an Error just because the label wasn't found, let's just log something and/or skip the logic if necessary |
||
|
|
||
| // This is a copy of the createStore function from idb-keyval, we need a custom implementation | ||
| // because we need to create the database manually in order to ensure that the store exists before we use it. | ||
| // If the store does not exist, idb-keyval will throw an error | ||
| // source: https://github.com/jakearchibald/idb-keyval/blob/9d19315b4a83897df1e0193dccdc29f78466a0f3/src/index.ts#L12 | ||
| function createStore(dbName: string, storeName: string): UseStore { | ||
| let dbp: Promise<IDBDatabase> | undefined; | ||
| let healAttemptsRemaining = HEAL_ATTEMPTS_MAX; | ||
|
|
||
| const attachHandlers = (db: IDBDatabase) => { | ||
| // Browsers may close idle IDB connections at any time, especially Safari. | ||
|
|
@@ -30,18 +66,27 @@ function createStore(dbName: string, storeName: string): UseStore { | |
| }; | ||
| }; | ||
|
|
||
| // Cache the open promise and attach handlers + rejection cleanup. | ||
| // On rejection, clears dbp so the next operation retries with a fresh indexedDB.open() | ||
| // instead of returning the same rejected promise. | ||
| // Guard: only clear if dbp hasn't been replaced by a concurrent heal/retry. | ||
| function cacheOpenPromise(openPromise: Promise<IDBDatabase>) { | ||
| dbp = openPromise; | ||
| const currentPromise = openPromise; | ||
| openPromise.then(attachHandlers, () => { | ||
| if (dbp !== currentPromise) { | ||
| return; | ||
| } | ||
| dbp = undefined; | ||
| }); | ||
| return openPromise; | ||
| } | ||
|
|
||
| const getDB = () => { | ||
| if (dbp) return dbp; | ||
| const request = indexedDB.open(dbName); | ||
| request.onupgradeneeded = () => request.result.createObjectStore(storeName); | ||
| dbp = IDB.promisifyRequest(request); | ||
|
|
||
| dbp.then( | ||
| attachHandlers, | ||
| // eslint-disable-next-line @typescript-eslint/no-empty-function | ||
| () => {}, | ||
| ); | ||
| return dbp; | ||
| return cacheOpenPromise(IDB.promisifyRequest(request)); | ||
| }; | ||
|
|
||
| // Ensures the store exists in the DB. If missing, bumps the version to trigger | ||
|
|
@@ -66,10 +111,7 @@ function createStore(dbName: string, storeName: string): UseStore { | |
| updatedDatabase.createObjectStore(storeName); | ||
| }; | ||
|
|
||
| dbp = IDB.promisifyRequest(request); | ||
| // eslint-disable-next-line @typescript-eslint/no-empty-function | ||
| dbp.then(attachHandlers, () => {}); | ||
| return dbp; | ||
| return cacheOpenPromise(IDB.promisifyRequest(request)); | ||
| }; | ||
|
|
||
| function executeTransaction<T>(txMode: IDBTransactionMode, callback: (store: IDBObjectStore) => T | PromiseLike<T>): Promise<T> { | ||
|
|
@@ -78,23 +120,64 @@ function createStore(dbName: string, storeName: string): UseStore { | |
| .then((db) => callback(db.transaction(storeName, txMode).objectStore(storeName))); | ||
| } | ||
|
|
||
| // If the connection was closed between getDB() resolving and db.transaction() executing, | ||
| // the transaction throws InvalidStateError. We catch it and retry once with a fresh connection. | ||
| function resetHealBudget<T>(result: T): T { | ||
| healAttemptsRemaining = HEAL_ATTEMPTS_MAX; | ||
| return result; | ||
| } | ||
|
|
||
| // Handles three recoverable error classes: | ||
| // 1. InvalidStateError — connection closed between getDB() resolving and db.transaction(). | ||
| // Retry once with a fresh connection. No budget limit (transient, always worth one reopen). | ||
| // 2. Backing store corruption (Chromium UnknownError) — drop cached connection and reopen. | ||
| // 3. Connection lost (Safari UnknownError) — IDB server terminated for backgrounded tabs. | ||
| // Both 2 and 3 share a heal budget (3 attempts, reset on success). | ||
| // Mirrors Dexie's PR1398_maxLoop pattern: https://github.com/dexie/Dexie.js/blob/master/src/functions/temp-transaction.ts | ||
| // Note: concurrent store() calls share the budget. Under overlapping failures each caller | ||
| // decrements independently, so the budget may drain faster than one-per-incident. This is | ||
| // acceptable — same as Dexie's approach — and the budget resets on any success. | ||
| return (txMode, callback) => | ||
| executeTransaction(txMode, callback).catch((error) => { | ||
| if (error instanceof DOMException && error.name === 'InvalidStateError') { | ||
| Logger.logAlert('IDB InvalidStateError, retrying with fresh connection', { | ||
| dbName, | ||
| storeName, | ||
| txMode, | ||
| errorMessage: error.message, | ||
| }); | ||
| dbp = undefined; | ||
| // Retry only once — this call is not wrapped, so if it also fails the error propagates normally. | ||
| return executeTransaction(txMode, callback); | ||
| } | ||
| throw error; | ||
| }); | ||
| executeTransaction(txMode, callback) | ||
| .then(resetHealBudget) | ||
| .catch((error) => { | ||
| if (isInvalidStateError(error)) { | ||
| Logger.logInfo('IDB InvalidStateError — dropping cached connection and retrying', { | ||
| dbName, | ||
| storeName, | ||
| txMode, | ||
| errorMessage: error instanceof Error ? error.message : String(error), | ||
| }); | ||
| dbp = undefined; | ||
| return executeTransaction(txMode, callback).then(resetHealBudget); | ||
| } | ||
|
|
||
| if (isBudgetedHealError(error) && healAttemptsRemaining > 0) { | ||
| healAttemptsRemaining--; | ||
| const label = getBudgetedHealErrorLabel(error); | ||
|
Comment on lines
+153
to
+155
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a reason not to keep the counter local and per-transaction? It seems we'll have one variable for the whole app? |
||
| Logger.logInfo(`IDB heal: ${label} error detected — dropping cached connection and reopening (${healAttemptsRemaining} attempts left)`, { | ||
| dbName, | ||
| storeName, | ||
| }); | ||
| dbp = undefined; | ||
| return executeTransaction(txMode, callback).then((result) => { | ||
| Logger.logInfo(`IDB heal: successfully recovered after ${label} error`, {dbName, storeName}); | ||
| return resetHealBudget(result); | ||
| }); | ||
| } | ||
|
|
||
| if (isBudgetedHealError(error)) { | ||
| Logger.logAlert(`IDB heal: ${getBudgetedHealErrorLabel(error)} error — heal budget exhausted, giving up`, { | ||
| dbName, | ||
| storeName, | ||
| }); | ||
| } else { | ||
| Logger.logAlert('IDB error is not recoverable, giving up', { | ||
| dbName, | ||
| storeName, | ||
| errorMessage: error instanceof Error ? error.message : String(error), | ||
| }); | ||
| } | ||
| throw error; | ||
| }); | ||
| } | ||
|
|
||
| export default createStore; | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟢 Code Quality: Good
The error detection functions are pure, focused, and readable.
🟡 Issue - Fragile string matching: The heal logic relies on exact error message substrings. Consider adding a fallback or a more robust pattern.
Why it matters: If a browser vendor changes the error wording slightly (which happens), the 884K/month errors will continue to crash the app despite having heal logic that should catch them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can end up include other class of errors like
QuotaExceededError: Encountered full disk while opening backing store for indexedDB.open..I agree that Chrome could change the errors some day but we can't really protect ourselves against this, it could change to something completely different from
backing storeand the suggestion would still fail.