Documentation Improvement: Warning about Promise serialization issues in IndexedDB persistence

[NathanVG]

Documentation Improvement: Warning about Promise serialization issues in IndexedDB persistence

Description

The current documentation for persistQueryClient with IndexedDB doesn't mention a critical issue that developers may encounter: Promise objects in loading queries cannot be serialized to IndexedDB, leading to DataCloneError exceptions.

This issue commonly occurs on initial application loads in new environments (like incognito mode) where all queries start in a loading state with no existing cache. The current documentation example doesn't address this scenario, potentially leading developers to spend significant time debugging these errors.

Current Documentation Issue

In the persistQueryClient documentation, the IndexedDB example doesn't include any handling for queries in a loading state or containing Promise objects:

import { createSyncStoragePersister } from '@tanstack/query-sync-storage-persister'
import { QueryClient } from '@tanstack/react-query'
import { persistQueryClient } from '@tanstack/react-query-persist-client'
import { get, set, del } from 'idb-keyval'

const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      cacheTime: 1000 * 60 * 60 * 24, // 24 hours
    },
  },
})

const persister = createSyncStoragePersister({
  storage: {
    getItem: async (key) => {
      const value = await get(key)
      return value
    },
    setItem: async (key, value) => {
      await set(key, value)
    },
    removeItem: async (key) => {
      await del(key)
    },
  },
})

persistQueryClient({
  queryClient,
  persister,
})

Suggested Improvements

1. Add a warning about Promise serialization issues with loading queries
2. Enhance the example to include proper query filtering via shouldDehydrateQuery
3. Add error handling for serialization failures
4. Provide a more comprehensive example that addresses these issues

Proposed Documentation Addition

// New section to add after the basic example:

/*
Important Note: IndexedDB cannot serialize Promise objects that exist in loading queries. 
This commonly causes DataCloneError exceptions on initial application loads when all queries 
start in a loading state. To prevent this, use the shouldDehydrateQuery option to filter out 
loading queries before persistence.
*/

// Enhanced example:
persistQueryClient({
  queryClient,
  persister,
  dehydrateOptions: {
    shouldDehydrateQuery: (query) => {
      // Only persist queries that have completed loading and don't have errors
      return query.state.status !== 'loading' && !query.state.error;
    },
  },
  // Add error handling for serialization failures
  onError: (error) => {
    console.error('Error persisting query cache:', error);
  }
})

Additional Recommendations

Consider adding a section on best practices for persistence that includes:

1. Filtering strategies for queries that should be persisted
2. Cache invalidation strategies
3. Handling size limitations of storage
4. Versioning the cache for app updates

Impact

This improvement would help developers avoid a common pitfall when implementing persistence with IndexedDB, reducing debugging time and improving the developer experience with Tanstack Query.

[TkDodo]

shouldDehydrateQuery is, per default, implemented in a way that only queries in ’success’ state are persisted, so unless you explicitly change that, you shouldn’t run into this problem.

query/packages/query-core/src/hydration.ts

Lines 108 to 110 in 6ca0eb7

	export function defaultShouldDehydrateQuery(query: Query) {
	return query.state.status === 'success'
	}

[NathanVG]

Yes this is what I understood from the docs as well, but we implemented an indexeddb persister, and shouldDehydrateQuery as described in the docs, but we were running into this issue as described above. Did we somehow opt out of this default?

<PersistQueryClientProvider
      client={queryClient}
      persistOptions={{
        persister,
        dehydrateOptions: {
          shouldDehydrateQuery: (query) => !!query.meta?.persist,
        },
      }}
    >
      {children}
      <ReactQueryDevtools buttonPosition="bottom-left" />
      {shouldShowProdDevtools && (
        <Suspense fallback={null}>
          <ReactQueryDevtoolsProduction buttonPosition="bottom-left" />
        </Suspense>
      )}
    </PersistQueryClientProvider>

Persister with a fallback to localstorage

import type { PersistedClient } from '@tanstack/react-query-persist-client';
import type { IDBPDatabase } from 'idb';
import { openDB } from 'idb';

/**
 * Checks if the provided data is a Blob or ArrayBuffer (binary data).
 * @param data - The data to check.
 * @returns True if the data is binary; otherwise, false.
 */
const isBinaryData = (data: unknown): boolean => {
  return data instanceof Blob || data instanceof ArrayBuffer;
};

/**
 * Checks if IndexedDB is available in the current environment.
 * @returns True if IndexedDB is available; otherwise, false.
 */
const isIndexedDBAvailable = (): boolean => {
  return !!window.indexedDB;
};

const TANSTACK_OBJECTSTORE_KEY = 'tanstack';

/**
 * Creates a persister for managing client state in IndexedDB.
 * @param dbName - The name of the IndexedDB database (default: 'tanstack-query-cache').
 * @param storeName - The name of the object store in IndexedDB (default: 'queries').
 * @returns An object with methods to persist, restore, and remove client state.
 */
export const createIDBPersister = (
  dbName = 'tanstack-query-cache', // Default database name
  storeName = 'queries' // Default store name
) => {
  const dbConfig = {
    upgrade(database: IDBPDatabase) {
      // Create the object store if it doesn't exist
      if (!database.objectStoreNames.contains(storeName)) {
        database.createObjectStore(storeName, { autoIncrement: false });
      }
    },
  };

  // Return an object with the methods to manage client states
  const persister = {
    persistClient: async (clientState: PersistedClient) => {
      if (!isIndexedDBAvailable()) {
        // If IndexedDB is not available, check for binary data before storing in LocalStorage
        if (isBinaryData(clientState)) {
          throw new Error('Cannot store binary data in LocalStorage. Use IndexedDB instead.');
        }
        localStorage.setItem(TANSTACK_OBJECTSTORE_KEY, JSON.stringify(clientState)); // Store in LocalStorage
        return; // Exit the function after storing in LocalStorage
      }

      const database = await openDB(dbName, 1, dbConfig);
      try {
        const tx = database.transaction(storeName, 'readwrite');
        const store = tx.objectStore(storeName);
        await store.put(clientState, TANSTACK_OBJECTSTORE_KEY); // Store the client state
        await tx.done; // Wait for the transaction to complete
      } finally {
        database.close(); // Close the database connection
      }
    },
    restoreClient: async () => {
      if (!isIndexedDBAvailable()) {
        const data = localStorage.getItem(TANSTACK_OBJECTSTORE_KEY);
        return data ? JSON.parse(data) : undefined; // Fallback to LocalStorage if IndexedDB is not available
      }
      const database = await openDB(dbName, 1, dbConfig);
      try {
        const tx = database.transaction(storeName, 'readonly');
        const store = tx.objectStore(storeName);
        return await store.get(TANSTACK_OBJECTSTORE_KEY); // Retrieve the client state
      } finally {
        database.close(); // Close the database connection
      }
    },
    removeClient: async () => {
      const database = await openDB(dbName, 1, dbConfig);
      try {
        const tx = database.transaction(storeName, 'readwrite');
        const store = tx.objectStore(storeName);
        await store.delete(TANSTACK_OBJECTSTORE_KEY); // Delete the client state
        await tx.done; // Wait for the transaction to complete
      } finally {
        database.close(); // Close the database connection
      }
    },
  };

  // Check for existing LocalStorage data and migrate it to IndexedDB
  const localStorageData = localStorage.getItem(TANSTACK_OBJECTSTORE_KEY);
  if (localStorageData) {
    const clientState = JSON.parse(localStorageData);
    const migrateLocalStorageToIndexedDB = async () => {
      if (isIndexedDBAvailable()) {
        await persister.persistClient(clientState); // Use the reference to persistClient
        localStorage.removeItem(TANSTACK_OBJECTSTORE_KEY); // Remove LocalStorage data after migration
      }
    };
    migrateLocalStorageToIndexedDB(); // Call the migration function
  }

  // Log a warning if IndexedDB is not available
  if (!isIndexedDBAvailable()) {
    console.warn('IndexedDB is not available. Falling back to LocalStorage for persistence.');
  }

  return persister; // Return the persister object
};

[TkDodo]

you overwrite the default behaviour, you need:

shouldDehydrateQuery: (query) => defaultShouldDehydrateQuery(query) && !!query.meta?.persist,

[NathanVG]

Ah okay, I see this is buried pretty deep in the dehydration article , I do think it would be a good idea to mention this again on the persistQueryClient page as a possible gotcha.