Redux-Persist v5 migrate from one storage system to another

[alanlanglois]

I'm working on a project where I need to migrate my data initially stored using AsyncStorage to another one: redux-persist-fs-storage

Based on this issue: #679 I've manage to get the data stored in AsyncStorage this way:

let persistor = persistStore(store, persistConfig, async () => {
  try {
    console.log("GET ASYNC STATE")
    const asyncState = await getStoredState({
      key: 'root',
      storage: AsyncStorage,
     })
     console.log("ASYNC >>>> " + asyncState )
       if (asyncState) {
     }
   } catch (error) {
    console.warn('getStoredState ASYNC error', error)
  }
});

I'm looking a way to inject these data (asyncState) into the new fileSystem store.

In v4 we could achieve that using a rehydrate method on the persistor.

fsPersistor.rehydrate(asyncState, { serial: false })
It's not a migration from v4 to v5, just from a store to another using v5.

Can't find a way to do it in v5. An input on how to do it would be great :)

Cheers

[hutchy2570]

Hi,

I've managed to get this to work using v5's migrate feature.

isStateEmpty is a function which you craft to determine whether the state is empty for your use case.

const migrate = async state => {
  // Migrate from async storage to fs https://github.com/robwalkerco/redux-persist-filesystem-storage#migration-from-previous-storage
  __DEV__ && console.log('Attempting migration');
  if (isStateEmpty(state)) {
    // if state from fs storage is empty try to read state from previous storage
    __DEV__ && console.log('FS state empty');
    try {
      const asyncState = await getStoredState({
        key: 'root',
        storage: AsyncStorage,
      });
      if (!isStateEmpty(asyncState)) {
        __DEV__ && console.log('Async state not empty. Attempting migration.');
        // if data exists in `AsyncStorage` - rehydrate fs persistor with it
        return asyncState;
      }
    } catch (getStateError) {
      __DEV__ && console.warn('getStoredState error', getStateError);
    }
  }
  __DEV__ && console.log('FS state not empty');
  return state;
};

You then just include this function as the migrate property of the PersistConfig passed to persistReducer

[forster-thomas]

@hutchy2570 where I put this code?

[piotr-cz]

This is my take on migrating from custom (legacy) storage

// configureStore.js
import migrations, { createMigrate } from './migrations'

export default function () {
  const persistConfig = {
    migrate: createMigrate(migrations),
    // rest of config 
  }

  // ...
}

// migrations/index.js
import { createMigrate as createReduxMigrate } from 'redux-persist'
import migrateLegacyState from './migrateLegacyState'

/**
 * Legacy migration wrapper
 */
export function createMigrate(migrations, config) {
  return (state, currentVersion) => {
    const reduxMigrate = createReduxMigrate(migrations, config)

    // If state from current storage is empty try migrate state from legacy storage
    // This also triggers versioned migrations
    if (!state) {
      try {
        console.log('redux-persist/legacy: no inbound state, running legacy state migration')

        state = migrateLegacyState(state)
      } catch (migrateLegacyStateError) {
        console.error('redux-persist/legacy: migration error', migrateLegacyStateError)
      }
    }

    return reduxMigrate(state, currentVersion)
  }
}

/**
 *  Versioned migrations
 */
export default {
}

[rt2zz]

its also possible to do this with a custom getStoredState implementation. Thats what we did for v4 to v5 migration: https://github.com/rt2zz/redux-persist/blob/master/docs/MigrationGuide-v5.md#experimental-v4-to-v5-state-migration

[piotr-cz]

Good point, @rt2zz
However I'd like to think about previous storage state as first step in migration (version -2) and run any usual migrations after state has been moved to new storage.

Looking at the persistReducer and getStoredState code it seems that the latter function is run at every hydration (after migrations) so I'd have to update logic in my migrateLegacyState function to migrate old storage structure to match last migration result

[raphaelrk]

My solution for migrating from one storage system to another -- an intermediate storage system:

(Edit: this works but still didn't solve the problem I was really having, which is that my app is crashing with out of memory issues. Logging "Out of memory" from the line writePromise = storage.setItem(storageKey, serialize(stagedState)).catch(onWriteFail); in redux-persist/lib/createPersistoid.js)

import { createStore, applyMiddleware } from 'redux';
import { createMigrate, persistStore, persistReducer } from 'redux-persist';
import storage from 'redux-persist/lib/storage';
import ExpoFileSystemStorage from "redux-persist-expo-filesystem"
import { PersistGate } from 'redux-persist/lib/integration/react';

// migrating from AsyncStorage to Expo FileSystem storage
// put in on May 15 2019, can probably remove in a couple months but don't have to
const MigratedStorage = {
  async getItem(key) {
    try {
      const res = await ExpoFileSystemStorage.getItem(key);
      if (res) {
        // Using new storage system
        return res;
      }
    } catch (e) {}

    // Using old storage system, should only happen once
    const res = await storage.getItem(key);
    storage.setItem(key, ''); // clear old storage
    return res;
  },
  setItem(key, value) {
    return ExpoFileSystemStorage.setItem(key, value);
  },
  removeItem(key) {
    return ExpoFileSystemStorage.removeItem(key);
  }
};

const migrations = {
  ...
};
const persistConfig = {
  key: 'root',
  storage: MigratedStorage,
  migrate: createMigrate(migrations, { debug: false }),
};
const reducer = persistReducer(persistConfig, rootReducer);
const store = createStore(reducer, applyMiddleware(thunkMiddleware));
const persistor = persistStore(store);

[hiennguyen92]

Hi,

I've managed to get this to work using v5's migrate feature.

isStateEmpty is a function which you craft to determine whether the state is empty for your use case.

const migrate = async state => {
  // Migrate from async storage to fs https://github.com/robwalkerco/redux-persist-filesystem-storage#migration-from-previous-storage
  __DEV__ && console.log('Attempting migration');
  if (isStateEmpty(state)) {
    // if state from fs storage is empty try to read state from previous storage
    __DEV__ && console.log('FS state empty');
    try {
      const asyncState = await getStoredState({
        key: 'root',
        storage: AsyncStorage,
      });
      if (!isStateEmpty(asyncState)) {
        __DEV__ && console.log('Async state not empty. Attempting migration.');
        // if data exists in `AsyncStorage` - rehydrate fs persistor with it
        return asyncState;
      }
    } catch (getStateError) {
      __DEV__ && console.warn('getStoredState error', getStateError);
    }
  }
  __DEV__ && console.log('FS state not empty');
  return state;
};

You then just include this function as the migrate property of the PersistConfig passed to persistReducer

I have followed this tutorial and it works fine only after I close the app and reopen it. if you don't close the app or reload it, it still uses the old data. can you guide me how to solve it?

[st1ng]

Hi,
I've managed to get this to work using v5's migrate feature.
isStateEmpty is a function which you craft to determine whether the state is empty for your use case.

const migrate = async state => {
  // Migrate from async storage to fs https://github.com/robwalkerco/redux-persist-filesystem-storage#migration-from-previous-storage
  __DEV__ && console.log('Attempting migration');
  if (isStateEmpty(state)) {
    // if state from fs storage is empty try to read state from previous storage
    __DEV__ && console.log('FS state empty');
    try {
      const asyncState = await getStoredState({
        key: 'root',
        storage: AsyncStorage,
      });
      if (!isStateEmpty(asyncState)) {
        __DEV__ && console.log('Async state not empty. Attempting migration.');
        // if data exists in `AsyncStorage` - rehydrate fs persistor with it
        return asyncState;
      }
    } catch (getStateError) {
      __DEV__ && console.warn('getStoredState error', getStateError);
    }
  }
  __DEV__ && console.log('FS state not empty');
  return state;
};

You then just include this function as the migrate property of the PersistConfig passed to persistReducer

I have followed this tutorial and it works fine only after I close the app and reopen it. if you don't close the app or reload it, it still uses the old data. can you guide me how to solve it?

This code do not work, because when you migrate to new storage, this storage is empty and "migrate" is not called for that storage on first launch.
I was able to successfully migrate to filesystem-storage by using simple code and replacing "getStoredState" in config (which is undocumented, but mentioned in V4>V5 migration)

import { getStoredState } from 'redux-persist';
import AsyncStorage from '@react-native-community/async-storage';

export default async (config) => {
    return getStoredState(config).catch(err => {
        return getStoredState({...config, storage: AsyncStorage});
    });
}

Then in config use

const persistorConfig = {
  key: 'reducer',
  storage: FilesystemStorage,
};
persistorConfig.getStoredState = newGetStoredState

This method also let you migrate any nested persisted reducer separately

[manjuy124]

We were having similar problem. We didn't change the storage location, but it stopped picking the data on launch once we upgraded our Async storage package. Following code is working for now(I feel it's still hacky so need good amount of testing).

import storage from '@react-native-async-storage/async-storage'

import { persistStore, persistReducer, createMigrate, getStoredState } from 'redux-persist' 
const persistConfig = {
  key: 'root',
  storage,
  whitelist: persisted_reducers,
  version: 0,
  migrate: createMigrate(persistMigrations, { debug: false }),
}

persistConfig.getStoredState = async config => {
  try {
    const isDataMigrationCompleted = await storage.getItem(‘data-migration-completed')
    let rehydratedState = {}
    if (!isDataMigrationCompleted) {
      const oldAsyncStorageInfo = await FileSystem.getInfoAsync(
        `${FileSystem.documentDirectory}RCTAsyncLocalStorage/manifest.json`,
      )
      if (oldAsyncStorageInfo && oldAsyncStorageInfo.exists) {
        const oldAsyncStorageContents = await FileSystem.readAsStringAsync(
          `${FileSystem.documentDirectory}RCTAsyncLocalStorage/manifest.json`,
        )
        const oldAsyncStorageObj = JSON.parse(oldAsyncStorageContents)
        if (oldAsyncStorageObj) {
          const persistKey = ‘OUR_PERSIST_KEY’ // in our case its persist:root
          if (oldAsyncStorageObj[persistKey]) {
            await storage.setItem(persistKey, oldAsyncStorageObj[persistKey])
            rehydratedState = await getStoredState(config)
            await storage.setItem('data-migration-completed', 'true')
            return rehydratedState
          }
          if (oldAsyncStorageObj[persistKey'] === null) {
            const keyHash = ‘—‘ // md5 hash of key "persistKey"
            const persistedData = await FileSystem.readAsStringAsync(
              `${FileSystem.documentDirectory}RCTAsyncLocalStorage/${keyHash}`,
            )
            await storage.setItem(persistKey, persistedData)
            rehydratedState = await getStoredState(config)
            await storage.setItem('data-migration-completed', 'true')
            return rehydratedState
          }
        }
      } else {
        rehydratedState = await getStoredState(config)
        await storage.setItem('data-migration-completed', 'true')
        return rehydratedState
      }
    } else {
      rehydratedState = await getStoredState(config)
      return rehydratedState
    }
  } catch (error) {
    // catching error and sending it to sentry
    const rehydratedState = await getStoredState(config)
    return rehydratedState
  }
}

[smontlouis]

@st1ng your solution is the simplest one ! Yet the BEST ! Thanks !!

[speed1992]

Successfully migrated from localstorage to indexedDB!

import { configureStore } from '@reduxjs/toolkit'
import { getPersistConfig } from 'redux-deep-persist'
import { persistReducer, persistStore } from 'redux-persist'
import DBstorage from 'redux-persist-indexeddb-storage'
import getStoredState from 'redux-persist/es/getStoredState'
import storage from 'redux-persist/lib/storage'
import philosophersDataReducer from '../components/home-page/homePageRedux/homePageRedux'

// old storage method
const persistConfig = getPersistConfig({
    key: 'root',
    storage,
    blacklist: ['currentData', 'originalData', 'options', 'quotesLoaded'],
    rootReducer: philosophersDataReducer,
})

// new storage method
const newPersistConfig = getPersistConfig({
    key: 'root',
    storage: DBstorage('myDB'),
    blacklist: ['currentData', 'originalData', 'options', 'quotesLoaded'],
    rootReducer: philosophersDataReducer,
    migrate: async (state) => {
        if (state === undefined) {
            const asyncState = await getStoredState(persistConfig)
            return asyncState
        } else return state
    },
})

const philosophersDataSlice = persistReducer(newPersistConfig, philosophersDataReducer)

export const store = configureStore({
    reducer: {
        philosophersData: philosophersDataSlice,
    },
    middleware: (getDefaultMiddleware) =>
        getDefaultMiddleware({
            serializableCheck: false,
        }),
    devTools: process.env.NODE_ENV !== 'production',
})

export const persistor = persistStore(store)