[cloud_firestore]: androidx.datastore.preferences OkioStorage InvalidProtocolBufferException

[Phillip-Schackier]

Is there an existing issue for this?

• I have searched the existing issues.

Which plugins are affected?

Core

Which platforms are affected?

Android

Description

We're experiencing crashes stemming from firestore internals in our production environment of our Flutter application.

Firebase Crashlytics reports crashes resulting from the androidx.datastore component, since we've upgraded to firebase_core 6.0.0.

See the logs for details.

This seems to be related to
firebase/firebase-android-sdk#7286
as well as these androidx.datastore issues:
https://issuetracker.google.com/issues/377923485
https://issuetracker.google.com/issues/346197747

We've attempted to downgrade, but have been unsuccessful so far.

The issue seems to have been. fixed with new versions of androidx.datastore, can you provide a roadmap when the Flutter Firestore packages will be upgraded to versions pulling in a new enough version of androidx.datastore that does not result in runtime crashes?

Reproducing the issue

Upgrading to the new firestore flutter plugins causes the runtime crash on some number of devices (a significant minority), we are not able to reproduce the issue in development currently.

Firebase Core version

6.0.0

Flutter Version

3.32.8

Relevant Log Output

Fatal Exception: java.lang.RuntimeException
Internal error in Cloud Firestore (26.0.0).
com.google.firebase.firestore.util.AsyncQueue.lambda$panic$3 (AsyncQueue.java:546)
android.os.Handler.handleCallback (Handler.java:938)
android.os.Handler.dispatchMessage (Handler.java:99)
android.os.Looper.loop (Looper.java:223)
android.app.ActivityThread.main (ActivityThread.java:7705)
java.lang.reflect.Method.invoke (Method.java)
com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:592)
com.android.internal.os.ZygoteInit.main (ZygoteInit.java:952)
Caused by B0.c
Unable to parse preferences proto.
androidx.datastore.core.CorruptionException.<init> (CorruptionException.java:25)
androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom (PreferencesMapCompat.java:34)
androidx.datastore.preferences.core.PreferencesSerializer.readFrom (PreferencesSerializer.jvm.kt:46)
androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl (OkioStorage.kt:180)
androidx.datastore.core.okio.OkioReadScope.readData (OkioStorage.kt:1)
androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend (StorageConnection.kt:74)
androidx.datastore.core.StorageConnectionKt$readData$2.invoke (StorageConnection.kt:25)
androidx.datastore.core.StorageConnectionKt$readData$2.invoke (StorageConnection.kt:25)
androidx.datastore.core.okio.OkioStorageConnection.readScope (OkioStorageConnection.java:113)
androidx.datastore.core.StorageConnectionKt.readData (StorageConnection.kt:74)
androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault (DataStoreImpl.kt:323)
androidx.datastore.core.DataStoreImpl.readDataOrHandleCorruption (DataStoreImpl.kt:365)
androidx.datastore.core.DataStoreImpl.access$readDataOrHandleCorruption (DataStoreImpl.kt:48)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend (DataStoreImpl.kt:437)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke (DataStoreImpl.kt:11)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke (DataStoreImpl.kt:11)
androidx.datastore.core.SingleProcessCoordinator.lock (SingleProcessCoordinator.kt:41)
androidx.datastore.core.DataStoreImpl$InitDataStore.doRun (DataStoreImpl.kt:434)
androidx.datastore.core.RunOnce.runIfNeeded (RunOnce.java:497)
androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure (DataStoreImpl.kt:266)
androidx.datastore.core.DataStoreImpl.access$readAndInitOrPropagateAndThrowFailure (DataStoreImpl.kt:48)
androidx.datastore.core.DataStoreImpl$readState$2.invokeSuspend (DataStoreImpl.kt:218)
kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:33)
kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:101)
kotlinx.coroutines.internal.LimitedDispatcher$Worker.run (LimitedDispatcher.java:113)
kotlinx.coroutines.scheduling.TaskImpl.run (Tasks.kt:89)
kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely (CoroutineScheduler.java:589)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask (CoroutineScheduler.kt:823)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker (CoroutineScheduler.kt:720)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run (CoroutineScheduler.kt:707)
Caused by androidx.datastore.preferences.protobuf.A
Protocol message contained an invalid tag (zero).
androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.<init> (InvalidProtocolBufferException.java:24)
androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.invalidTag (InvalidProtocolBufferException.java:110)
androidx.datastore.preferences.protobuf.CodedInputStream$StreamDecoder.readTag (CodedInputStream.java:2074)
androidx.datastore.preferences.protobuf.CodedInputStreamReader.getFieldNumber (CodedInputStreamReader.java:59)
androidx.datastore.preferences.protobuf.MessageSchema.mergeFromHelper (MessageSchema.java:2973)
androidx.datastore.preferences.protobuf.MessageSchema.mergeFrom (MessageSchema.java:2955)
androidx.datastore.preferences.protobuf.GeneratedMessageLite.parsePartialFrom (GeneratedMessageLite.java:1599)
androidx.datastore.preferences.protobuf.GeneratedMessageLite.parseFrom (GeneratedMessageLite.java:1742)
androidx.datastore.preferences.PreferencesProto$PreferenceMap.parseFrom (PreferencesProto.java:198)
androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom (PreferencesMapCompat.java:32)
androidx.datastore.preferences.core.PreferencesSerializer.readFrom (PreferencesSerializer.jvm.kt:46)
androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl (OkioStorage.kt:180)
androidx.datastore.core.okio.OkioReadScope.readData (OkioStorage.kt:1)
androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend (StorageConnection.kt:74)
androidx.datastore.core.StorageConnectionKt$readData$2.invoke (StorageConnection.kt:25)
androidx.datastore.core.StorageConnectionKt$readData$2.invoke (StorageConnection.kt:25)
androidx.datastore.core.okio.OkioStorageConnection.readScope (OkioStorageConnection.java:113)
androidx.datastore.core.StorageConnectionKt.readData (StorageConnection.kt:74)
androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault (DataStoreImpl.kt:323)
androidx.datastore.core.DataStoreImpl.readDataOrHandleCorruption (DataStoreImpl.kt:365)
androidx.datastore.core.DataStoreImpl.access$readDataOrHandleCorruption (DataStoreImpl.kt:48)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend (DataStoreImpl.kt:437)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke (DataStoreImpl.kt:11)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke (DataStoreImpl.kt:11)
androidx.datastore.core.SingleProcessCoordinator.lock (SingleProcessCoordinator.kt:41)
androidx.datastore.core.DataStoreImpl$InitDataStore.doRun (DataStoreImpl.kt:434)
androidx.datastore.core.RunOnce.runIfNeeded (RunOnce.java:497)
androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure (DataStoreImpl.kt:266)
androidx.datastore.core.DataStoreImpl.access$readAndInitOrPropagateAndThrowFailure (DataStoreImpl.kt:48)
androidx.datastore.core.DataStoreImpl$readState$2.invokeSuspend (DataStoreImpl.kt:218)
kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:33)
kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:101)
kotlinx.coroutines.internal.LimitedDispatcher$Worker.run (LimitedDispatcher.java:113)
kotlinx.coroutines.scheduling.TaskImpl.run (Tasks.kt:89)
kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely (CoroutineScheduler.java:589)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask (CoroutineScheduler.kt:823)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker (CoroutineScheduler.kt:720)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run (CoroutineScheduler.kt:707)

Flutter dependencies

cloud_firestore: 6.0.0
fake_cloud_firestore: 4.0.0
firebase_analytics: 12.0.0
firebase_auth: 6.0.0
firebase_core: 4.0.0
firebase_crashlytics: 5.0.0
firebase_remote_config: 6.0.0
firebaseapis: 0.2.0

com.google.firebase:firebase-bom:33.4.0

Additional context and comments

We've tried to upgrade the firebase-bom to 34.0.0, but that made no difference.

[SelaseKay]

Hi @Phillip-Schackier, thanks for the report. As you rightly said, this issue stems from the native SDK and would require a fix on their end. I'll keep this opened for tracking purposes.

[Phillip-Schackier]

Hi @SelaseKay thank you for your rapid response.
As far as I understand the issue the native SDK team has already fixed / attempted to mitigate the data corruption: https://developer.android.com/jetpack/androidx/releases/datastore#1.1.5
flutterfire has recently updated Android SDK to version 34.0.0, I suspect that version also pulls in the bom version 34.0.0, which then introduces the faulty version of androidx.datastore.preferences.
If a newer version of datastore could be used in flutterfire that may solve the issue, or am I misunderstanding the issue and there isn't anything that can be done on the Flutter side of things?

[Phillip-Schackier]

As far as I can tell the fix to upgrade androidx.datastore is already in the firebase-android-sdk:

firebase/firebase-android-sdk@296ab29

Is there some kind of roadmap that Google can provide as to when the fix will be available so that crashes will start to disappear?

[Phillip-Schackier]

Hi @SelaseKay,

as of now, this is still not resolved in production, and the crashes (due to androidx.datastore.preferences / OkioStorage / InvalidProtocolBufferException) continue to occur.

We really need:
A timeline for when a fix is expected (e.g. which upcoming version of cloud_firestore or firebase_core will include the required update so that androidx.datastore is at a version that avoids this bug).

Any workarounds or guidance in the meantime, even partial ones. For example:
Maybe a patch or hotfix we can apply locally until the upstream change is merged.

We appreciate all the work already, but this is continuously impacting our users in production. Any updates, even approximate, are very welcome.

Thank you!

[eli1stark]

We encountered the same issue in production; it crashes certain devices due to the error that @Phillip-Schackier described.

[Phillip-Schackier]

Hey, just an update.
This issue seems to be resolved since FirebaseSDKVersion 34.2.0.
I've noticed that this repository has been updated to FirebaseSDKVersion 34.4.0 in https://github.com/firebase/flutterfire/blob/main/packages/firebase_core/firebase_core/android/gradle.properties
We're currently still using a fork of this repository using FirebaseSDKVersion 34.2.0, internally.
Reinstalling the app seems to result in no further crashes of this kind.