Seeing very high latency for local fetches from firestore on device db

[raghavvijind]

[READ] Step 1: Are you in the right place?

We are facing a very high latency upto 140ms to 200ms when trying to fetch firestore's local database. We understand that remote fetches can take a higher time but local fetches taking upto 150ms is not acceptable.

[REQUIRED] Step 2: Describe your environment

• Android Studio version: Android Studio Meerkat | 2024.3.1 Canary 6
• Firebase Component: Firestore
• Component version: 33.7.0

[REQUIRED] Step 3: Describe the problem

We are facing a very high latency upto 140ms to 200ms when trying to fetch firestore's local database. We understand that remote fetches can take a higher time but local fetches taking upto 150ms is not acceptable.

Steps to reproduce:

Any fetches from local database are latent, takes upto 150ms

Relevant Code:

private suspend fun executeDocumentTask(pageName: String?, fireStoreTask: Task, source: String, stylesList: String = EMPTY, tracer : Tracer? = null): DocumentSnapshot? = suspendCancellableCoroutine { coroutine ->
val startTimeMs = System.currentTimeMillis()
try {
tracer?.log("fireStoreManager.executeDocumentTask.addOnCompleteListener.start")
fireStoreTask.addOnCompleteListener {
tracer?.log("fireStoreManager.executeDocumentTask.inOnCompleteListener.result")
if (it.isSuccessful) {
application.trackInNewRelic(eventName = "firestore_fetch_success", attributes = mutableMapOf(
"reason" to "Exception - ${it.exception.toString()} || Reason - ${it.exception?.message.toString()}",
"source" to source,
"duration" to System.currentTimeMillis() - startTimeMs
))
if (it.result.exists() && "server".equals(source, true)) {
application.trackInNewRelic(eventName = "firestore_fetch_no_items", attributes = mutableMapOf(
"stylesNotFound" to stylesList,
"page" to (pageName ?: EMPTY)
))
}
if (coroutine.isActive) coroutine.resume(it.result)
} else {
application.trackInNewRelic(eventName = "firestore_fetch_failed", attributes = mutableMapOf(
"reason" to "Exception - ${it.exception.toString()} || Reason - ${it.exception?.message.toString()}",
"source" to source,
"duration" to System.currentTimeMillis() - startTimeMs
))
if (coroutine.isActive) coroutine.resume(null)
}
}
} catch (exception: Exception) {
application.trackInNewRelic(eventName = "firestore_fetch_failed", attributes = mutableMapOf(
"reason" to "Exception - ${exception.toString()} || Reason - ${exception.message.toString()}",
"source" to source,
"duration" to System.currentTimeMillis() - startTimeMs
))
if (coroutine.isActive) coroutine.resume(null)
}
}

val widgetPageDocumentFromCache = withContext(mainDispatcher) {
tracer?.log("getWidgetUrlFromPage.widgetPageDocumentFromCache.main.start")
val data = executeDocumentTask(pageName, documentRef.get(Source.CACHE), "cache", tracer =tracer)
tracer?.log("getWidgetUrlFromPage.widgetPageDocumentFromCache.main.end")
data
}

[cherylEnkidu]

Hi @raghavvijind ,

Thank you so much for reporting this. Can you provide us some more information to investigate this issue? For example, does this behaviour only show up after SDK upgrade to a certain version? Which Android device and what operation system version you are using?

[raghavvijind]

We were on firebase BOM 33.7.0. Difficult to mention the exact BOM version from which this issue might have started coming in as we started using local storage of firestore only a couple of months back and it seems very latent. Coincidentally, a similar issue exists for the iOS sdk as well. It seems as if there is common functionality in both the sdks that needs to be checked and solved. Will this issue be fixed anytime soon? We have started working on moving the persistence layer to an in-house solution in parallel as this is becoming a big reason for latencies in our app.

…

On Wed, Feb 19, 2025 at 1:10 AM cherylEnkidu ***@***.***> wrote: Hi @raghavvijind <https://github.com/raghavvijind> , Thank you so much for reporting this. Can you provide us some more information to investigate this issue? For example, does this behaviour only show up after SDK upgrade to a certain version? Which Android device and what operation system version you are using? — Reply to this email directly, view it on GitHub <#6673 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6MBEJQ7NP7YRRDTFC4J3KL2QOEEDAVCNFSM6AAAAABWQWRKM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRWG43TCOJXG4> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: cherylEnkidu]*cherylEnkidu* left a comment (firebase/firebase-android-sdk#6673) <#6673 (comment)> Hi @raghavvijind <https://github.com/raghavvijind> , Thank you so much for reporting this. Can you provide us some more information to investigate this issue? For example, does this behaviour only show up after SDK upgrade to a certain version? Which Android device and what operation system version you are using? — Reply to this email directly, view it on GitHub <#6673 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6MBEJQ7NP7YRRDTFC4J3KL2QOEEDAVCNFSM6AAAAABWQWRKM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRWG43TCOJXG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- ***Confidentiality Notice:*  ***The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed to. If you have received this message by mistake, please notify the sender immediately by e-mail and delete all copies from your system along with any attachments. Please be aware any unauthorized use, reproduction, or distribution of this message or part thereof is strictly prohibited.*

[cherylEnkidu]

We recently fixed a performance issue in the Web SDK, but this bug should not affect the Android and iOS SDKs.

In your case, enabling auto cache index creation might help:
Configure Offline Query Indexes.

By default, the SDK's cache system scans all documents in a collection to find the target. With cache indexing enabled, queries can locate results more efficiently.

[cherylEnkidu]

We would like your feedbacks for using this feature if possible :) Right now we set a threshold which creates cache indexes conservatively. If you would like to try more, I can explain how to change the SDK to create more cache indexes.

[raghavvijind]

Sure Cheryl. Thanks for sharing the documentation. We will test this and come back to you.

…

On Tue, Mar 11, 2025 at 8:23 AM cherylEnkidu ***@***.***> wrote: We would like your feedbacks for using this feature if possible :) Right now we set a threshold which creates cache indexes conservatively. If you would like to try more, I can explain how to change the SDK to create more cache indexes. — Reply to this email directly, view it on GitHub <#6673 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6MBEJSLN655EJFEE6EHNF32TZM3RAVCNFSM6AAAAABWQWRKM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMJSGM4DGOJUGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: cherylEnkidu]*cherylEnkidu* left a comment (firebase/firebase-android-sdk#6673) <#6673 (comment)> We would like your feedbacks for using this feature if possible :) Right now we set a threshold which creates cache indexes conservatively. If you would like to try more, I can explain how to change the SDK to create more cache indexes. — Reply to this email directly, view it on GitHub <#6673 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6MBEJSLN655EJFEE6EHNF32TZM3RAVCNFSM6AAAAABWQWRKM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMJSGM4DGOJUGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- ***Confidentiality Notice:*  ***The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed to. If you have received this message by mistake, please notify the sender immediately by e-mail and delete all copies from your system along with any attachments. Please be aware any unauthorized use, reproduction, or distribution of this message or part thereof is strictly prohibited.*