Firestore query failed with code=RESOURCE_EXHAUSTED, description=Quota exceeded.

[thomasdao]

Describe your environment

• Android Studio version: Arctic Fox 2020.3.1 Patch 3
• Firebase Component: Firestore
• Component version:

implementation platform('com.google.firebase:firebase-bom:28.4.1')
implementation 'com.google.firebase:firebase-firestore-ktx'

[REQUIRED] Step 3: Describe the problem

My app queries from Firestore, and when the result contains more than 2000 items, the query always fails with error WatchStream]: (c98c54b) Stream closed with status: Status{code=RESOURCE_EXHAUSTED, description=Quota exceeded., cause=null}. The detail error stacktrace can be downloaded here.

Note:

• This problem only happens on Android SDK. I am able to query on iOS and Web without a problem.
• This problem happens when each row contains quite significant amount of data. In my test, the final result contains around 5000 items, total size is about 120MB
• I am on Blaze plan
• I have waited for more than 30 minutes before executing a new query

Steps to reproduce:

Simply execute a query. The query will fail for user with a lot of data.

Relevant Code:

fun getAll() {
    val ref = FirebaseFirestore.getInstance()
        .collection("users")
        .document(userId)
        .collection("notes")
        .limit(2500)

    // This query always failed with the limit >= 2500
    ref.get(Source.SERVER)
        .addOnSuccessListener { results ->
            Log.d("Test","results size" + results.size())
        }.addOnFailureListener {
            it.printStackTrace()
            Log.d("Test", "exception ${it.localizedMessage}")
        }
}

[aguatno]

Hi @thomasdao thanks for reporting. One possible reason that this happens is because you might be crossing a limit that exist in Firestore regardless of plan. The system might return “resource exhausted” in various different scenarios.

I would recommend you accessing the Google Cloud Platform console to get a more accurate depiction of the usage your database is experiencing. I suggest this as sometimes the Firebase Console isn't as accurate as the GCP one.

[thomasdao]

@aguatno thanks for replying. As I mentioned in the original report, the query is successful for iOS and Web Firestore SDK, this problem only happens for Android. I don't think this problem is due to the limit (otherwise the iOS and Web query should have failed).

[thebrianchen]

@thomasdao This seems like a backend issue, since the SDK only retries and surfaces the errors. Can you file a ticket via Firebase support channel so the backend team can take a look at your project's logs?

[thomasdao]

@thebrianchen I contacted support - the backend team did not see any error with "RESOUCE_EXHAUSTED" in my project log.

Francisco at support team has been able to reproduce this issue and find out the problem is due to the way Firestore Android SDK handle the cache. If the cache folder is deleted when the query returns, the exception does not happen, however I hope the Firestore Android team could confirm if this is indeed the root case.

[thebrianchen]

@thomasdao Yes, after looking at Francisco's report + the absence of backend logs, it's clear that the error is thrown by SQLite after the cache is full. To workaround this, you can either clear the cache directory with Android utility methods, or increase the size of the cache via Firestore settings.

Internally, we'll look at improving documentation around this error case and improving the error messages. I'm going to close the issue, but feel free to ask any other questions you may have!

[thomasdao]

@thebrianchen I would prefer to keep this issue open since it is a quite serious bug and affect a lot of our users. The work around to clear the cache is only a work around - we don't know if it is a real root cause, and if it is the root cause, it would be better to fix on the SDK side. I think "improving documentation and error message" is not a correct approach to this problem.

[thomasdao]

Update: when I apply the work around today - it no longer works! It looks like the work around is not a real root cause to this problem and this problem still happens regularly for me.

[thomasdao]

@thebrianchen can you reopen this issue? This is serious bug and I don’t understand why you are so eager to close it without even investigating the issue

[bacarPereira]

In my case I exceeded the amount of written documents / day in firestore (Spark plan)

.