Firestore IndexedDB performance

[Zloka]

Describe your environment

• Operating System version: macOS 10.15.7
• Browser version: Google Chrome, Version 90.0.4430.212 (Official Build) (x86_64)
• Firebase SDK version: ^7.19.1
• Firebase Product: Firestore

Some more background that I believe is relevant:

• I use Dexie (https://dexie.org/) to download and store binary files offline, so that my users (students) can study offline. In extreme cases, this may be up to 2Gb of data.
• I have enabled persistence for users.
• I use the same query for a collection with 200 documents and collection with 18000 documents (see query below). In the current state, there is only one document with a matching id in the entire collection, so it should be easy to find . The data transfer should be low regardless, since only a single document is returned

The problem

For the "small collection" of 200 documents, the query runs in 700ms. For the "large collection" of 18000 documents, the query runs in 8000-15000ms.

Query

app
  .firestore()
  .collection(collection)
  .where('id', '==', id)
  .orderBy('updatedAt', 'desc')
  .limit(1)
  .get({ source: 'server' })

Theory

There is some performance issue with how the Firebase/Firestore JS SDK persists documents with regards to IndexedDB performance. For some reason, the (completely unrelated) binary data heavily affects the performance of the query as a whole.
By either clearing my local storage or disabling persistence, the query will run very quickly for the large collection as well.

Unresolved

If my theory is correct, I don't understand why the size of the queried collection matters. I have run the query against collections of various sizes and the response time jumps from 700ms to 8000+ ms once I reach a few thousand documents.

Any ideas? 🤔

[rafikhan]

Hi @Zloka - a couple fo things.

1. You didn't state this explicitly but it sounds like you might be storing binary data within Firestore. In case you are we would advise against it for cost and performance reasons.
2. As for query performance you are unfortunately hitting a known issue with the Firestore SDKs. Namely, that if you use the persistence feature, query performance will degrade when you pull down data for large collection. This is an area we definitely want to improve but is a very large project that will take time for us to complete. Unfortunately I can't give you any more details on this. Depending on your requirements you'll likely need to manage this separately.

[Zloka]

Hello @rafikhan!

1. I am not storing binary data within Firestore. I am storing binary data separately, locally in IndexedDB. This data has nothing to do with Firestore, other than that the Firestore offline cache also uses IndexedDB. The binary data that I download is all stored in Firebase Cloud Storage.
2. I'm not sure I understand why the query performance degrades? I am requesting a version from the server, and I limited it to 1 document. Shouldn't only 1 document then be cached offline? Why does the offline cache perform worse because the collection (on the server) is large, if only one document is fetched?

[schmidt-sebastian]

The SDK always overlays local edits even when the SDK fetches data from the server (including when source: server is set). This is required to provide a consistent view of all data and ensures that the SDK never raises snapshots with two different versions of the same document. To facilitate this, all queries are processed from our local cache. The backend merely acts as the source of the data in our cache.

source: server is a setting that forces us to bring the cache in sync before we raise a snapshot, but it does not change how we serve results. The only API that bypasses the cache is our transaction API, but this API only facilitates single document lookups and not query execution.

[Zloka]

@schmidt-sebastian I did not get what precisely you mean by "overlays" in this context, could you clarify?

I've browsed other issues (for example #2591) but I'm not sure it matches what I am seeing. Essentially, I don't have a lot of Firestore data offline, I only query individual documents and would be fine with keeping my Firestore cache quite small (10-20mb, smaller than the default, would probably be acceptable).

What it appears to boil down to is that when my unrelated IndexedDB data grows in size, Firestore performance degrades. I will get on with testing this in as much isolation as I can, but does that seem like a statement that should be false?

[schmidt-sebastian]

"overlays local edits" was my attempt to describe latency compensation. If you make an update to a document and read it, the result will contain your update even if your network is slow or you are offline. This makes app development much easier, as it allows you to use Firestore as the source of truth for all your documents, whether the user has edited them or now. The flipside is that we need to process all queries locally again, and hence you see the slowdown.

We are right now working on adding indexing support to our client, which will address this specific issue. This is a major project though and instead of waiting for its completion, I highly recommend changing your data model to ensure that the collections you download to your devices are relatively small.

[Zloka]

@schmidt-sebastian I see.

I highly recommend changing your data model to ensure that the collections you download to your devices are relatively small.

By "collections", I suppose you mean that the amount of documents and query and fetch to the device is relatively small? My collection in itself is quite large (i.e. lots of documents), but I only ever query a small amount of them, based on user interactions. I suppose this would be similar to having some basic users collection, where each user practically only ever interacts with their own document. This should be a fine model, no?

We are right now working on adding indexing support to our client, which will address this specific issue. This is a major project though and instead of waiting for its completion.

Even if I won't wait for its completion, that is nice to hear. Keep up the good work! 👍

[schmidt-sebastian]

@Zloka Yes, the number of documents on the server does not matter here. You should try to limit the number of documents that are downloaded locally.

[jorroll]

Cross posting this from #2457 which is closely related.

We've also ran into problems with Firestore persistence performance in our internal company messaging application. It got to be such a problem that we've disabled Firestore persistence. If it's helpful, here's a video showcasing the performance impact we saw.

• https://www.loom.com/share/81a7023707de4c90bdbdb8079ad27f0b

Note, I have no plans to create a full reproduction at this time.

[wu-hui]

Just to give an update:

The SDK cache index support is out, you might want to give it a try if you have a large collection saved in the cache, but your queries typically only need a small portion of the collection.

See: https://firebase.google.com/docs/reference/js/firestore_#setindexconfiguration