FR: keep counts of documents in collections

[kenyk7]

Sería genial si agregaran campos meta a las colleciones de firestore por ejemplo un campo meta que devuelva la cantidad de items de esa collección, ya que he visto que se usa bastante y lo uso mucho y a veces llega a ser un problema común si no se trata con cuidado, seria genial si estuviera agregado por defecto el los resultados de las collecciones del nuevo cloud firestore.

Ejemplo de respuesta:

{
  "users": {
    "data": {
      "uid1": {
        "name": "Keny"
      },
      "uid2": {
        "name": "Kalín"
      }
    },
    "meta": {
      "count": 2
    }
  }
}

[wilhuff]

Unfortunately, keeping counts of items in a collection is very hard to do and while maintaining reasonable performance in a distributed database like Firestore. This is because each update to the count becomes a participant in a transaction, and this makes each write substantially more expensive. This isn't something we can do by default.

It is possible to keep counts for yourself though as you can see in our documentation there are trade-offs you'll have to make in implementing this.

[riless]

Any progression about this issue ? will this feature be available in 1 month ? 1 year ? 10 years ? never ?

It will help me choose the right provider for my future projects.

[wilhuff]

As a general rule we cannot make forward looking statements like the ones you're asking for. Please evaluate Firestore for your needs based on the features we have, not the features we might have someday.

[darren-dev]

Okay, sure, but SURELY there could be a backend task, independent of a tracked counter (or auto index number), that we could call that would simply give a total count (in the fastest / most efficient way possible) back to us, without the expensive need of the developer pulling all the data?

Surely locking your potential clients (Of which I would assume a lot are hobbyists) into a potential costly query every time they want one number is a bit of a restrictive move?

One of the semi-official ways of 'the developer' keeping track is to have an independent 'counters' collection alongside the rest. Surely if we can do it, a system as powerful as Firebase could too? Plus, the above relies on the developer knowing exactly what to do before even setting up their collection(s), as well as that function NEVER having any bugs or issues (As any would affect the counters).

[rsgowman]

Unfortunately, there's not much I can add here that @wilhuff hasn't already touched on.

Yes, it's possible that there could be a backend task that tracks counts, but it would require the backend to update the counter for each new document, which would both decrease scalability and increase costs. Most collections don't need this counter, so enforcing this cost in all cases would be detrimental. Sharding the counter (as described in the above link) would help mitigate some of this, but wouldn't eliminate it entirely. We just can't do this by default.

It might be possible to opt-in to this behaviour? Such functionality certainly doesn't exist today, but if you feel really strongly about it, you could request it on the backend tracker, listed here: https://cloud.google.com/support/docs/issue-trackers. However, I believe the backend team is mostly prioritizing issues that can't already be solved by users directly, and in this case, there's already a documented method to maintain these counters yourself.

[darren-dev]

Thanks for the reply; I think anyone that's actively using Firebase can appreciate every iota of functionality they can squeeze out of it... It's definitely the best service in comparison to anything else out there.

I can understand your point about priority, as well as your point to a cost and scaling issue, I would just like to raise the very real-world issue of actually paying for such a potentially large query just for a number.

Although this might not be a 'fix now' issue, I think it would be very wise to have an open conversation on how to meet half way - your idea about an opt in service isn't bad. Sadly I don't know the fundamentals of the technology used for Firestore, so I can't offer suggestions, I can only say one should look into this.