New document ID not unique after fork

[jdsutton7]

Originally reported here: firebase/firebase-admin-python#548

If firebase_admin is imported before os.fork(), auto-generated document IDs are repeated causing data to be overwritten.

Environment details

• OS type and version: Linux Mint 20
• Python version: Python 3.6.13
• pip version: pip 21.1
• google-cloud-firestore version: 2.1.0

Steps to reproduce

Import firebase_admin, fork, create firestore client, write a new document.

Code example

#!/usr/bin/env python3.6

import os
import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore

pid = os.fork()

cred = credentials.Certificate('service_account.json')
firebase_admin.initialize_app(cred)

client = firestore.client()

if pid == 0:
    reference = client.collection('Test').document()
    reference.set({'a': 'b'})

    print('Child: ' + reference.id)
else:
    reference = client.collection('Test').document()
    reference.set({'a': 'c'})

    print('Parent: ' + reference.id)

Output:

Child: TCsit2S0VQgvqm0aHNTc
Parent: TCsit2S0VQgvqm0aHNTc

[crwilcox]

This is the function that generates auto id.

python-firestore/google/cloud/firestore_v1/base_collection.py

Line 451 in 1d5c19a

def _auto_id() -> str:

The seed is determined at import time. To get unique seeds, I think auto_id needs to explicitly call seed. This could be done once per fork, but it may be inexpensive enough to just do this each and every time an _auto_id is needed.

random.seed()

[crwilcox]

Followup: I think for Go, possibly other clients, we do not set an id at all. This allows the server to choose the id, and the returned result contains the id as selected. It may be worth investigating if that would be a better approach, relying on the server side logic rather than implementing this in the client.

[Sylphias]

I'm also currently facing this issue. Firestore is constantly generating the same auto_id regardless of how many times I re-initialize my firebase client in my app.

The temporary workaround that I've tested is to run random.seed() (as @crwilcox has mentioned above) before .add() is called. This generates the auto_id properly.

I'm running this with a streamlit application and I hypothesize that it runs os.fork before my code is run.

[jdsutton7]

This issue should arguably be a higher priority as it can result in production data being silently overwritten. Another quick temporary fix is to provide your own IDs with uuid.uuid4() <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

…

On Mon, Jun 28, 2021 at 2:42 AM Kai Wei ***@***.***> wrote: I'm also currently facing this issue. Firestore is constantly generating the same auto_id regardless of how many times I re-initialize my firebase client in my app. The temporary workaround that I've tested is to run random.seed() (as @crwilcox <https://github.com/crwilcox> has mentioned above) before .add() is called. This generates the auto_id properly. I'm running this with a streamlit application and I hypothesize that it runs os.fork before my code is run. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#346 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AT422XMFHX6VPXTPKZYKOH3TVA7Y5ANCNFSM435IGONA> .

[crwilcox]

Thanks for the update @jdsutton7 . I have looked into this and it seems to be a narrow repro on the oldest supported python version. In short 3.7+ is fine and this appears to be an issue with 3.6 versions of Python specifically.

On Ubuntu 20, I have used CPython versions

• 3.9.6
• 3.8.11
• 3.7.11
• 3.6.12 -> repros
• 3.6.14 -> repros

The PR currently open will resolve this.

[stphnthiel]

I am still seeing that same effect on python 3.8.10 on Ubuntu 20.04.3 LTS (NVIDIA Docker image), but it is very hard to reproduce.

This also explains my somewhat big frustration with this issue, as it has been silently overwriting production data in a current project. What is especially hard is that the Firestore documentation treats these two ways to add data the same:

city = City(name=u'Tokyo', state=None, country=u'Japan')
db.collection(u'cities').add(city.to_dict())

new_city_ref = db.collection(u'cities').document()

# later...
new_city_ref.set({
    # ...
})

However, if you use approach no. 2 and run into this issue with existing IDs being returned by .document() you will overwrite existing data without any warning! The .add() approach at least throws a document exists.

I can’t emphasize enough that this should be addressed with the highest priority IMHO.

Further going to the bottom of this, I am very much interested in why such a peculiar approach to generating IDs was chosen in the first place? All of this frustration could be easily avoided by choosing established methods to generating truly unique IDs in Firestore per default, e.g. UUID4.

My current approach is to call random.seed() before any adding any data...