This repository was archived by the owner on Mar 2, 2026. It is now read-only.
Memory leak in document.get #835
Description
Environment details
- OS type and version: MacOS Ventura 3.14.1
- Python version: 3.12.1
- pip version: pip 23.2.1
google-cloud-firestoreversion: 2.14.0
Code example
Running this file through memray:
python -m memray run mem_debug.py
import google.auth
from google.cloud import firestore
from memory_profiler import profile
DOCUMENT = 'my-document-key'
PROJECT = 'my-project'
COLLECTION = 'my-collection'
def check_exists(collection):
document_ref = collection.document(DOCUMENT)
document = document_ref.get()
exists = document.exists
return exists
def fs():
for x in range(1,1000):
creds, _ = google.auth.default()
fc = firestore.Client(
project=PROJECT,
credentials=creds
)
collection = fc.collection(COLLECTION)
check_exists(collection)
if __name__ == '__main__':
fs()
I did a further digging, and the leak appears to be in response_iter = self._client._firestore_api.batch_get_documents
File:
Line # Mem usage Increment Occurrences Line Contents
=============================================================
365 37.5078 MiB 37.5078 MiB 1 @profile(stream=sys.stdout, precision=4)
366 def get(
367 self,
368 field_paths: Iterable[str] = None,
369 transaction=None,
370 retry: retries.Retry = gapic_v1.method.DEFAULT,
371 timeout: float = None,
372 ) -> DocumentSnapshot:
373 """Retrieve a snapshot of the current document.
374
375 See :meth:`~google.cloud.firestore_v1.base_client.BaseClient.field_path` for
376 more information on **field paths**.
377
378 If a ``transaction`` is used and it already has write operations
379 added, this method cannot be used (i.e. read-after-write is not
380 allowed).
381
382 Args:
383 field_paths (Optional[Iterable[str, ...]]): An iterable of field
384 paths (``.``-delimited list of field names) to use as a
385 projection of document fields in the returned results. If
386 no value is provided, all fields will be returned.
387 transaction (Optional[:class:`~google.cloud.firestore_v1.transaction.Transaction`]):
388 An existing transaction that this reference
389 will be retrieved in.
390 retry (google.api_core.retry.Retry): Designation of what errors, if an y,
391 should be retried. Defaults to a system-specified policy.
392 timeout (float): The timeout for this request. Defaults to a
393 system-specified value.
394
395 Returns:
396 :class:`~google.cloud.firestore_v1.base_document.DocumentSnapshot`:
397 A snapshot of the current document. If the document does not
398 exist at the time of the snapshot is taken, the snapshot's
399 :attr:`reference`, :attr:`data`, :attr:`update_time`, and
400 :attr:`create_time` attributes will all be ``None`` and
401 its :attr:`exists` attribute will be ``False``.
402 """
403 37.5078 MiB 0.0000 MiB 1 from google.cloud.firestore_v1.base_client import _parse_batch_get
404
405 37.5078 MiB 0.0000 MiB 1 request, kwargs = self._prep_batch_get(field_paths, transaction, retry, timeout)
406
407 45.3242 MiB 7.8125 MiB 3 response_iter = self._client._firestore_api.batch_get_documents(
408 39.9453 MiB 0.0000 MiB 1 request=request,
409 39.9492 MiB 0.0039 MiB 1 metadata=self._client._rpc_metadata,
410 39.9492 MiB 0.0000 MiB 1 **kwargs,
411 )
412
413 45.3242 MiB 0.0000 MiB 1 get_doc_response = next(response_iter, None)
414
415 45.3242 MiB 0.0000 MiB 1 if get_doc_response is not None:
416 45.3750 MiB 0.0508 MiB 2 return _parse_batch_get(
417 45.3242 MiB 0.0000 MiB 1 get_doc_response=get_doc_response,
418 45.3242 MiB 0.0000 MiB 1 reference_map={self._document_path: self},
419 45.3242 MiB 0.0000 MiB 1 client=self._client,
420 )
421
422 logger.warning(
423 "`batch_get_documents` unexpectedly returned empty "
424 "stream. Expected one object.",
425 )
426
427 return DocumentSnapshot(
428 self,
429 None,
430 exists=False,
431 read_time=_datetime_to_pb_timestamp(datetime.datetime.now()),
432 create_time=None,
433 update_time=None,
434 )
This is my pip freeze output:
cachetools==5.3.2
certifi==2023.11.17
charset-normalizer==3.3.2
google-api-core==2.15.0
google-auth==2.26.2
google-cloud-core==2.4.1
google-cloud-firestore==2.14.0
googleapis-common-protos==1.62.0
grpcio==1.60.0
grpcio-status==1.60.0
idna==3.6
Jinja2==3.1.3
linkify-it-py==2.0.2
markdown-it-py==3.0.0
MarkupSafe==2.1.4
mdit-py-plugins==0.4.0
mdurl==0.1.2
memory-profiler==0.61.0
memray==1.11.0
proto-plus==1.23.0
protobuf==4.25.2
psutil==5.9.8
pyasn1==0.5.1
pyasn1-modules==0.3.0
Pygments==2.17.2
requests==2.31.0
rich==13.7.0
rsa==4.9
textual==0.47.1
typing_extensions==4.9.0
uc-micro-py==1.0.2
urllib3==2.1.0