feat(firestore): literals pipeline stage by Linchin · Pull Request #16028 · googleapis/google-cloud-python

Linchin · 2026-03-05T00:27:41Z

Succeeding googleapis/python-firestore#1170 for the monorepo migration.

gemini-code-assist · 2026-03-05T00:28:01Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the Firestore client library by introducing a new literals pipeline stage. This stage provides a mechanism to inject a fixed set of predefined document objects directly into a pipeline, offering greater flexibility for testing individual pipeline stages and serving as initial inputs for complex operations like join conditions. The change includes the necessary class implementation, method integration into the base pipeline, and corresponding test coverage.

Highlights

New literals pipeline stage: Introduced a new literals method to the _BasePipeline class, allowing users to define documents from a fixed set of predefined objects. This stage is particularly useful for testing other pipeline stages in isolation or as inputs for join conditions.
Implementation of Literals class: Added the Literals class in pipeline_stages.py to represent the new pipeline stage, including its constructor and methods for protobuf serialization.
Comprehensive testing: Included new system-level and unit tests to ensure the correct functionality, behavior, and integration of the literals pipeline stage.

Changelog

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py
- Added the literals method to _BasePipeline, enabling the definition of documents from a fixed set.
- Included detailed docstrings with examples and behavior descriptions for the new literals method.
packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_stages.py
- Introduced the Literals class, inheriting from Stage, to represent the new pipeline stage.
- Implemented the __init__ and _pb_args methods for the Literals class to handle document arguments and their protobuf serialization.
packages/google-cloud-firestore/tests/system/pipeline_e2e/general.yaml
- Appended a new system test case for the literals pipeline stage, demonstrating its usage with a sample document and asserting the expected results and protobuf structure.
packages/google-cloud-firestore/tests/unit/v1/test_pipeline.py
- Updated the test_pipeline_methods parameterized test to include the new literals stage, ensuring it can be correctly appended to a pipeline.
packages/google-cloud-firestore/tests/unit/v1/test_pipeline_stages.py
- Added a new test class TestLiterals with unit tests for the Literals stage's constructor, representation (repr), and protobuf serialization (_to_pb).

Activity

No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a new literals pipeline stage, which allows specifying a fixed set of documents as the starting point of a pipeline. The implementation includes the literals method on the pipeline builder, the Literals stage class, and corresponding unit and end-to-end tests. My review focuses on improving the clarity and correctness of the type hints and docstrings for the new functionality. I've suggested changes to make the API easier to understand and use correctly.

gemini-code-assist · 2026-03-05T00:29:59Z

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

            stages.FindNearest(field, vector, distance_measure, options)
        )

+    def literals(self, *documents: str | Selectable) -> "_BasePipeline":


The type hint for *documents is incomplete. It should include dict as documents are often passed as dictionaries, which is not covered by str | Selectable.

Suggested change

def literals(self, *documents: str | Selectable) -> "_BasePipeline":

def literals(self, *documents: dict | str | Selectable) -> "_BasePipeline":

You can probably ignore this, unless that's how other languages handle it

Actually, looking at go, it seems like it accepts dicts, but not strings?

I don't know much about this stage, but from what I've seen, it's supposed to deal with maps. So maybe this should be def literals(self, *documents: Map | dict[str, CONSTANT_TYPE] | Selectable)?

Upon further thoughts, I think it should be def literals(self, *documents: dict | Expression):. In this case both Constant and Map are child classes of Expression.

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_stages.py

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

daniel-sanche · 2026-03-05T22:25:45Z

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

            stages.FindNearest(field, vector, distance_measure, options)
        )

+    def literals(self, *documents: str | Selectable) -> "_BasePipeline":


You can probably ignore this, unless that's how other languages handle it

packages/google-cloud-firestore/tests/system/pipeline_e2e/general.yaml

daniel-sanche · 2026-03-05T22:34:23Z

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

+            ...     {"name": "alice", "age": 40}
+            ... ]
+            >>> pipeline = client.pipeline()
+            ...     .literals(Constant.of(documents))


Looking at the code, it seems like:

Constant isn't a Selectable

Constant doesn't seem like it supports dict types. (We do have a Map, which serves that purpose, but it doesn't seem Selectable either)

Thanks for catching this! I spent some time to check the internal docs, and I think Expression class and dict should be supported per the following language:

While literal values are the most common, it is also possible to pass in expressions, which will be evaluated and returned, making it possible to test out different query / expression behavior without first needing to create some test data.

daniel-sanche · 2026-03-05T22:36:58Z

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

            stages.FindNearest(field, vector, distance_measure, options)
        )

+    def literals(self, *documents: str | Selectable) -> "_BasePipeline":


Actually, looking at go, it seems like it accepts dicts, but not strings?

I don't know much about this stage, but from what I've seen, it's supposed to deal with maps. So maybe this should be def literals(self, *documents: Map | dict[str, CONSTANT_TYPE] | Selectable)?

packages/google-cloud-firestore/tests/unit/v1/test_pipeline_stages.py

daniel-sanche · 2026-03-05T22:38:39Z

packages/google-cloud-firestore/tests/system/pipeline_e2e/general.yaml

+                  stringValue: "Douglas Adams"
+                title:
+                  stringValue: "The Hitchhiker's Guide to the Galaxy"
+          name: literals


We should also have tests here that cover the different input types we support

Good catch! I added additional type to test.

I think we still need more examples. Both of these are dict-like, so it seems like a pretty basic test. What if someone passes in Constant(1)? Or Constant("test").byte_length()? We say we support all expressions, how are non-dict types represented?

It would also be good to add some extra stages, to make sure this works like others

You can use gemini to create a few extra test scenarios. I usually don't include assert_proto on all of them, because it can be excessive

Referring to the golang implementation, the only type accepted is a list of key-value pairs. So I don't think Constant(1) or Constant("test").byte_length() are in scope.

But I think it's a good idea to add more tests. I will also use mapValue instead of Constant.

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

daniel-sanche · 2026-03-10T22:40:19Z

packages/google-cloud-firestore/tests/system/pipeline_e2e/general.yaml

+                  stringValue: "Douglas Adams"
+                title:
+                  stringValue: "The Hitchhiker's Guide to the Galaxy"
+          name: literals


I think we still need more examples. Both of these are dict-like, so it seems like a pretty basic test. What if someone passes in Constant(1)? Or Constant("test").byte_length()? We say we support all expressions, how are non-dict types represented?

It would also be good to add some extra stages, to make sure this works like others

You can use gemini to create a few extra test scenarios. I usually don't include assert_proto on all of them, because it can be excessive

packages/google-cloud-firestore/tests/unit/v1/test_pipeline_stages.py

daniel-sanche · 2026-03-10T22:46:41Z

packages/google-cloud-firestore/tests/system/pipeline_e2e/general.yaml

+        - Constant:
+            value:
+              genre: "Science Fiction"
+              year: 1979


This looks like it resolves to Constant({"genre": "Science Fiction", "year": 1979}). But our type hints forbid passing dicts as constants like this (mostly because Python's typing system isn't powerful enough to type nested datastructures, but still)

We provide a Map, which is supposed to fill that gap. Maybe you should test that out, and see if it would work as a declared type here?

daniel-sanche · 2026-03-10T22:51:48Z

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

            stages.FindNearest(field, vector, distance_measure, options)
        )

+    def literals(self, *documents: Expression | dict) -> "_BasePipeline":


Are you sure it makes sense to accept Expression here? Maybe it should just be dict | Map?

Expressions can evaluate to anything, and I don't know if it makes sense to have an int or bool Document

daniel-sanche · 2026-03-10T23:08:03Z

packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py

+        Returns documents from a fixed set of predefined document objects.
+
+        This stage is commonly used for testing other stages in isolation,
+        though it can also be used as inputs to join conditions.


This seems to conflict with the statement later:

The literals(...) stage can only be used as the first stage in a pipeline (or sub-pipeline)

See https://github.com/googleapis/google-cloud-python/pull/16028/changes#r2955450923

Linchin · 2026-03-18T18:33:14Z

packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_source.py

+        """
+        Returns documents from a fixed set of predefined document objects.
+
+        This stage is commonly used for testing other stages in isolation,


@daniel-sanche I think what the documentation means is that, the Literals stage could serve as one of the inputs of a join operation, which is not contradictory to it being a first stage, so I think it is reasonable to keep this sentence and make it consistent with documentation.

I'd still prefer to take it out since we don't yet support the join stage. But if you think it seems to match the coming feature, we can probably leave it

daniel-sanche · 2026-03-18T23:56:18Z

packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_source.py

+        """
+        Returns documents from a fixed set of predefined document objects.
+
+        This stage is commonly used for testing other stages in isolation,


I'd still prefer to take it out since we don't yet support the join stage. But if you think it seems to match the coming feature, we can probably leave it

daniel-sanche · 2026-03-18T23:59:09Z

packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_source.py

        return self._create_pipeline(stages.Documents.of(*docs))
+
+    def literals(
+        self, *documents: Mapping[str, Expression | CONSTANT_TYPE]


I think we should keep this as dict instead of Mapping, just because Firestore has its own custom Map type, which could get confusing

Thanks for catching this, I have corrected it.

daniel-sanche · 2026-03-19T00:00:54Z

packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_source.py

+
+        Args:
+            *documents: One or more documents to be returned by this stage. Each can be a `dict`
+                       or an `Expression`.


Each can be a dict or an Expression.

This seems out of date? The annotations only support dicts (which seems to be right, since that's how other languages handle it)

Thanks for catching this, I think Expression means the type of value in the dict could be of Expression type. I have updated the language to make it more accurate.

daniel-sanche · 2026-03-19T00:02:58Z

packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_stages.py

+        args = []
+        for doc in self.documents:
+            if hasattr(doc, "_to_pb"):
+                args.append(doc._to_pb())


is this ever called? doc should be a dict, right? This section seems unreachable

Thanks so much, I have removed this part, and also removed the related unit test.

daniel-sanche · 2026-03-19T00:08:32Z

packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_stages.py

+
+    def __init__(self, *documents: Mapping[str, Expression | CONSTANT_TYPE]):
+        super().__init__("literals")
+        self.documents = documents  # type: ignore


And why are you ignoring the type warning? What error were you seeing?

If I remove # type: ignore, I get the following error message:

pipeline_stages.py:351: error: Need type annotation for "documents" [var-annotated]

If I change the type hint to be the same as init like the following

self.documents: tuple[dict[str, Expression | CONSTANT_TYPE], ...] = documents

I get these mypy errors:

.nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | str], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment] .nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | int], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment] .nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | float], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment] .nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | bool], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment] .nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | datetime], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment] .nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | bytes], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment] .nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | GeoPoint], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment] .nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | Vector], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment] .nox/mypy-3-12/lib/python3.12/site-packages/google/cloud/firestore_v1/pipeline_stages.py:351: error: Incompatible types in assignment (expression has type "tuple[dict[str, Expression | None], ...]", variable has type "tuple[dict[str, Expression | CONSTANT_TYPE], ...]") [assignment]

It seems mypy couldn't make out that str, int etc. are subtypes of CONSTANT_TYPE. I wonder if there is a more proper way to do type hint here? Otherwise I don't think we lose much coverage with type ignore.

daniel-sanche · 2026-03-19T00:13:32Z

packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_stages.py

+
+    def __init__(self, *documents: Mapping[str, Expression | CONSTANT_TYPE]):
+        super().__init__("literals")
+        self.documents = documents  # type: ignore


Consider converting Python-native types into Expressions here, like we do in the other stages. This should make building _pr_args later cleaner, since everything would be an Expression with _to_pb

Something like:

def __init__(self, *documents: Mapping[str, Expression | CONSTANT_TYPE]): super().__init__("literals") self.documents = [ {k: v if isinstance(v, Expression) else Constant.of(v) for k,v in doc} for doc in documents ]

Thanks, I have fixed it.

daniel-sanche · 2026-03-19T00:16:12Z

packages/google-cloud-firestore/tests/unit/v1/test_pipeline_source.py

+        from google.cloud.firestore_v1.pipeline_expressions import Field
+
+        instance = self._make_client().pipeline()
+        documents = (Field.of("a"), {"name": "joe"})


Field.of("a") isn't a dict

Thanks, I have corrected it.

daniel-sanche · 2026-03-19T00:18:12Z

packages/google-cloud-firestore/tests/unit/v1/test_pipeline_stages.py

+        return stages.Literals(*args, **kwargs)
+
+    def test_ctor(self):
+        val1 = Constant.of({"a": 1})


I think you should change this, since Constant doesn't support dict inputs (even if it technically works)

You could make this {"a": Constant.of(1)} to be conformant

It looks like this Constant.of(dict) pattern is used in a few other places in the tests

Thanks, I have corrected this and also fixed other tests where the input is not a dict.

Linchin · 2026-03-20T22:38:00Z

daniel-sanche
2 days ago
I'd still prefer to take it out since we don't yet support the join stage. But if you think it seems to match the coming feature, we can probably leave it

Makes sense, I have taken out the languange.

feat: literals pipeline stage

2a8213c

Linchin requested review from a team as code owners March 5, 2026 00:27

Linchin changed the title ~~feat: literals pipeline stage~~ feat(firestore): literals pipeline stage Mar 5, 2026

gemini-code-assist bot reviewed Mar 5, 2026

View reviewed changes

Linchin assigned daniel-sanche Mar 5, 2026

daniel-sanche requested changes Mar 5, 2026

View reviewed changes

Linchin added 6 commits March 6, 2026 21:37

add dict as supported type and update docstring

28acee9

correct docstring and type annotation

a2e11ed

type annotation

a97cb13

add more unit tests

cc1b737

correct docstring

3861223

add cases to system test

7bad13c

Linchin requested a review from daniel-sanche March 9, 2026 21:38

fix system test

36e45bf

daniel-sanche requested changes Mar 10, 2026

View reviewed changes

daniel-sanche reviewed Mar 10, 2026

View reviewed changes

Linchin added 4 commits March 11, 2026 20:36

move literals to pipeline_source.py

816498e

correct system test

09f59f6

type annotation and add unit tests

11c22ee

docstring

941101f

Linchin commented Mar 18, 2026

View reviewed changes

Linchin added 5 commits March 18, 2026 18:38

type annotation

ccb159a

add more system test

6aae503

update to accept Expression

fa30d5f

lint

db98e44

lint

4a85ee6

daniel-sanche reviewed Mar 19, 2026

View reviewed changes

remove languange about join stage

5627c75

use dict instead of Mapping for type hint

0dbfc77

Linchin added 4 commits March 20, 2026 22:52

improve wording for args docstring

837e52c

remove unused code and test

fc2880d

correct unit test cases

a515903

remove unused typing

eee827e

	def literals(self, *documents: str \| Selectable) -> "_BasePipeline":
	def literals(self, *documents: dict \| str \| Selectable) -> "_BasePipeline":

Conversation

Linchin commented Mar 5, 2026

Uh oh!

gemini-code-assist bot commented Mar 5, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniel-sanche Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Linchin Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniel-sanche Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniel-sanche Mar 5, 2026 •

edited

Loading

Linchin Mar 9, 2026 •

edited

Loading

daniel-sanche Mar 5, 2026 •

edited

Loading