Skip to content

test: parametrize to_arrow empty stream tests and update docstrings - #17919

Merged
chalmerlowe merged 3 commits into
mainfrom
bugfix/352600521-empty-table-materialization
Jul 28, 2026
Merged

test: parametrize to_arrow empty stream tests and update docstrings#17919
chalmerlowe merged 3 commits into
mainfrom
bugfix/352600521-empty-table-materialization

Conversation

@chalmerlowe

@chalmerlowe chalmerlowe commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Problem

When reading from an empty BigQuery table using the Read API, the stream may return no messages. Previously, this could lead to an AttributeError when trying to parse the schema from a None parser. While a fallback was added to return an empty table, this fallback uses an empty schema (zero columns), which can cause downstream consumers (like Vertex Ray) to fail if they expect specific columns to be present.

Solution

  1. Updated Documentation: Added explanatory comments and updated docstrings in reader.py to clarify the flow between ReadRowsStream.to_arrow() and ReadRowsIterable.to_arrow(), and to inform users about the fallback behavior.
  2. Added Parametrized Tests: Added comprehensive tests to test_reader_v1_arrow.py to verify the behavior of to_arrow() on empty streams, both with and without a read_session provided.

Notes to Reviewers

  • This PR does not change functionality but adds testing and documentation around existing behavior to prevent future regression and aid debugging. The functionality that should ensure the correct behavior in internal bug #352600521 was added in February in PR # 15575. This PR is less about fixing the bug versus documenting the correct use of the code to avoid issues.
  • To guarantee the correct schema (non-empty) for empty streams, callers should provide the read_session to ReadRowsStream.to_arrow().

Fixes internal bug #352600521

@chalmerlowe chalmerlowe self-assigned this Jul 27, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds documentation and inline comments to clarify the behavior of to_arrow() on empty streams in ReadRowsIterable and ReadRowsStream. It also introduces a new unit test, test_to_arrow_empty_stream, to verify empty stream handling. Feedback on the unit test highlights a potential TypeError when testing ReadRowsIterable because it does not accept a read_session argument, and suggests a code modification to conditionally handle this case.

@chalmerlowe
chalmerlowe marked this pull request as ready for review July 28, 2026 15:20
@chalmerlowe
chalmerlowe requested review from a team as code owners July 28, 2026 15:20
@chalmerlowe
chalmerlowe requested review from shuoweil and removed request for a team July 28, 2026 15:20
@chalmerlowe
chalmerlowe merged commit ff58ec9 into main Jul 28, 2026
39 checks passed
@chalmerlowe
chalmerlowe deleted the bugfix/352600521-empty-table-materialization branch July 28, 2026 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants