Skip to content

Null values in GBQ not correctly converted to NaN in pandas #408

@snauhaus

Description

@snauhaus

I'm experiencing an issue where the data frame returned by pd.read_gbq contains null values that are not recognized by Pandas. That is, df.loc[df.foobar.isnull()] or df.loc[df.foobar.isna()] returns rows where foobar is actually not null/na. And, vice versa, for notnull() and notna() the same happens.

The problem does not exist when first downloading from GBQ as csv and importing that file.

Attached two screenshots of the issue. Unfortunately I cannot provide a data example.

MicrosoftTeams-image (1)

MicrosoftTeams-image

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the googleapis/python-bigquery-pandas API.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions