Skip to content

Conversation

@KamranImaaz
Copy link
Contributor

@KamranImaaz KamranImaaz commented Jan 4, 2026

Fixes #60096

What

Adds a safety condition to OpenLineage extraction to prevent errors when
BigQueryUpdateTableSchemaOperator, BigQueryUpdateTableOperator, or
BigQueryUpsertTableOperator do not return a table resource.
Also removes duplicate and unused imports and updates type annotations
to use dict | None where applicable.

Why

OpenLineage callbacks run after task execution and should not cause
task failures when metadata is unavailable.

@boring-cyborg boring-cyborg bot added area:providers provider:google Google (including GCP) related issues labels Jan 4, 2026
@KamranImaaz
Copy link
Contributor Author

Hi @potiuk, I’ve addressed #60096 with a safety guard for OpenLineage and cleaned up the PR.
Would appreciate your review when convenient. Thanks!

@jroachgolf84
Copy link
Collaborator

@KamranImaaz, do you think it might make sense to add a unit test or two to validate this new logic?

@KamranImaaz
Copy link
Contributor Author

@KamranImaaz, do you think it might make sense to add a unit test or two to validate this new logic?

Yes @jroachgolf84 currently working on it. Thanks for pointing it out.

@KamranImaaz
Copy link
Contributor Author

@jroachgolf84 I have added unit test to cover the case when _table is None. Requesting you to check it.

Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR title doesn't confirm to standards -- it doesn't say anything much.

@KamranImaaz KamranImaaz changed the title Fix bigquery openlineage 60096 Prevent OpenLineage(Google Provider) failure when BigQuery table metadata is missing. Fix(# 60096) Jan 5, 2026
@KamranImaaz
Copy link
Contributor Author

PR title doesn't confirm to standards -- it doesn't say anything much.

@kaxil I have updated the title. pls check it.

@KamranImaaz KamranImaaz requested a review from kaxil January 5, 2026 19:01
@kaxil kaxil changed the title Prevent OpenLineage(Google Provider) failure when BigQuery table metadata is missing. Fix(# 60096) Prevent OpenLineage failure when BigQuery table metadata is missing Jan 5, 2026
@potiuk
Copy link
Member

potiuk commented Jan 5, 2026

You need to rebase it and fix the errors

@KamranImaaz
Copy link
Contributor Author

You need to rebase it and fix the errors

Sure @potiuk

@KamranImaaz
Copy link
Contributor Author

You need to rebase it and fix the errors

Hi @potiuk I have gone through the CI Checks error specifically build documentation(docs only). So far what I have understood is that type annotations for execute method(ex: def execute()-> dict | None) is causing error. I'm i correct?

@potiuk
Copy link
Member

potiuk commented Jan 6, 2026

You definitely have checks that can be fixed by installing prek and running it as suggested in both - the error and contributor's guide, and you have some doc error that you can try to build locally (again following our contributing guidelines describing how to build doscs)l

Also you have to rebase and resolve the conflicts you have - so rebasing (also described in the contribution guide) is something you should start with.

@KamranImaaz
Copy link
Contributor Author

You definitely have checks that can be fixed by installing prek and running it as suggested in both - the error and contributor's guide, and you have some doc error that you can try to build locally (again following our contributing guidelines describing how to build doscs)l

Also you have to rebase and resolve the conflicts you have - so rebasing (also described in the contribution guide) is something you should start with.

Yes I have rebased it and resolved the conflicts. Need to check the errors locally. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

issue in OpenLineage callback when table is None, moving imports up, type annotations.

4 participants