Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

865 the hook needs to contain similar kwargs as the google providers #866

Conversation

erdos2n
Copy link
Contributor

@erdos2n erdos2n commented Feb 7, 2023

I have added the location parameter in the hook. I do think this needs to get upgraded to integrate the use_legacy_sql parameter, as well, but that change may not be so simple given it could change the requirements.txt.

Copy link
Contributor

@pankajastro pankajastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rajaths010494
Copy link
Contributor

rebase once the PR gets merged #867
and mypy issue should be resolved

Copy link
Collaborator

@phanikumv phanikumv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please add this additional param to example DAG as well?

@pankajastro
Copy link
Contributor

pankajastro commented Feb 8, 2023

FYI: Look like google async lib have some issue with non us location apache/airflow#29307

@erdos2n
Copy link
Contributor Author

erdos2n commented Feb 8, 2023

One of the errors, currently, is a permission issue. It would make sense that it's a region issue, but I haven't verified for sure, since it's not my service-account.

We should include this and just update the docs to reflect a possible issue with non-US locations.

@codecov
Copy link

codecov bot commented Feb 8, 2023

Codecov Report

Base: 98.61% // Head: 98.61% // No change to project coverage 👍

Coverage data is based on head (3b7ebdb) compared to base (b58089d).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #866   +/-   ##
=======================================
  Coverage   98.61%   98.61%           
=======================================
  Files          89       89           
  Lines        4987     4987           
=======================================
  Hits         4918     4918           
  Misses         69       69           
Impacted Files Coverage Δ
...nomer/providers/google/cloud/operators/bigquery.py 100.00% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@pankajastro
Copy link
Contributor

could you please add this additional param to example DAG as well?

We have an example here using location param https://github.com/astronomer/astronomer-providers/blob/main/astronomer/providers/google/cloud/example_dags/example_bigquery_impersonation_chain.py#L92 do you mean something else?

@rajaths010494
Copy link
Contributor

rajaths010494 commented Feb 24, 2023

Can we fix the failing tests as well? If you need any help with tests do let us know.

@erdos2n
Copy link
Contributor Author

erdos2n commented Feb 24, 2023

I've gone ahead and removed the use_legacy_sql kwarg from this PR since it was failing the tests. I think it's best to open another issue for this, as the current issue with the location kwarg is breaking a DAG for a customer. I will open a new, much more broad issue for update the bigquery.py file to meet the current kwargs of the provider package.

@rajaths010494
Copy link
Contributor

rajaths010494 commented Feb 24, 2023

Can you rebase the branch?
There are mypy issues also if can resolve them.

@rajaths010494 rajaths010494 merged commit 12096b1 into main Mar 7, 2023
@rajaths010494 rajaths010494 deleted the 865-the-hook-needs-to-contain-similar-kwargs-as-the-google-providers branch March 7, 2023 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The Hook needs to contain similar kwargs as the Google Providers
4 participants