Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BigQueryToMsSqlOperator #15422

Merged
merged 19 commits into from Jun 12, 2021

Conversation

subkanthi
Copy link
Contributor

@subkanthi subkanthi commented Apr 18, 2021

closes: #15145


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@boring-cyborg boring-cyborg bot added area:providers provider:google Google (including GCP) related issues labels Apr 18, 2021
@subkanthi subkanthi marked this pull request as draft April 19, 2021 01:31
self.location = location
self.impersonation_chain = impersonation_chain
try:
self.dataset_id, self.table_id = dataset_table.split('.')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about project.dataset.table ? You can utilise method from bq library to parse string into database/table reference

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, let me check that.

Copy link
Contributor Author

@subkanthi subkanthi May 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im looking at https://googleapis.dev/python/bigquery/latest/reference.html and I can't find the method to parse string into database/table, can u please point me to the right library.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

table = Table(table_ref=TableReference.from_string(external_project_dataset_table, project_id))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@turbaszek made the change.

Kanthi and others added 5 commits April 25, 2021 09:25
Co-authored-by: Tomek Urbaszek <turbaszek@gmail.com>
Code change based on review comments.

Co-authored-by: Tomek Urbaszek <turbaszek@gmail.com>
@subkanthi subkanthi marked this pull request as ready for review May 1, 2021 02:04
@subkanthi subkanthi requested a review from turbaszek May 1, 2021 13:38
@eladkal eladkal changed the title Subkanthi/bigquery mssql operator 15145 Add BigQueryToMsSqlOperator May 2, 2021
@eladkal eladkal linked an issue May 2, 2021 that may be closed by this pull request
@turbaszek
Copy link
Member

Hey @subkanthi do you need some help?

@subkanthi
Copy link
Contributor Author

Hey @subkanthi do you need some help?
@turbaszek , Thanks, I will make that change today.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@subkanthi
Copy link
Contributor Author

Hey @subkanthi do you need some help?
@turbaszek , Thanks, I will make that change today.

Hi @turbaszek can you please review again, I fixed the unit test that was failing because of the project id, Thanks

Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small typo. but otherwise good

@potiuk potiuk merged commit 7f8f75e into apache:main Jun 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Big Query to MS SQL operator
4 participants