Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve speed when listing table comments in BigQuery #21581

Merged
merged 1 commit into from
Apr 30, 2024

Conversation

ebyhr
Copy link
Member

@ebyhr ebyhr commented Apr 17, 2024

Description

Test results with 46 tables:

SELECT * FROM system.metadata.table_comments WHERE schema_name = 'test';
16s → 2s

Release notes

(x) Release notes are required, with the following suggested text:

# BigQuery
* Improve speed when listing table comments. ({issue}`21581`)

@cla-bot cla-bot bot added the cla-signed label Apr 17, 2024
@github-actions github-actions bot added the bigquery BigQuery connector label Apr 17, 2024
@ebyhr ebyhr force-pushed the ebi/bigquery-bulk-table-comments branch 5 times, most recently from 8fe0a80 to cef7a46 Compare April 17, 2024 07:41
@ebyhr ebyhr marked this pull request as ready for review April 17, 2024 08:26
@ebyhr ebyhr marked this pull request as draft April 17, 2024 08:32
@ebyhr ebyhr force-pushed the ebi/bigquery-bulk-table-comments branch 2 times, most recently from 5546249 to bf0e99f Compare April 22, 2024 12:16
Comment on lines 418 to 422
private static String unquote(String quoted)
{
return quoted.substring(1, quoted.length() - 1)
.replace("\"\"", "\"");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The option_value is quoted as BigQuery official documentation says: https://cloud.google.com/bigquery/docs/information-schema-table-options#example_2

Special characters are tested in BaseConnectorTest#testCommentTableSpecialCharacter().

@ebyhr ebyhr marked this pull request as ready for review April 22, 2024 12:22
@ebyhr ebyhr requested review from wendigo and hashhar April 22, 2024 12:24
@findinpath findinpath requested a review from pajaks April 23, 2024 07:06
@wendigo wendigo removed their request for review April 25, 2024 07:40
@ebyhr ebyhr force-pushed the ebi/bigquery-bulk-table-comments branch from bf0e99f to e4c9e33 Compare April 25, 2024 07:49
@ebyhr ebyhr force-pushed the ebi/bigquery-bulk-table-comments branch from e4c9e33 to 9fe2a53 Compare April 25, 2024 07:58
Copy link
Member

@hashhar hashhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you.

@ebyhr ebyhr force-pushed the ebi/bigquery-bulk-table-comments branch 3 times, most recently from fa421f9 to 35602ca Compare April 25, 2024 10:03
@ebyhr ebyhr force-pushed the ebi/bigquery-bulk-table-comments branch from 35602ca to 4291c1b Compare April 30, 2024 23:09
@ebyhr ebyhr merged commit 15eb33d into master Apr 30, 2024
19 checks passed
@ebyhr ebyhr deleted the ebi/bigquery-bulk-table-comments branch April 30, 2024 23:46
@github-actions github-actions bot added this to the 446 milestone Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bigquery BigQuery connector cla-signed
Development

Successfully merging this pull request may close these issues.

None yet

4 participants