feat: added labels and tags fetching for BigQuery #93
Conversation
There was a problem hiding this comment.
LGTM already, @manuelzander. Thanks for tackling this. :) Let me know when it's ready to come out of Draft, and I'll run my own local tests and merge.
| if "tags" in tags_dict: | ||
| for tag in tags_dict["tags"]: | ||
| if not "column" in tag: | ||
| table_tag = tag |
There was a problem hiding this comment.
Huh. I didn't know you could do not x in y. I always write it x not in y. Learning something new every day :)
There was a problem hiding this comment.
I could change it to make it clearer for everybody :)
| def test_table_part_of_table_date_range(self, mock_build): | ||
| mock_build.return_value = MockBigQueryClient( | ||
| @patch("whale.extractor.base_bigquery_extractor.datacatalog_v1") | ||
| def test_table_part_of_table_date_range(self, mock_datacatalogue, mock_bigquery): |
There was a problem hiding this comment.
BRITISH ENGLISH!? UNACCEPTABLE!
Just kidding, please don't change it. :)
There was a problem hiding this comment.
Oh haha my bad, I mixed it up 😂 Google style...
@rsyi Are you happy with labels being stored as |
|
Yeah I was wondering about this too, but it seemed like bigquery tags are more like weird, nested json annotations than they are plain string tags, so we can't just throw away everything but the field names... I think this is fine for now. Another option (though a bit more work) could be to create a separate |
|
@rsyi I've just added an additional test (no tags for a given BQ table). |
datacatalog_v1Python client to fetch tags for a given BQ tableTableMetadataandColumnMetadatawhere appropriateTableMetadata