Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bigquery: SchemaFromJSON doesn't support policyTags #5867

Closed
Nabil372 opened this issue Apr 7, 2022 · 1 comment · Fixed by #5877
Closed

bigquery: SchemaFromJSON doesn't support policyTags #5867

Nabil372 opened this issue Apr 7, 2022 · 1 comment · Fixed by #5877
Assignees
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@Nabil372
Copy link

Nabil372 commented Apr 7, 2022

Hi there!

SchemaFromJSON ignores policyTags in JSONs which are generated by protoc-gen-bq-schema.
This function no longer does what it's described to do:
"SchemaFromJSON takes a JSON BigQuery table schema definition (as generated by https://github.com/GoogleCloudPlatform/protoc-gen-bq-schema) and returns a fully-populated Schema."

Example JSON taken from protoc-gen-bq-schema:

[
 {
  "name": "a",
  "type": "INTEGER",
  "mode": "REQUIRED",
  "policyTags": {
   "names": [
    "private"
   ]
  }
 },
 {
  "name": "b",
  "type": "STRING",
  "mode": "NULLABLE",
  "policyTags": {
   "names": [
    "public"
   ]
  }
 },
 {
  "name": "nested",
  "type": "RECORD",
  "mode": "REQUIRED",
  "fields": [
   {
    "name": "a",
    "type": "INTEGER",
    "mode": "REQUIRED",
    "policyTags": {
     "names": [
      "private"
     ]
    }
   },
   {
    "name": "b",
    "type": "STRING",
    "mode": "NULLABLE"
   }
  ]
 }
]

The function is still able to produce a bigquery.Schema but it ignores the policyTags.

@Nabil372 Nabil372 added the triage me I really want to be triaged. label Apr 7, 2022
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the BigQuery API. label Apr 7, 2022
@shollyman
Copy link
Contributor

Will take a look at this as a FR.

@shollyman shollyman added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. priority: p2 Moderately-important priority. Fix may not be included in next release. and removed triage me I really want to be triaged. labels Apr 8, 2022
shollyman added a commit to shollyman/google-cloud-go that referenced this issue Apr 8, 2022
This PR does two things: It enhances SchemaFromJSON to
work directly with the underlying TableFieldSchema messages
from the discovery API definition, and adds a FormatJSONFields
method to Schema to export the same format consumed by SchemaFromJSON.

With this, we're able clear up the existing internal duplicate logic
for this special case, and we manage to address two different feature
requests at the same time.

Fixes: googleapis#5833
Fixes: googleapis#5867
shollyman added a commit that referenced this issue Apr 12, 2022
* feat(bigquery): enhance SchemaFromJSON

This PR does two things: It enhances SchemaFromJSON to
work directly with the underlying TableFieldSchema messages
from the discovery API definition, and adds a FormatJSONFields
method to Schema to export the same format consumed by SchemaFromJSON.

With this, we're able clear up the existing internal duplicate logic
for this special case, and we manage to address two different feature
requests at the same time.

Fixes: #5833
Fixes: #5867


Co-authored-by: Steffany Brown <30247553+steffnay@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants