-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Open
Labels
api: bigquerystorageIssues related to the BigQuery Storage API.Issues related to the BigQuery Storage API.priority: p3Desirable enhancement or fix. May not be included in next release.Desirable enhancement or fix. May not be included in next release.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.‘Nice-to-have’ improvement, new feature or different behavior or design.
Description
Is your feature request related to a problem? Please describe.
The java implementation can convert bigquery's schema to proto and write json without preparing proto files.
I think it would be very convenient to have something equivalent to these in the python implementation.
References:
- https://cloud.google.com/bigquery/docs/write-api-streaming
- https://github.com/googleapis/java-bigquerystorage/blob/main/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQTableSchemaToProtoDescriptor.java
Describe the solution you'd like
I would like to add API implementation or sample code for it.
It will look like this (although I haven't actually run it).
bq_table_schema_type_map = {
TableFieldSchema.Type.BOOL: FieldDescriptorProto.Type.TYPE_BOOL,
...
}
bq_table_schema_mode_map = {
TableFieldSchema.Mode.NULLABLE: FieldDescriptorProto.Label.LABEL_OPTIONAL,
...
}
def convert_bq_field_to_proto_field(bq_field: TableFieldSchema, index: int) -> FieldDescriptorProto:
mode = bq_field.mode
field_name = bq_field.name
type_ = bq_field.type_
return FieldDescriptorProto(
name=field_name,
type=bq_table_schema_type_map[type_],
label=bq_table_schema_mode_map[mode],
number=index
)
def convert_bq_table_schema_to_descriptor_proto(table_schema: TableSchema, scope: str,
dependency_map: Dict = None) -> DescriptorProto:
index = 1
fields: List[FieldDescriptorProto] = []
for bq_field in table_schema.fields:
index += 1
fields.append(convert_bq_field_to_proto_field(bq_field, index))
descriptor_proto = DescriptorProto(name=scope, field=fields)
return descriptor_proto
...Are there any plans to add such a feature? Also, would it be possible for me to contribute?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
api: bigquerystorageIssues related to the BigQuery Storage API.Issues related to the BigQuery Storage API.priority: p3Desirable enhancement or fix. May not be included in next release.Desirable enhancement or fix. May not be included in next release.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.‘Nice-to-have’ improvement, new feature or different behavior or design.