Skip to content

Conversion from bq schema to proto #14277

@richwomanbtc

Description

@richwomanbtc

Is your feature request related to a problem? Please describe.

The java implementation can convert bigquery's schema to proto and write json without preparing proto files.
I think it would be very convenient to have something equivalent to these in the python implementation.
References:

Describe the solution you'd like

I would like to add API implementation or sample code for it.
It will look like this (although I haven't actually run it).

bq_table_schema_type_map = {
    TableFieldSchema.Type.BOOL: FieldDescriptorProto.Type.TYPE_BOOL,
    ...
}

bq_table_schema_mode_map = {
    TableFieldSchema.Mode.NULLABLE: FieldDescriptorProto.Label.LABEL_OPTIONAL,
    ...
}

def convert_bq_field_to_proto_field(bq_field: TableFieldSchema, index: int) -> FieldDescriptorProto:
    mode = bq_field.mode
    field_name = bq_field.name
    type_ = bq_field.type_
    return FieldDescriptorProto(
        name=field_name,
        type=bq_table_schema_type_map[type_],
        label=bq_table_schema_mode_map[mode],
        number=index
    )

def convert_bq_table_schema_to_descriptor_proto(table_schema: TableSchema, scope: str,
                                                dependency_map: Dict = None) -> DescriptorProto:
    index = 1
    fields: List[FieldDescriptorProto] = []
    for bq_field in table_schema.fields:
        index += 1
        fields.append(convert_bq_field_to_proto_field(bq_field, index))

    descriptor_proto = DescriptorProto(name=scope, field=fields)
    return descriptor_proto

...

Are there any plans to add such a feature? Also, would it be possible for me to contribute?

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: bigquerystorageIssues related to the BigQuery Storage API.priority: p3Desirable enhancement or fix. May not be included in next release.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions