Skip to content

FileSystem.create does not pass full Metadata to GCS #21046

@damccorm

Description

@damccorm

When I call FileSystem.create(foo, mime_type="text/plain", compression_type=CompressionTypes.BZIP2) I would expect the following metadata set on GCS Blob:
bq. content_type: text/plain
bq. content_encoding: application/bzip2

What we currently get is:
bq. content_type: application/x-bz2

Looking at the code it seems to be a problem with GcsIO interface, which does not seem to expose content_type and content_encoding properly.

Imported from Jira BEAM-12714. Original Jira may contain additional context.
Reported by: abergmeier.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions