Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bigquery how extract to csv with UTF-8 with BOM #1299

Closed
nosovk opened this issue Feb 3, 2019 · 5 comments
Closed

bigquery how extract to csv with UTF-8 with BOM #1299

nosovk opened this issue Feb 3, 2019 · 5 comments
Assignees
Labels
api: bigquery Issues related to the BigQuery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. type: question Request for information or clarification. Not an issue.

Comments

@nosovk
Copy link

nosovk commented Feb 3, 2019

Big-query allows to export to CSV. Supported encoding is UTF-8.
It's good, because allows to work with non Latin characters.
But unfortunately CSV from BQ do not have BOM. I mean that windows applications like excel or notepad couldn't recognize UTF-8 inside, here is a result:
image
It's not hard to fix an issue adding option to add BOM to export.

Or probably there is some workaround?
I tried to add BOM to column name, to have BOM character be a first letter in first column. Unfortunate for me BQ prohibits usage of special characters in column names.
Related issue: #292

@jeanbza jeanbza added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. type: question Request for information or clarification. Not an issue. api: bigquery Issues related to the BigQuery API. labels Feb 3, 2019
@jeanbza
Copy link
Member

jeanbza commented Feb 3, 2019

Thanks for splitting this out! /cc @JesseLovelace @shollyman @weslem

@shollyman
Copy link
Contributor

Thanks for highlighting I see someone's already pinged https://issuetracker.google.com/issues/35905733 on the public issue tracker for this, which is the right place for changing the behavior of an extract job that writes a CSV output.

The client libraries have no control over whether CSV exports write a UTF8 BOM, but its possible to consume the structured row results form a table/query via the API and then write the CSV output yourself.

@jeanbza
Copy link
Member

jeanbza commented Feb 20, 2019

Closing in favour of https://b.corp.google.com/issues/35905733.

@jeanbza jeanbza closed this as completed Feb 20, 2019
@nosovk
Copy link
Author

nosovk commented Aug 17, 2020

Closing in favour of https://b.corp.google.com/issues/35905733.

this is an internal link, could you please provide current status of an issue?

@codyoss
Copy link
Member

codyoss commented Aug 17, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

4 participants