bigquery how extract to csv with UTF-8 with BOM

[albertog]

Will be very very interesting add posibility to add BOM byte sequence to file extracted from bigquery in order to be reader properly with excel. Now the csv exported work ok but manually you need to choose utf-8.

Similarity you can choose a custom delimiter will be nice that we can choice encoding w/o BOM and emit a csv compatible with excel. thanks

probably 80% of bigquery csv output will be reader by ms excel.

see:
http://stackoverflow.com/questions/2223882/whats-different-between-utf-8-and-utf-8-without-bom

[bradfitz]

/cc @robpike

[robpike]

Are you saying that Excel only works with UTF-8 if there is a leading BOM? That is rebarbative.

[jba]

Alberto, I don't think we will change BigQuery to implement something that is not recommended. But it should be easy to do this yourself: once you download the csv, just prepend the three BOM bytes to the file.

[albertog]

No @robpike I am saying that if your extract a bigquery csv file and open directly from Excel character as "ñ, º, € , ü, etc " appear strange and end user "not programer" think that underling data is erroneous and create a bad experience and issue. If you manually import selecting a right encoding utf-8 work propertly but end user must be explicitly select a type of encoding something that not every end user know.

@jba I understand but the fact is that excel use this encoding and it a standard in windows office environment currently so we are requesting a option and of course a feature of api not a default behaviour to improve end user experience I assume you can post-process the file but you need add logic, spent resources, complexity and time.

Thanks for answer me.

[okdave]

@albertog I don't think this is a feasible request for this client library since the actual creation of the CSV file happens in the BigQuery service itself, and the Go code doesn't get to influence how it's made. I'd suggest opening a feature request on the google-bigquery issue tracker. I don't know if they would want to support a non-standard CSV encoding, but maybe there would be an option to support an Excel-specific output.

If you're using the google.golang.org/cloud/storage library to actually download the extracted CSV onto the local machine, then it should be simple to prepend the three bytes yourself, or to re-encode the file from UTF-8 into the machine's default encoding (which, I understand, varies between windows installations).

[albertog]

okdave it make sense. I will do it

[marcosdiez]

Currently it's not possible to do it. There is a bug request for it on Google. If you are affected by this issue, please go there and star it: https://issuetracker.google.com/issues/35905733

[nosovk]

Is it possible to start first column name with BOM character?

[jeanbza]

@nosovk This thread is dead. Do you mind opening a new issue?