When exporting to a CSV with unicode characters, contents can become garbled

[zhanghw89]

Describe the bug

when i export a csv with Chinese word ,it has a Chinese garbled. the version is V1.2.10

[netniV]

Please give more description and steps that you have taken.

[TheWitness]

Seems we've seen this one before. The CSV output is likely not UTF8.

[TheWitness]

Here is one article: https://stackoverflow.com/questions/4348802/how-can-i-output-a-utf-8-csv-in-php-that-excel-will-read-properly

Then I found this on secure.php.net

if you want make UTF-8 file for excel, use this:

$fp = fopen($filename, 'w');
//add BOM to fix UTF-8 in Excel
fputs($fp, $bom =( chr(0xEF) . chr(0xBB) . chr(0xBF) ));

[TheWitness]

And this one looks much more elegant:

   
    $data = "Some utf-8 characters ąćżźćł" 
     
    header("Content-Type: application/octet-stream"); 
    header("Content-Transfer-Encoding: binary"); 
    header('Expires: '.gmdate('D, d M Y H:i:s').' GMT'); 
    header('Content-Disposition: attachment; filename = "Export '.date("Y-m-d").'.tsv"'); 
    header('Pragma: no-cache'); 
     
    //these characters will make correct encoding to excel 
    echo chr(255).chr(254).iconv("UTF-8", "UTF-16LE//IGNORE", $data); 

[TheWitness]

Please test the latest commit in the 1.2.x branch for graph_xport.php. Excel does not detect it as CSV any longer, at least not automatically, but it still looks right.

[TheWitness]

Any feedback?

[TheWitness]

@zhanghw89, does it work for you?

[zhanghw89]

I finally solved the problem based on this url. https://kindevil.com/archives/552.html

[TheWitness]

Please upload your modified file.

[netniV]

@TheWitness, I would agree with the diagnosis and fix so far. If it needs a further review, tag me.

[TheWitness]

I'm not happy with the fix at the point. It's able to export, but Excel no longer thinks the file is CSV, and you have to use text to columns to have Excel process the file like a normal ASCII CSV file.

[TheWitness]

Just went to the site recommend, made the one small update, and now it's loading as CSV just fine with the correct encoding.