Cant export to UNICODE CSV file using Export Data Wizard

[lobbko]

Hi everyone,

I need to export multiple tables from PBI Desktop to CSV files. When I do it through query run, with output file I am able to pick option UTF vs. UNICODE, but when I do it through Export Data Wizard, there is no option to change export data type to UNICODE and default is UTF.

Reason why I need UNICODE data type is due to special characters in my native language, which cant be recognized in UTF data type.

Reason why I need to use Export Data Wizard is due to CSV file naming, through this wizard each csv file is named by table name.
Using Output file method and running query
EVALUATE table name 1
EVALUATE table name 2
EVALUATE table name 3

Is saving CSV file based on defined name, not various table names.

Is there possibility to change somehow default export data type to UNICODE using Export Data Wizard?

Thank
Best regards
Marek

[dgosbell]

We probably could add this as an option. But we actually switched the default file format to UTF-8 a number of years ago since it is the more modern format and there should not be any characters it cannot encode. What application are you using to read in this data that you have exported?

Are you able to attach some example csv files exported with the 2 different settings with a few rows which demonstrate some of these problem characters. You could even do something simple like EVALUATE { "example text" } to generate an example row so you don't have to use any real data.

[lobbko]

Hi Darren,
thank you so much for your quick response.
The application I`m opening csvs with is excel. See the comparison in attachment please.
Database contains characters from Slovak language (stands for Slovakia as country).
When I download data from the source, everything is presented well, but problems is when I am exporting it to CSV.
I also tried each combination of regional settings in windows 10 as well as settings of Dax Studio even advance setting when connecting to data model -> locale (all different options = Slovak/EN(US)/EN(UK), with relation to regional settings of windows.

Thanks
Marek

example.xlsx

[dgosbell]

Thanks for the examples, that really helps. Part of the issue appears to be excel assuming that the file is using the 1252 code page. One short term work around is that instead of opening the file directly, open a blank Excel file then in the Data tab choose the From text/CSV option and choose your csv file.

When you do this you will see an window like the following which shows that Excel has chosen the ANSI 1252 codepage to decode the file

But you can change this to the "65001 Unicode UTF-8" code page and the values will be decoded correctly

So it's not that the data is being lost it's just an encoding/decoding mismatch.

But this should just be a short term work around, it should not be difficult for me to add a dropdown to allow you to select the Unicode(UTF-16) vs UTF-8 encoding styles

[lobbko]

Darren,
thank you so much for you response. I just found out on Friday, the workaround you have described, but did not have time to update my previous post, sorry for that.

If the dropdown option is not so difficult for you, I would appreciate it so much, if you would find some time to add this feature up to DAX studio.

Many many thanks for you willingness.
Marek

[dgosbell]

It's not hard at all to add that option. The export to CSV will have the option at the bottom of the following screenshot in the next release. This issue will automatically close once I publish that release.