Spanish accents not parsing correctly when importing from CSV file to JSON #169

Closed
AlmogRnD opened this Issue Feb 10, 2015 · 13 comments

Projects

None yet

7 participants

@AlmogRnD

I'm parsing a CSV file with spanish accents, I set the encoding to UTF-8 however it's not parsing correctly

Here is what I'm getting back in the results object
name: "GREGORIO BERNABE �LVAREZ"

and I have the following config setup

Papa.parse(evt.target.files[0], {
header: true,
dynamicTyping: true,
encoding: "utf-8",
}

@mholt
Owner
mholt commented Feb 10, 2015

Is the file really UTF-8 encoded? (And are all its characters UTF-8 encoded as well?) If not, be sure to specify a proper encoding that includes all the characters found in the file.

@AlmogRnD

I think it has to do with the file reader not getting the encoding setting or with the jQuery select casing an issue. Here is my code

<input type="file"  id="csv-file" name="file"/>
function handleFileSelect(evt) {
    if ( !(evt.target && evt.target.files && evt.target.files[0]) ) {
        return;
    }

    Papa.parse(evt.target.files[0], {
        header: true,
        dynamicTyping: true,
        encoding: "UTF-8",
        before: function(file, inputElem)
        {
            console.log(file);
            // executed before parsing each file begins;
            // what you return here controls the flow
        },
        error: function(err, file, inputElem, reason)
        {
            console.log(err);
            // executed if an error occurs while loading the file,
            // or if before callback aborted for some reason
        },
        complete: function (results) {
            renderDataSet(results);
        }
    });
}
@mholt
Owner
mholt commented Feb 10, 2015

I've confirmed that the file reader is indeed getting the encoding setting correctly. What's your input file look like?

@AlmogRnD

I'm saving the file as CSV

@mholt
Owner
mholt commented Feb 10, 2015

I mean what are its contents? Until I can reproduce the problem I can't produce a fix.

@AlmogRnD

Ok the issue is with Microsoft excel it doesn't save it in UTF-8 format - http://stackoverflow.com/questions/4221176/excel-to-csv-with-utf8-encoding

I just copied everything into google sheets and saved it there.

@AlmogRnD AlmogRnD closed this Feb 10, 2015
@mholt
Owner
mholt commented Feb 10, 2015

Ah, good to know. You're not the first that has had that problem - for example, #64. Maybe I will make this an FAQ on the website.

Glad you got it working! And thanks for your reports.

@AlmogRnD

Yea I would recommend adding this to the docs with some more info.

@fabiocbinbutter

Deleted my last comment, that Excel workaround does nothing...

@Benczyk
Benczyk commented Feb 27, 2015

Had the same problem and lost a lot of time due to this weird Excel bug.

What works for me: instead of exporting your Excel to CSV, save the file as .txt UTF-16
Just tested a file with the papaparse demo, and it works (with or without header).

(found the solution lost in the middle of this stackoverflow thread http://stackoverflow.com/questions/4221176/excel-to-csv-with-utf8-encoding)

@arebena
arebena commented Aug 6, 2015

Try to put a line like :

encoding: "ISO-8859-1"

in your papa config.
Works for me...

@rwilliams-scottlogic

Looked into this as I was using it, some notes that may help people:

  • Excel will save in the system locale code page. For our users this will be Windows-1252.
  • So it should follow that we set this to "CP1252". This works in Chrome and Firefox but not IE10 and IE11.
  • Setting this to ISO-8859-1 (of which CP1252 is a superset) strangely works in all browsers, including for characters which exist in CP1252 but not ISO-8859-1.
@qroft
qroft commented Jul 20, 2016

I am still having trouble in loading the 2 CSV files correctly form this page:
http://www.coflp.org/descargas/
If i download the "Turnos de Guardia.csv" i can see that the format is correct but within the console.log the special characters show broken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment