read csv thousands separator

[hayd]

From this SO question, with input to include thousands:

In [1]: s = '06.02.2013;13:00;1.000,215;0,215;0,185;0,205;0,00'

In [2]: pd.read_csv(StringIO(s), sep=';', header=None, parse_dates={'Dates': [0, 1]}, index_col=0, decimal=',')
Out[2]:
                              2      3      4      5  6
Dates
2013-06-02 13:00:00  10.000,215  0.215  0.185  0.205  0

In [3]: pd.read_csv(StringIO(s), sep=';', header=None, parse_dates={'Dates': [0, 1]}, index_col=0, decimal=',', thousands='.')
Out[3]:
                        2      3      4      5  6
Dates
6022013 13:00   1.000,215  0.215  0.185  0.205  0

Note: the Dates column (as well as the thousands not being converted.

[hayd]

Possibly this is a dupe of #2594

[jreback]

this work? http://pandas.pydata.org/pandas-docs/dev/io.html#thousand-separators

[hayd]

@jreback not sure what you're asking?

[jreback]

is this a bug? e.g. the dtypes not working with thousands or a bug in thousands sep? or a feature request (which is there already)?

[hayd]

@jreback this is a bug, thousand separator doesn't seem to work

in the docs you link to it says

For large integers that have been written with a thousands separator, you can set the thousands keyword to True so that integers will be parsed correctly:

in the docstring for read_csv it asks for:

thousands : str, default None
Thousands separator

Doesn't seem to be working with '.', as well as screwing up the dates.

[jreback]

ahh ok...i c now.....

[jreback]

closed by #4598

[hayd]

@jreback The date aspect of this pr is still not fixed. For some reason the thousands separator attacks the date and makes it a string.

[hayd]

Wow, so this is a very edge case... it's cos the date column is just 06.02.2013 which is read as a number 0602013... it's possible dates are sometimes written this way on the continent (along with . thousands): http://en.wikipedia.org/wiki/Date_and_time_notation_in_Europe

Not sure what solution is.

[jreback]

but it should ignore dates columns entirely (for thousands parsing...).....hmmm...why don't you open a separate issue and can cross-link it