to_clipboard is no longer Excel compatible

[dalito]

Using pandas-0.18.0rc1-cp27-cp27m-win_amd64.whl from Christoh Gohlkes site:

import pandas as pd
df = pd.DataFrame({u'a':[1,2], u'degC_°C':[3,4]})
df.to_clipboard()
df.to_clipboard(excel=True, sep='\t')
df.to_clipboard(excel=False, sep='\t')
df.to_clipboard(sep=',')

Always gives the same content in the clipboard:

   a  degC_°C
0  1        3
1  2        4

which cannot be pasted to Excel since it does not use tab as separator.
Copy/paste worked fine in pandas 0.17.1 except for unicode characters.

Output of pd.show_versions():

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.11.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: AMD64 Family 22 Model 0 Stepping 1, AuthenticAMD
byteorder: little
LC_ALL: None
LANG: de_DE

pandas: 0.18.0rc1
nose: None
pip: 8.0.2
setuptools: 19.6.1
Cython: 0.23.4
numpy: 1.10.4
scipy: 0.17.0
statsmodels: None
xarray: None
IPython: 4.1.0rc1
sphinx: 1.3.5
patsy: None
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8

[jreback]

Copy/paste worked fine in pandas 0.17.1 except for unicode characters.

and the clipboard routines have been updated to use unicode, see #9263

[jreback]

AFAICT this didn't work previously. Is the issue that the sep is not getting passed?

[dalito]

Yes, the new issue in 0.18.0rc1 (vs. 0.17.x) is that the clipboard content does no longer have a separator (only spaces). Even when I explicitly pass a separator by using the keyword argument, that separator is not used.

[jreback]

ok, certainly could be an issue, can you step thru it and see where its not being passed.

[jorisvandenbossche]

Note that this only happens if there are unicode characters, if you leave out the °, it still works as expected using tabs

[dalito]

OK. I have stepped through the sequence of calls and found out that all works fine if an encoding is specified when calling to_clipboard:

df = pd.DataFrame({u'a':[1,2], u'degC_°C':[3,4]})
df.to_clipboard(encoding='cp1252')

Without specifying an encoding, an UnicodeEncodeError ('ascii' codec can't encode character...) is raised in to_csv which is called from to_clipboard here. The error is suppressed so that the user has no idea what went wrong and that an encoding should be passed as kwarg to "to_clipboard". Another consequence of suppressing the error is that a separator passed as kwarg is ignored.

I am not sure how to fix this best. Either an error could be raised with a message that an encoding is needed or a default encoding could be used (cp1252 for windows utf8 for all other?).

[jreback]

ahh on

why don't we catch the encoding error then raise a helpful message that u need to pass an encoding (rather than just going on)

[dalito]

Yes that is what I though, too. I'll prepare a PR.