Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

assert_frame_equal cannot handle differences in with unicode data #20503

janrito opened this Issue Mar 27, 2018 · 1 comment


None yet
3 participants
Copy link

commented Mar 27, 2018

Code Sample, a copy-pastable example if possible

# Your code here
assert_frame_equal(pd.DataFrame(['á']), pd.DataFrame(['á'])) # => True

assert_frame_equal(pd.DataFrame(['á']), pd.DataFrame(['é']))
UnicodeEncodeError                        Traceback (most recent call last)
<ipython-input-4-b108172c2dda> in <module>()
----> 1 assert_frame_equal(pd.DataFrame(['á']), pd.DataFrame(['é']))

.env/lib/python2.7/site-packages/pandas/util/testing.pyc in assert_frame_equal(left, right, check_dtype, check_index_type, check_column_type, check_frame_type, check_less_precise, check_names, by_blocks, check_exact, check_datetimelike_compat, check_categorical, check_like, obj)
   1311                 check_datetimelike_compat=check_datetimelike_compat,
   1312                 check_categorical=check_categorical,
-> 1313                 obj='DataFrame.iloc[:, {0}]'.format(i))

.env/lib/python2.7/site-packages/pandas/util/testing.pyc in assert_series_equal(left, right, check_dtype, check_index_type, check_series_type, check_less_precise, check_names, check_exact, check_datetimelike_compat, check_categorical, obj)
   1179                                      check_less_precise=check_less_precise,
   1180                                      check_dtype=check_dtype,
-> 1181                                      obj='{0}'.format(obj))
   1183     # metadata comparison

pandas/src/testing.pyx in pandas._testing.assert_almost_equal (pandas/src/testing.c:4156)()

pandas/src/testing.pyx in pandas._testing.assert_almost_equal (pandas/src/testing.c:3274)()

.env/lib/python2.7/site-packages/pandas/util/testing.pyc in raise_assert_detail(obj, message, left, right, diff)
   1011 {1}
   1012 [left]:  {2}
-> 1013 [right]: {3}""".format(obj, message, left, right)
   1015     if diff is not None:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 1: ordinal not in range(128)

Problem description

The difference is caught correctly by assert_frame_equals but it raises an error when trying to print the differences.

Expected Output

AssertionError: DataFrame.iloc[:, 0] are different

DataFrame.iloc[:, 0] values are different (100.0 %)
[left]:  [á]
[right]: [é]

Output of pd.show_versions()

python 2.7.15

commit: None
python-bits: 64
OS: Darwin
OS-release: 17.4.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LOCALE: None.None

pandas: 0.22.0
pytest: None
pip: 9.0.3
setuptools: 39.0.1
Cython: None
numpy: 1.14.2
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.2
pytz: 2018.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

This comment has been minimized.

Copy link

commented Mar 27, 2018

This does work on python 3, PR to fix is welcome!

@chris-b1 chris-b1 added this to the Next Major Release milestone Mar 27, 2018

@jreback jreback modified the milestones: Next Major Release, 0.23.0 Apr 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.