Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: fix floating precision formatting in presence of inf #24863

Merged
merged 3 commits into from Jan 23, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.24.0.rst
Expand Up @@ -1751,6 +1751,7 @@ I/O
- Bug in :meth:`DataFrame.to_stata`, :class:`pandas.io.stata.StataWriter` and :class:`pandas.io.stata.StataWriter117` where a exception would leave a partially written and invalid dta file (:issue:`23573`)
- Bug in :meth:`DataFrame.to_stata` and :class:`pandas.io.stata.StataWriter117` that produced invalid files when using strLs with non-ASCII characters (:issue:`23573`)
- Bug in :class:`HDFStore` that caused it to raise ``ValueError`` when reading a Dataframe in Python 3 from fixed format written in Python 2 (:issue:`24510`)
- Bug in :func:`DataFrame.to_string()` and more generally in the floating ``repr`` formatter. Zeros were not trimmed if ``inf`` was present in a columns while it was the case with NA values. Zeros are now trimmed as in the presence of NA (:issue:`24861`).

Plotting
^^^^^^^^
Expand Down
14 changes: 9 additions & 5 deletions pandas/io/formats/format.py
Expand Up @@ -1414,16 +1414,20 @@ def _trim_zeros(str_floats, na_rep='NaN'):
"""
trimmed = str_floats

def _is_number(x):
return (x != na_rep and not x.endswith('inf'))

def _cond(values):
non_na = [x for x in values if x != na_rep]
return (len(non_na) > 0 and all(x.endswith('0') for x in non_na) and
not (any(('e' in x) or ('E' in x) for x in non_na)))
finite = [x for x in values if _is_number(x)]
return (len(finite) > 0 and all(x.endswith('0') for x in finite) and
not (any(('e' in x) or ('E' in x) for x in finite)))

while _cond(trimmed):
trimmed = [x[:-1] if x != na_rep else x for x in trimmed]
trimmed = [x[:-1] if _is_number(x) else x for x in trimmed]

# leave one 0 after the decimal points if need be.
return [x + "0" if x.endswith('.') and x != na_rep else x for x in trimmed]
return [x + "0" if x.endswith('.') and _is_number(x) else x
for x in trimmed]


def _has_names(index):
Expand Down
33 changes: 33 additions & 0 deletions pandas/tests/io/formats/test_format.py
Expand Up @@ -1465,6 +1465,39 @@ def test_to_string_format_na(self):
'4 4.0 bar')
assert result == expected

def test_to_string_format_inf(self):
# Issue #24861
tm.reset_display_options()
df = DataFrame({
'A': [-np.inf, np.inf, -1, -2.1234, 3, 4],
'B': [-np.inf, np.inf, 'foo', 'foooo', 'fooooo', 'bar']
})
result = df.to_string()

expected = (' A B\n'
'0 -inf -inf\n'
'1 inf inf\n'
'2 -1.0000 foo\n'
'3 -2.1234 foooo\n'
'4 3.0000 fooooo\n'
'5 4.0000 bar')
assert result == expected

df = DataFrame({
'A': [-np.inf, np.inf, -1., -2., 3., 4.],
'B': [-np.inf, np.inf, 'foo', 'foooo', 'fooooo', 'bar']
})
result = df.to_string()

expected = (' A B\n'
'0 -inf -inf\n'
'1 inf inf\n'
'2 -1.0 foo\n'
'3 -2.0 foooo\n'
'4 3.0 fooooo\n'
'5 4.0 bar')
assert result == expected

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a general principle, we try to include the original example from the issue in the tests as well, if possible. I'm not sure I see that here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original issue was with rounding (I suppose that was how the issue was discovered), but the rounding itself is not related to it, so I think this test case is actually better.

def test_to_string_decimal(self):
# Issue #23614
df = DataFrame({'A': [6.0, 3.1, 2.2]})
Expand Down