Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to_html formatter not called for float values in a mixed-type column #13021

Open
patricksurry opened this issue Apr 28, 2016 · 8 comments · Fixed by #25983
Open

to_html formatter not called for float values in a mixed-type column #13021

patricksurry opened this issue Apr 28, 2016 · 8 comments · Fixed by #25983
Labels
Bug IO HTML read_html, to_html, Styler.apply, Styler.applymap

Comments

@patricksurry
Copy link

patricksurry commented Apr 28, 2016

When column x has float64 type, the custom formatter is applied to every value

t = pd.DataFrame([[0], [10.4], [3]], columns=['x'])
display(HTML(t.to_html(formatters=dict(x='xxx'.format))))

   x
------
0  xxx
1  xxx
2  xxx

But when column x is mixed type, the formatter isn't called for (only) floating point values

t = pd.DataFrame([['a'], [0], [10.4], [3]], columns=['x'])
display(HTML(t.to_html(formatters=dict(x='xxx'.format))))

   x
------
0  xxx
1  xxx
2  10.4
3  xxx

output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.11.final.0
python-bits: 64
OS: Darwin
OS-release: 14.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.0
nose: None
pip: 1.5.4
setuptools: 20.7.0
Cython: None
numpy: 1.11.0
scipy: 0.17.0
statsmodels: None
xarray: None
IPython: 4.1.2
sphinx: None
patsy: None
dateutil: 2.4.2
pytz: 2016.3
blosc: None
bottleneck: None
tables: 3.2.2
numexpr: 2.5.2
matplotlib: 1.5.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.12
pymysql: None
psycopg2: None
jinja2: None
boto: None

@jreback
Copy link
Contributor

jreback commented Apr 28, 2016

what exactly are you expecting here?

@jreback jreback added Output-Formatting __repr__ of pandas objects, to_string IO HTML read_html, to_html, Styler.apply, Styler.applymap labels Apr 28, 2016
@patricksurry
Copy link
Author

patricksurry commented Apr 28, 2016

I'm expecting it to call my format function for each value in the column. It doesn't seem to call the formatter at all. Obviously 'xxx'.format is a stupid example - but it illustrates that it's not applying the function to the 10.4 value when it occurs in a mixed-type column. A more realistic formatter would be something like lambda x: x if type(x) is str else '${:,.0f}'.format(x) which displays:

    x
0   a
1   $0
2   10.4
3   $3

when I'd expect $10 for the 3rd row.

@jreback
Copy link
Contributor

jreback commented Apr 28, 2016

how so? show an expected result (with and w/o object columns)

@patricksurry
Copy link
Author

patricksurry commented Apr 28, 2016

def f(x):
    return x if type(x) is str else '${:,.0f}'.format(x)

t = pd.DataFrame([['a'], [0], [10.4], [3]], columns=['x'])
display(HTML(t.to_html(formatters=dict(x=f))))

produces

    x
0   a
1   $0
2   10.4
3   $3

instead of what I expected:

    x
0   a
1   $0
2   $10
3   $3

or am I missing something? (the 'xxx'.format in my example was just to illustrate that it doesn't call my function at all for the 10.4 value)

@patricksurry
Copy link
Author

patricksurry commented Apr 28, 2016

If instead you create the data frame without the 'a' row, so the column type is float64, you get the expected result:

t = pd.DataFrame([[0], [10.4], [3]], columns=['x'])
display(HTML(t.to_html(formatters=dict(x=f))))

    x
0   $0
1   $10
2   $3

@jobevers
Copy link

This is still an issue in 0.21.1

INSTALLED VERSIONS

commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.10.0-42-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.1
pytest: 3.0.7
pip: 9.0.1
setuptools: 36.0.1
Cython: None
numpy: 1.13.3
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 6.1.0
sphinx: 1.6.3
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 1.5.3
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.1.4
pymysql: 0.7.11.None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@jreback
Copy link
Contributor

jreback commented Dec 28, 2017

@jobevers hence the open status

@jreback jreback added this to the Next Major Release milestone Dec 28, 2017
@franz101
Copy link

franz101 commented Nov 6, 2018

Also having problems with formatters

@jreback jreback modified the milestones: Contributions Welcome, 0.25.0 Apr 4, 2019
@jreback jreback reopened this Apr 5, 2019
@jreback jreback modified the milestones: 0.25.0, Contributions Welcome Jul 3, 2019
@mroeschke mroeschke added the Bug label May 7, 2020
@mroeschke mroeschke removed the Output-Formatting __repr__ of pandas objects, to_string label Apr 24, 2021
@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO HTML read_html, to_html, Styler.apply, Styler.applymap
Projects
None yet
6 participants