New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Followup to Conditional HTML Styling #11610

Open
TomAugspurger opened this Issue Nov 16, 2015 · 40 comments

Comments

Projects
None yet
10 participants
@TomAugspurger
Contributor

TomAugspurger commented Nov 16, 2015

Follows #10250

For 0.17.1

  • Nicer table styling for the pydata.org website
  • Include a visual example in 0.17.1.rst
  • remove doc/source/html-styling.html and find a way to include doc/source/html-styling.ipynb in the doc build (should use --template=basic)
  • update print_versions
  • update requirements_all.txt to include jinja

For 0.18.0 / Future

  • sparsify MultiIndex repr (maybe push till 0.18)
  • easy alignment styles #12144
  • Template modification: This is for things like wrapping base64 encoded values in img tags, urls, etc. flows into...
  • Break the large template in Styler.template into smaller blocks. Let people extend that. We could (maybe) allow users to choose which template to use to render each column/cell with solving the template modification problem
  • Truncated repr
  • hook into pd.options, allow setting of default reprs with styles
  • Refactor parts of Styler into a BaseStyler, maybe add a LaTeX styler (maybe deprecate / replace the to_html and to_latex methods; Jinja templates are much more pleasant to work with), xref #11700
  • Categoricals / Boolens builtin stylings

will add more as we go.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Nov 16, 2015

Contributor

@TomAugspurger I had to slightly change style.rst to get this to appear in the index. You may want to further modify.

Contributor

jreback commented Nov 16, 2015

@TomAugspurger I had to slightly change style.rst to get this to appear in the index. You may want to further modify.

@jankatins

This comment has been minimized.

Show comment
Hide comment
@jankatins

jankatins Nov 16, 2015

Contributor

A comment on the API of the highlighter:

def color_negative_red(val):
    """
    Takes a scalar and returns a string with
    the css property `'color: red'` for negative
    strings, black otherwise.
    """
    color = 'red' if val < 0 else 'black'
    return 'color: %s' % color

This basically only works for html representation ("color: red" is css speak) and assumes that the return value should got to the css.

In latex (see e.g. this example), you prefix/suround the value with a command:

[...]
\usepackage[table]{xcolor}% http://ctan.org/pkg/xcolor
  Some & \cellcolor{blue!25}coloured & contents \\

would render the second cell blue (by using a command from a special package).

So for latex you probably need templates ala \cellcolor{blue!25}%s or \textbf{%s} (which make the value bold).

Contributor

jankatins commented Nov 16, 2015

A comment on the API of the highlighter:

def color_negative_red(val):
    """
    Takes a scalar and returns a string with
    the css property `'color: red'` for negative
    strings, black otherwise.
    """
    color = 'red' if val < 0 else 'black'
    return 'color: %s' % color

This basically only works for html representation ("color: red" is css speak) and assumes that the return value should got to the css.

In latex (see e.g. this example), you prefix/suround the value with a command:

[...]
\usepackage[table]{xcolor}% http://ctan.org/pkg/xcolor
  Some & \cellcolor{blue!25}coloured & contents \\

would render the second cell blue (by using a command from a special package).

So for latex you probably need templates ala \cellcolor{blue!25}%s or \textbf{%s} (which make the value bold).

@kynan

This comment has been minimized.

Show comment
Hide comment
@kynan

kynan Nov 16, 2015

@TomAugspurger I think you made a great start on this! A few ideas for making this approach potentially more flexible:

  1. Allow extra attributes on the <table>.

    Common use case: You want to export an html table with sortable column, e.g. using sortable. For that you would need the following opening tag:

    <table class="sortable-theme-bootstrap" data-sortable>
    
  2. Support "external" styling via existing custom CSS style sheets.

    For this to work, the current approach with unique ids per cell which are then targeted with a auto generated CSS doesn't really work. Instead it would be useful to be able to assign additional classes (or data attributes) to cells based on the Styler rules.

    Example use case: assign class positive to all values > 0 and negative to all values < 0

    A potential advantage of this would be much smaller size of generated code (since we don't need a custom CSS block for each cell) and better performance when rendering in the browser (fewer rules to apply).

  3. Allow setting arbitrary attributes on table cells based on rules.

This extends the previous suggestion and would allow using exported html tables with any kind of JavaScript library using specific attributes.

Sorry for being so late in making these suggestions - I didn't manage to read through all of #10250 before it was merged.

kynan commented Nov 16, 2015

@TomAugspurger I think you made a great start on this! A few ideas for making this approach potentially more flexible:

  1. Allow extra attributes on the <table>.

    Common use case: You want to export an html table with sortable column, e.g. using sortable. For that you would need the following opening tag:

    <table class="sortable-theme-bootstrap" data-sortable>
    
  2. Support "external" styling via existing custom CSS style sheets.

    For this to work, the current approach with unique ids per cell which are then targeted with a auto generated CSS doesn't really work. Instead it would be useful to be able to assign additional classes (or data attributes) to cells based on the Styler rules.

    Example use case: assign class positive to all values > 0 and negative to all values < 0

    A potential advantage of this would be much smaller size of generated code (since we don't need a custom CSS block for each cell) and better performance when rendering in the browser (fewer rules to apply).

  3. Allow setting arbitrary attributes on table cells based on rules.

This extends the previous suggestion and would allow using exported html tables with any kind of JavaScript library using specific attributes.

Sorry for being so late in making these suggestions - I didn't manage to read through all of #10250 before it was merged.

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Nov 16, 2015

Contributor

@kynan fantastic, thanks for the feedback. I'll go through it in more detail later.

Your item 1. sounds pretty simple. Is attribute the correct term for the items in the opening tag? <table class="sortable-theme-bootstrap" data-sortable> We could include a method like .set_table_attributes for that.

Contributor

TomAugspurger commented Nov 16, 2015

@kynan fantastic, thanks for the feedback. I'll go through it in more detail later.

Your item 1. sounds pretty simple. Is attribute the correct term for the items in the opening tag? <table class="sortable-theme-bootstrap" data-sortable> We could include a method like .set_table_attributes for that.

@kynan

This comment has been minimized.

Show comment
Hide comment
@kynan

kynan Nov 16, 2015

@TomAugspurger I believe attribute is the common term and also the one the W3C uses. set_table_attributes sounds sensible to me.

kynan commented Nov 16, 2015

@TomAugspurger I believe attribute is the common term and also the one the W3C uses. set_table_attributes sounds sensible to me.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Nov 18, 2015

Contributor

@TomAugspurger merged your PR; I'll leave you to close when you are ready.

Contributor

jreback commented Nov 18, 2015

@TomAugspurger merged your PR; I'll leave you to close when you are ready.

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Nov 18, 2015

Contributor

Thanks, I want to get a better solution in place for including notebooks in the sphinx build, but that works for now.

@kynan, for your second item, assigning classes to cells. My current thinking is to have a method on Styler (would it be terrible to call it .classify?) that takes a function to be evaluated and a class to assign to the cells where that function evaluates to True. The limitation here is that the class that's assigned doesn't get to refer to the data. So you couldn't (easily) do something like our .background_gradient, which is why I discarded this approach originally. But it might make sense to have in addition to the one-class-per cell approach we have. This will probably need to wait for the next release though.

Contributor

TomAugspurger commented Nov 18, 2015

Thanks, I want to get a better solution in place for including notebooks in the sphinx build, but that works for now.

@kynan, for your second item, assigning classes to cells. My current thinking is to have a method on Styler (would it be terrible to call it .classify?) that takes a function to be evaluated and a class to assign to the cells where that function evaluates to True. The limitation here is that the class that's assigned doesn't get to refer to the data. So you couldn't (easily) do something like our .background_gradient, which is why I discarded this approach originally. But it might make sense to have in addition to the one-class-per cell approach we have. This will probably need to wait for the next release though.

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Nov 18, 2015

Contributor

docs are up if anyone sees anything. It does still have the [In] and [Out] tags and ¶ markers I might be able to hide.

Contributor

TomAugspurger commented Nov 18, 2015

docs are up if anyone sees anything. It does still have the [In] and [Out] tags and ¶ markers I might be able to hide.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Nov 18, 2015

Contributor

@TomAugspurger yep look great!

on the css side, I think its possible if we tag with the SAME names, e.g.

df.style.highlight_null(css='null_class').background_gradient(css='gradients_class')

then as long as you tag THOSE cells with that class it would work. we could have default class names (based on the function name), and have this kw to override.

Contributor

jreback commented Nov 18, 2015

@TomAugspurger yep look great!

on the css side, I think its possible if we tag with the SAME names, e.g.

df.style.highlight_null(css='null_class').background_gradient(css='gradients_class')

then as long as you tag THOSE cells with that class it would work. we could have default class names (based on the function name), and have this kw to override.

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Nov 18, 2015

Contributor

The df.style.highlight_null(css='null_class') I could see working like that, since that fits the binary "if this condition is true, apply this style".

This could be my limited understanding of CSS, but I don't see how you could accomplish df.style.background_gradient(css='gradients_class') in CSS, just knowing that these columns have this class. (I think) you'd need a class per color you want to assign.

I suppose we could add a data attribute to each cell with the value of that cell... You might be able to pull off some CSS wizardry to accomplish it in that case, but I don't see the average python use being able to write or customize that.

Contributor

TomAugspurger commented Nov 18, 2015

The df.style.highlight_null(css='null_class') I could see working like that, since that fits the binary "if this condition is true, apply this style".

This could be my limited understanding of CSS, but I don't see how you could accomplish df.style.background_gradient(css='gradients_class') in CSS, just knowing that these columns have this class. (I think) you'd need a class per color you want to assign.

I suppose we could add a data attribute to each cell with the value of that cell... You might be able to pull off some CSS wizardry to accomplish it in that case, but I don't see the average python use being able to write or customize that.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Nov 18, 2015

Contributor

@TomAugspurger I think you would actually construct the classes WITH the in this case the level embedded, (for some you wouldn't need to do this), maybe something like 'gradient_level_0_class` (e.g. say you ten levels of gradient. but this is a refinement.

Contributor

jreback commented Nov 18, 2015

@TomAugspurger I think you would actually construct the classes WITH the in this case the level embedded, (for some you wouldn't need to do this), maybe something like 'gradient_level_0_class` (e.g. say you ten levels of gradient. but this is a refinement.

@jreback jreback modified the milestones: Next Major Release, 0.17.1 Nov 18, 2015

@mattilyra

This comment has been minimized.

Show comment
Hide comment
@mattilyra

mattilyra Nov 20, 2015

I've been playing around with the new styling features and have a few comments, overall this is a great new addition.

The highlight_min, highlight_max and highlight_null would be a lot better if instead of taking a color argument they would actually take the css format string or **kwargs that correspond to css style names - this would allow:

  1. using inverted colors (background-color: black; color: white)
  2. other formatting options than background shading (font-weight: bold)

The documentation is also a little confusing in terms of debugging the styling functions.

Debugging Tip: If you're having trouble writing your style function, try just passing it into df.apply. Styler.apply uses that internally, so the result should be the same.```

The full stop and space between apply and Styler is somewhat difficult to spot (I only noticed it when pasting the quote here) - I was looking for df.apply.Styler.apply which obviously doesn't exist, a little rewording would fix this. (You need to look at the text at https://pandas-docs.github.io/pandas-docs-travis/style.html to spot it)

Debugging Tip: If you're having trouble writing your style function, try passing it into df.apply. Internally Styler.apply uses that, so the result should be the same.```

mattilyra commented Nov 20, 2015

I've been playing around with the new styling features and have a few comments, overall this is a great new addition.

The highlight_min, highlight_max and highlight_null would be a lot better if instead of taking a color argument they would actually take the css format string or **kwargs that correspond to css style names - this would allow:

  1. using inverted colors (background-color: black; color: white)
  2. other formatting options than background shading (font-weight: bold)

The documentation is also a little confusing in terms of debugging the styling functions.

Debugging Tip: If you're having trouble writing your style function, try just passing it into df.apply. Styler.apply uses that internally, so the result should be the same.```

The full stop and space between apply and Styler is somewhat difficult to spot (I only noticed it when pasting the quote here) - I was looking for df.apply.Styler.apply which obviously doesn't exist, a little rewording would fix this. (You need to look at the text at https://pandas-docs.github.io/pandas-docs-travis/style.html to spot it)

Debugging Tip: If you're having trouble writing your style function, try passing it into df.apply. Internally Styler.apply uses that, so the result should be the same.```

@jreback jreback modified the milestones: 0.18.0, Next Major Release Nov 20, 2015

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Nov 20, 2015

Contributor

On your first point, I agree that would be useful. If we end up going with a .classify that takes a function returning booleans (pd.isnull) and assigns classes where that's True, we should be able to handle that pretty easily. I think we should hold off adding more keywords to .highlight_null until we decide what to do there.

I just pushed a PR to clarify the documentation. That was confusing, thanks.

Contributor

TomAugspurger commented Nov 20, 2015

On your first point, I agree that would be useful. If we end up going with a .classify that takes a function returning booleans (pd.isnull) and assigns classes where that's True, we should be able to handle that pretty easily. I think we should hold off adding more keywords to .highlight_null until we decide what to do there.

I just pushed a PR to clarify the documentation. That was confusing, thanks.

@jorisvandenbossche

This comment has been minimized.

Show comment
Hide comment
@jorisvandenbossche

jorisvandenbossche Nov 20, 2015

Member

@TomAugspurger to repeat, awesome work!

A question on the 'provisional status'. First, as I said on gitter, I think it is a good idea to put the same provisional note from the notebook in the whatsnew note (experimental = can still change + feedback wanted).
Secondly, we could also emit a warning about this on first usage to be even more explicit? (but maybe that's a bit too intrusive). But if it is only on the first import of the style module, and not each time you use it, maybe it is OK?

Question for the docs: the built notebook in html form is still in the source code. Is this on purpose? (as eg in the latest PR you only updated the notebook and not the html file)

Member

jorisvandenbossche commented Nov 20, 2015

@TomAugspurger to repeat, awesome work!

A question on the 'provisional status'. First, as I said on gitter, I think it is a good idea to put the same provisional note from the notebook in the whatsnew note (experimental = can still change + feedback wanted).
Secondly, we could also emit a warning about this on first usage to be even more explicit? (but maybe that's a bit too intrusive). But if it is only on the first import of the style module, and not each time you use it, maybe it is OK?

Question for the docs: the built notebook in html form is still in the source code. Is this on purpose? (as eg in the latest PR you only updated the notebook and not the html file)

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Nov 20, 2015

Contributor

Having the generated HTML is not intended, I thought I deleted that. I'll remove it in my PR adding the provisional note.

For the warning. I never was a fan of always getting the warnings when using IPy widgets. I can go either way though. At the very least I'm going to add a note to the docstring for Styler.

Contributor

TomAugspurger commented Nov 20, 2015

Having the generated HTML is not intended, I thought I deleted that. I'll remove it in my PR adding the provisional note.

For the warning. I never was a fan of always getting the warnings when using IPy widgets. I can go either way though. At the very least I'm going to add a note to the docstring for Styler.

@jorisvandenbossche

This comment has been minimized.

Show comment
Hide comment
@jorisvandenbossche

jorisvandenbossche Nov 20, 2015

Member

Another small note on the docs: maybe it would be good to include a link the notebook on nbviewer? As this actually still looks better than the one included in the docs (the table styling (the borders) is 'uglier')

Member

jorisvandenbossche commented Nov 20, 2015

Another small note on the docs: maybe it would be good to include a link the notebook on nbviewer? As this actually still looks better than the one included in the docs (the table styling (the borders) is 'uglier')

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Nov 20, 2015

Contributor

I was trying to figure out how to include a link that points to the same version of the notebook, but adding the link changes the notebook :) I suppose we just link to https://github.com/pydata/pandas/blob/master/doc/source/html-styling.ipynb, understanding that the contents at that URL can change?

Contributor

TomAugspurger commented Nov 20, 2015

I was trying to figure out how to include a link that points to the same version of the notebook, but adding the link changes the notebook :) I suppose we just link to https://github.com/pydata/pandas/blob/master/doc/source/html-styling.ipynb, understanding that the contents at that URL can change?

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Dec 4, 2015

Contributor

@TomAugspurger just came across this from xlsxwriter here
might be a nugget we could steal....

Contributor

jreback commented Dec 4, 2015

@TomAugspurger just came across this from xlsxwriter here
might be a nugget we could steal....

@jankatins

This comment has been minimized.

Show comment
Hide comment
@jankatins

jankatins Dec 7, 2015

Contributor

This is an interesting approach in R: https://github.com/renkun-ken/formattable -> see the last example

Contributor

jankatins commented Dec 7, 2015

This is an interesting approach in R: https://github.com/renkun-ken/formattable -> see the last example

@joekane3 joekane3 referenced this issue Dec 15, 2015

Closed

Style source #11844

@joekane3

This comment has been minimized.

Show comment
Hide comment
@joekane3

joekane3 Dec 15, 2015

Hi,

This feature is great - thanks.

I wanted a way to style a column based on data in another column. I couldn't see a way to do this so made a change to the .bar() styler method. Suggestions on how else to perform such a thing would be appreciated.

image

Apologies if I have not done this correctly.

joekane3 commented Dec 15, 2015

Hi,

This feature is great - thanks.

I wanted a way to style a column based on data in another column. I couldn't see a way to do this so made a change to the .bar() styler method. Suggestions on how else to perform such a thing would be appreciated.

image

Apologies if I have not done this correctly.

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Dec 15, 2015

Contributor

@joekane3 something like that should be possible through the .apply method. Your style function will get the entire DataFrame, so you can use the values in column A to apply styles in columns B. Your function should return a DataFrame with background-color: <color> for column B and empty strings everywhere else.

Of course, you'll have to do all the conversion from values to colors on your own. Pandas just uses matplotlib internally, so that's probably your best bet.

Contributor

TomAugspurger commented Dec 15, 2015

@joekane3 something like that should be possible through the .apply method. Your style function will get the entire DataFrame, so you can use the values in column A to apply styles in columns B. Your function should return a DataFrame with background-color: <color> for column B and empty strings everywhere else.

Of course, you'll have to do all the conversion from values to colors on your own. Pandas just uses matplotlib internally, so that's probably your best bet.

@monkh

This comment has been minimized.

Show comment
Hide comment
@monkh

monkh Dec 30, 2015

I'm new to github, sorry if this is wrong place to post this.
I don't think there is but is there a way to hide the index column when outputting a styled DataFrame through the .render() function?

Also it seems like the Styler is going to (in the future) make .to_html() obsolete.

monkh commented Dec 30, 2015

I'm new to github, sorry if this is wrong place to post this.
I don't think there is but is there a way to hide the index column when outputting a styled DataFrame through the .render() function?

Also it seems like the Styler is going to (in the future) make .to_html() obsolete.

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Dec 30, 2015

Contributor

Correct, the index is unstyleable right now. I plan to fix that in the future, including an option to hide it.

We'll always have to_html, but the implementation might reuse the code here.

Contributor

TomAugspurger commented Dec 30, 2015

Correct, the index is unstyleable right now. I plan to fix that in the future, including an option to hide it.

We'll always have to_html, but the implementation might reuse the code here.

@mjmyers

This comment has been minimized.

Show comment
Hide comment
@mjmyers

mjmyers Jul 18, 2016

Correct, the index is unstyleable right now. I plan to fix that in the future, including an option to hide it.

@TomAugspurger : Was there ever any headway on this? I can't find mention of it in the docs. I've had to do some pretty hacky css to hide the index while using the styler.

mjmyers commented Jul 18, 2016

Correct, the index is unstyleable right now. I plan to fix that in the future, including an option to hide it.

@TomAugspurger : Was there ever any headway on this? I can't find mention of it in the docs. I've had to do some pretty hacky css to hide the index while using the styler.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Jul 18, 2016

Contributor

actually a bit of work here: #11655

Contributor

jreback commented Jul 18, 2016

actually a bit of work here: #11655

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Jul 20, 2016

Contributor

@mjmyers nothing for styling the index yet though. The The big thing is finding an API that's nice to work with. Some possibilities

  • Adding a target={data,index,columns} keyword to relevant functions for what to style
  • Adding dedicated methods like apply_axis/apply_labels or apply_index/apply_columns
  • Adding a namespace df.style.index/columns.<method>

But I haven't thought too much about it yet.

Contributor

TomAugspurger commented Jul 20, 2016

@mjmyers nothing for styling the index yet though. The The big thing is finding an API that's nice to work with. Some possibilities

  • Adding a target={data,index,columns} keyword to relevant functions for what to style
  • Adding dedicated methods like apply_axis/apply_labels or apply_index/apply_columns
  • Adding a namespace df.style.index/columns.<method>

But I haven't thought too much about it yet.

@jreback jreback modified the milestones: Next Major Release, High Level Issue Tracking Sep 24, 2017

@TomAugspurger TomAugspurger removed this from the High Level Issue Tracking milestone Jul 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment