Followup to Conditional HTML Styling

[TomAugspurger]

Follows #10250

For 0.17.1

• Nicer table styling for the pydata.org website
• Include a visual example in 0.17.1.rst
• remove doc/source/html-styling.html and find a way to include doc/source/html-styling.ipynb in the doc build (should use --template=basic)
• update print_versions
• update requirements_all.txt to include jinja

For 0.18.0 / Future

• sparsify MultiIndex repr (maybe push till 0.18)
• easy alignment styles Right align numbers in HTML repr of tables #12144
• Template modification: This is for things like wrapping base64 encoded values in img tags, urls, etc. flows into...
• Break the large template in Styler.template into smaller blocks. Let people extend that. We could (maybe) allow users to choose which template to use to render each column/cell with solving the template modification problem
• Truncated repr
• hook into pd.options, allow setting of default reprs with styles
• Refactor parts of Styler into a BaseStyler, maybe add a LaTeX styler (maybe deprecate / replace the to_html and to_latex methods; Jinja templates are much more pleasant to work with), xref CLN/API: implemented to_html in terms of .style #11700
• Categoricals / Boolens builtin stylings

will add more as we go.

[jreback]

@TomAugspurger I had to slightly change style.rst to get this to appear in the index. You may want to further modify.

[jankatins]

A comment on the API of the highlighter:

def color_negative_red(val):
    """
    Takes a scalar and returns a string with
    the css property `'color: red'` for negative
    strings, black otherwise.
    """
    color = 'red' if val < 0 else 'black'
    return 'color: %s' % color

This basically only works for html representation ("color: red" is css speak) and assumes that the return value should got to the css.

In latex (see e.g. this example), you prefix/suround the value with a command:

[...]
\usepackage[table]{xcolor}% http://ctan.org/pkg/xcolor
  Some & \cellcolor{blue!25}coloured & contents \\

would render the second cell blue (by using a command from a special package).

So for latex you probably need templates ala \cellcolor{blue!25}%s or \textbf{%s} (which make the value bold).

[kynan]

@TomAugspurger I think you made a great start on this! A few ideas for making this approach potentially more flexible:

1. Allow extra attributes on the <table>.

   Common use case: You want to export an html table with sortable column, e.g. using sortable. For that you would need the following opening tag:

   <table class="sortable-theme-bootstrap" data-sortable>
   

2. Support "external" styling via existing custom CSS style sheets.

   For this to work, the current approach with unique ids per cell which are then targeted with a auto generated CSS doesn't really work. Instead it would be useful to be able to assign additional classes (or data attributes) to cells based on the Styler rules.

   Example use case: assign class positive to all values > 0 and negative to all values < 0

   A potential advantage of this would be much smaller size of generated code (since we don't need a custom CSS block for each cell) and better performance when rendering in the browser (fewer rules to apply).

3. Allow setting arbitrary attributes on table cells based on rules.

This extends the previous suggestion and would allow using exported html tables with any kind of JavaScript library using specific attributes.

Sorry for being so late in making these suggestions - I didn't manage to read through all of #10250 before it was merged.

[TomAugspurger]

@kynan fantastic, thanks for the feedback. I'll go through it in more detail later.

Your item 1. sounds pretty simple. Is attribute the correct term for the items in the opening tag? <table class="sortable-theme-bootstrap" data-sortable> We could include a method like .set_table_attributes for that.

[kynan]

@TomAugspurger I believe attribute is the common term and also the one the W3C uses. set_table_attributes sounds sensible to me.

[jreback]

@TomAugspurger merged your PR; I'll leave you to close when you are ready.

[TomAugspurger]

Thanks, I want to get a better solution in place for including notebooks in the sphinx build, but that works for now.

@kynan, for your second item, assigning classes to cells. My current thinking is to have a method on Styler (would it be terrible to call it .classify?) that takes a function to be evaluated and a class to assign to the cells where that function evaluates to True. The limitation here is that the class that's assigned doesn't get to refer to the data. So you couldn't (easily) do something like our .background_gradient, which is why I discarded this approach originally. But it might make sense to have in addition to the one-class-per cell approach we have. This will probably need to wait for the next release though.

[TomAugspurger]

docs are up if anyone sees anything. It does still have the [In] and [Out] tags and ¶ markers I might be able to hide.

[jreback]

@TomAugspurger yep look great!

on the css side, I think its possible if we tag with the SAME names, e.g.

df.style.highlight_null(css='null_class').background_gradient(css='gradients_class')

then as long as you tag THOSE cells with that class it would work. we could have default class names (based on the function name), and have this kw to override.

[TomAugspurger]

The df.style.highlight_null(css='null_class') I could see working like that, since that fits the binary "if this condition is true, apply this style".

This could be my limited understanding of CSS, but I don't see how you could accomplish df.style.background_gradient(css='gradients_class') in CSS, just knowing that these columns have this class. (I think) you'd need a class per color you want to assign.

I suppose we could add a data attribute to each cell with the value of that cell... You might be able to pull off some CSS wizardry to accomplish it in that case, but I don't see the average python use being able to write or customize that.

[jreback]

@TomAugspurger I think you would actually construct the classes WITH the in this case the level embedded, (for some you wouldn't need to do this), maybe something like 'gradient_level_0_class` (e.g. say you ten levels of gradient. but this is a refinement.

[mattilyra]

I've been playing around with the new styling features and have a few comments, overall this is a great new addition.

The highlight_min, highlight_max and highlight_null would be a lot better if instead of taking a color argument they would actually take the css format string or **kwargs that correspond to css style names - this would allow:

1. using inverted colors (background-color: black; color: white)
2. other formatting options than background shading (font-weight: bold)

The documentation is also a little confusing in terms of debugging the styling functions.

Debugging Tip: If you're having trouble writing your style function, try just passing it into df.apply. Styler.apply uses that internally, so the result should be the same.```

The full stop and space between apply and Styler is somewhat difficult to spot (I only noticed it when pasting the quote here) - I was looking for df.apply.Styler.apply which obviously doesn't exist, a little rewording would fix this. (You need to look at the text at https://pandas-docs.github.io/pandas-docs-travis/style.html to spot it)

Debugging Tip: If you're having trouble writing your style function, try passing it into df.apply. Internally Styler.apply uses that, so the result should be the same.```

[TomAugspurger]

On your first point, I agree that would be useful. If we end up going with a .classify that takes a function returning booleans (pd.isnull) and assigns classes where that's True, we should be able to handle that pretty easily. I think we should hold off adding more keywords to .highlight_null until we decide what to do there.

I just pushed a PR to clarify the documentation. That was confusing, thanks.

[jorisvandenbossche]

@TomAugspurger to repeat, awesome work!

A question on the 'provisional status'. First, as I said on gitter, I think it is a good idea to put the same provisional note from the notebook in the whatsnew note (experimental = can still change + feedback wanted).
Secondly, we could also emit a warning about this on first usage to be even more explicit? (but maybe that's a bit too intrusive). But if it is only on the first import of the style module, and not each time you use it, maybe it is OK?

Question for the docs: the built notebook in html form is still in the source code. Is this on purpose? (as eg in the latest PR you only updated the notebook and not the html file)

[TomAugspurger]

Having the generated HTML is not intended, I thought I deleted that. I'll remove it in my PR adding the provisional note.

For the warning. I never was a fan of always getting the warnings when using IPy widgets. I can go either way though. At the very least I'm going to add a note to the docstring for Styler.

[jorisvandenbossche]

Another small note on the docs: maybe it would be good to include a link the notebook on nbviewer? As this actually still looks better than the one included in the docs (the table styling (the borders) is 'uglier')

[TomAugspurger]

I was trying to figure out how to include a link that points to the same version of the notebook, but adding the link changes the notebook :) I suppose we just link to https://github.com/pydata/pandas/blob/master/doc/source/html-styling.ipynb, understanding that the contents at that URL can change?

[monkh]

I'm new to github, sorry if this is wrong place to post this.
I don't think there is but is there a way to hide the index column when outputting a styled DataFrame through the .render() function?

Also it seems like the Styler is going to (in the future) make .to_html() obsolete.

[TomAugspurger]

Correct, the index is unstyleable right now. I plan to fix that in the future, including an option to hide it.

We'll always have to_html, but the implementation might reuse the code here.

[mjmyers]

Correct, the index is unstyleable right now. I plan to fix that in the future, including an option to hide it.

@TomAugspurger : Was there ever any headway on this? I can't find mention of it in the docs. I've had to do some pretty hacky css to hide the index while using the styler.

[jreback]

actually a bit of work here: #11655

[TomAugspurger]

@mjmyers nothing for styling the index yet though. The The big thing is finding an API that's nice to work with. Some possibilities

• Adding a target={data,index,columns} keyword to relevant functions for what to style
• Adding dedicated methods like apply_axis/apply_labels or apply_index/apply_columns
• Adding a namespace df.style.index/columns.<method>

But I haven't thought too much about it yet.

[attack68]

since this tracker is quite old and many items on it have been addressed or evolved I will close it in favour of more recent discussion. pls re open if you consider it useful.