Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conditional HTML styling hides MultiIndex structure #11655

Closed
mattilyra opened this issue Nov 20, 2015 · 16 comments
Closed

Conditional HTML styling hides MultiIndex structure #11655

mattilyra opened this issue Nov 20, 2015 · 16 comments

Comments

@mattilyra
Copy link

@mattilyra mattilyra commented Nov 20, 2015

I noticed that the styling doesn't quite work with MultiIndex data frames. For example if we have a dataframe with two cross validated scores and take their meanand sem using groupby to get a MultiIndex dataframe

import itertools
import pandas as pd
import numpy as np

jobs = itertools.product(['a', 'b', 'c', 'd'], np.arange(1e-4, 1e-3, .0003), range(10))
rows = []
for v, v2, itr in jobs:
    rows.append({'param_1': v, 'param_2': v2, 'iter': itr,
                 'score_1': np.random.randint(0, 100, size=(1,))[0],
                 'score_2': np.random.rand(1, )[0]})
df_multi = pd.DataFrame(rows)
agg = df_multi.groupby(by=['param_1', 'param_2'])[['score_1', 'score_2']].agg(['mean', 'sem'])
agg.style.highlight_max()

screen shot 2015-11-20 at 12 04 13

where as without the styling the index names are preserved and the score_1 and score_2 column headers span 2 columns which improves legibility.

screen shot 2015-11-20 at 12 02 56

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Nov 20, 2015

IIRC this is 'not implemented', cc @TomAugspurger as wanted to get basic stuff working.

patches are welcome!

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Nov 20, 2015

see last check box in #11610

@jreback jreback changed the title Conditional HTML styling hides `MultiIndex` structure Conditional HTML styling hides MultiIndex structure Nov 20, 2015
@TomAugspurger

This comment has been minimized.

Copy link
Contributor

@TomAugspurger TomAugspurger commented Nov 20, 2015

Yep. This one isn't too bad to do, I just ran out of time. It's a matter of modifying the template to be aware of the index levels.

@mattilyra

This comment has been minimized.

Copy link
Author

@mattilyra mattilyra commented Nov 20, 2015

@jreback ah I see, didn't read the post properly. I guess in that case this issue is redundant and can be closed.

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Nov 20, 2015

@mattilyra its fine. you can post a fix and directly reference this.

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Feb 13, 2016

this still open @TomAugspurger ?

@jreback jreback modified the milestones: Next Major Release, 0.18.0 Feb 13, 2016
@TomAugspurger

This comment has been minimized.

Copy link
Contributor

@TomAugspurger TomAugspurger commented Feb 13, 2016

Yeah, still open.

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Jul 6, 2016

@TomAugspurger did you have a fix for this already? (even if not prod ready), just want to give a try.

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

@TomAugspurger TomAugspurger commented Jul 6, 2016

@jreback I can get something together tonight, if that's not too late.

The basic idea is to use the logic in pandas.core.format._get_level_lengths on each of the MI elements. We'll put a couple more pieces of state in the internal dict is_visible, row_span and col_span.

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Jul 6, 2016

that would be totally fantastic!

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

@TomAugspurger TomAugspurger commented Jul 7, 2016

@jreback OK, the minimal version (1 whole test) is here: https://github.com/TomAugspurger/pandas/tree/style-sparse-mi-2

I'll see what refactoring I can do and submit a pull request sometime soonish.

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Jul 7, 2016

image

hmm, doesn't seem to change the display. (using pandas 0.18.0 as a comparison). The column labels are not showing sparse (as what we normally do in an html display)

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

@TomAugspurger TomAugspurger commented Jul 7, 2016

Ahh, only did index labels. Give me 20 minutes :)

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

@TomAugspurger TomAugspurger commented Jul 7, 2016

@jreback pushed 4655534 which fix it.

https://gist.github.com/643402b241e6d945118942cb8ccff3ac

Index names are messed up with a MultiIndex, but if you just set those to "" then the None shouldn't appear.

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Jul 7, 2016

that looks great! thanks!

I realize that the CSS injection can be done thru set_table_style I think as well.

@jreback

This comment has been minimized.

Copy link
Contributor

@jreback jreback commented Jul 18, 2016

@TomAugspurger so this fixes the single level index case (I don't have a good repro :<
but); also only barely tested on a single level columns (this works with the multi-level nicley as that was the point).

def _get_level_lengths(index):
    '''
    Given an index, find the level lenght for each element.

    Result is a dictionary of (level, inital_position): span
    '''
    sentinel = com.sentinel_factory()
    levels = index.format(sparsify=sentinel, adjoin=False, names=False)

    if index.nlevels == 1:
        return { (0, i):1 for i, value in enumerate(levels) }

    lengths = {}
    for i, lvl in enumerate(levels):
        for j, row in enumerate(lvl):
            if row != sentinel:
                last_label = j
                lengths[(i, last_label)] = 1
            else:
                lengths[(i, last_label)] += 1

    return lengths
@jreback jreback mentioned this issue Jul 18, 2016
6 of 13 tasks complete
@jreback jreback modified the milestones: 0.19.0, Next Major Release Jul 24, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.