ENH: Add normalize to crosstab

[nickeubank]

It'd be great to have a simple normalization option for cross tab to get shares rather than frequencies.

Something that would do something like:

def normalize(x):
    return len(x)/len(w_mobile.language)

pd.crosstab(w_mobile.language,w_mobile.carrier, values=w_mobile.language, aggfunc=normalize)

as just an option.

(The ability to do row-normalizations and column normalizations would also be great -- so all entries in a row add to 1 or all entries in a column add to 1). Similar in behavior (for row normalizations) as:

l = list()
df = pd.DataFrame({'carrier':['a','a','b','b','b'], 'language':['english','spanish', 'english','spanish','spanish']})

for i in df.carrier.unique():
    temp = df.query('carrier=="{}"'.format(i)).language.value_counts(normalize=True)
    temp.name = i
    l.append(temp)

ctab = pd.concat(l, axis=1)


Out[1]: 
                a         b
 english  0.5  0.333333
spanish  0.5  0.666667

But with a command like: pd.crosstab(df.language, df.carrier, normalization='row')

[sinhrks]

I also use this type of operation in pivot_table. Is it can be a part of aggfunc, maybe pct, pct_row and pct_col?

[nickeubank]

I like it. I'll try a few things and submit a PR.

[nickeubank]

Relatedly, crosstab also has a bug -- it counts np.nan in margin totals even when dropna=True.

df = pd.DataFrame({'a':[1,2,2,2,2,np.nan],'b':[3,3,4,4,4,4]})
pd.crosstab(df.a,df.b, margins=True)
Out[233]: 
b    3  4  All
a             
1.0  1  0    1
2.0  1  3    4
All  2  4    6

Not related to #12558 i don't think

[jreback]

might be #4003

[nickeubank]

@jreback don't think so -- that's double counting. This is only happens on columns with np.nan and increments by num of np.nans.

[jreback]

ok if you can't find a related one, then pls open a new issue

[jreback]

in fact, if you can, pls open a new issue (we'll call it and i'll tag it master), and can list a checkbox for all of the crosstab issues. (each individual one has an issue and we just refernce things) like #11485

[nickeubank]

Posted PR to #12578 . Input welcome @sinhrks