Unstack with mixed dtypes coerces everything to object

[potash]

Related to #2929, if I unstack a dataframe with mixed dtypes they all get coerced to object and I have to recast to go back which is surprisingly slow (30 seconds for 400k rows and 400 np.float32 columns)

Is there any reason pandas doesn't keep the np.float32 dtype, especially since it supports missing values so even when there are missing index/column positions it shouldn't pose a problem?

[jreback]

pls show a copy-pastable example

[potash]

Ah, looking for an example helped me narrow down the bug. It is specific to passing a list of levels to unstack, even when that list only has a single entry. E.g. compare:

> df = pd.DataFrame({'state':['IL', 'MI'], 'index':['a','a'], 'value1':[1.0,1.0], 'value2':['c','c'] })
> df.set_index(['state','index']).unstack(['index']).dtypes

        index
value1  a        object
value2  a        object
dtype: object

> df.set_index(['state','index']).unstack('index').dtypes
index
value1  a        float64
value2  a         object
dtype: object

So a workaround in my case with multiple levels is to replace unstack(['index1', 'index2']) with unstack('index1').unstack('index2') and indeed I checked that it works.

[jreback]

so looks like what you want is: pydata#9023

which is almost finished. in fact if you are looking for something to do...could use some updating :)

[jreback]

i'll mark this as a bug, which may be independent. want to see if you can put in a fix with the existing framework?

[potash]

Thanks! I will try but I do not use pandas from master and I've never played with the source so it won't be quick.

[kordek]

Picking this up to take a look