Assignment to mixed-type DataFrame with float32 coerce to float64

[fnielsen]

Assignment to mixed-type DataFrame with float32 numpy array coerce to float64

>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame([[1, 2, 'a'], [3, 4, 'b']], dtype=np.float32)
>>> A = df.iloc[:, 0:1].values
>>> df.iloc[:, 0:1] = A
>>> df[0].dtype
dtype('float64')

Shouldn't the result be dtype('float32')?

See also http://stackoverflow.com/questions/35230388/why-does-pandas-coerce-my-numpy-float32-to-float64

[fnielsen]

It may be worth to note that float64 coercing with float32 can be considerably slow. I have another Stackoverflow question about that:

http://stackoverflow.com/questions/35184681/assignment-of-pandas-dataframe-with-float32-and-float64-slow

The coercing here is over 10 times as slow.

[jreback]

This area of the code is quite complex and inference is pretty tricky, but I suppose.

In [13]: df.ix[:,0:1] = DataFrame(A)

In [14]: df
Out[14]: 
     0    1  2
0  1.0  2.0  a
1  3.0  4.0  b

In [15]: df.dtypes
Out[15]: 
0    float32
1    float32
2     object
dtype: object

[jreback]

submitting a PR would help move this along.

[jreback]

This is around here

[fnielsen]

The DataFrame trick does not work in all cases. If the columns are named

import pandas as pd
import numpy as np
columns = ['c0', 'c1', 'c2']
df = pd.DataFrame([[1, 2, 'a'], [3, 4, 'b']], columns=columns, dtype=np.float32)
A = df.ix[:, 0:1].values
df.dtypes
df.ix[:, 0:1] = pd.DataFrame(A)
df.dtypes

The result yields a wrong datatype for the first column (c0) while the second column (c1) is correct:

c0    float32
c1    float32
c2     object
dtype: object
c0    float64
c1    float32
c2     object
dtype: object

This is contrary df.ix[:, 0:1] = A where all numerical columns wrongly coerce to float64.

[jreback]

This is not a trick, but a defined aligning operation. Assigning w/o the proper columns doesn't make much sense.

In [13]: df.ix[:, 0:1] = pd.DataFrame(A,columns=['c0'])

In [14]: df
Out[14]: 
    c0   c1 c2
0  1.0  2.0  a
1  3.0  4.0  b

In [15]: df.dtypes
Out[15]: 
c0    float32
c1    float32
c2     object
dtype: object