Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG/ENH: dtype change on enlargement #6485

Open
jreback opened this issue Feb 26, 2014 · 5 comments
Open

BUG/ENH: dtype change on enlargement #6485

jreback opened this issue Feb 26, 2014 · 5 comments
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves setitem-with-expansion

Comments

@jreback
Copy link
Contributor

jreback commented Feb 26, 2014

http://stackoverflow.com/questions/22044766/adding-row-to-pandas-dataframe-changes-dtype/22045011#22045011

A bit non-trival (maybe coerce back after) if possible?

maybe just change this to an actual append?

In [1]: df = DataFrame({'a' : range(10)}, dtype='i4')

In [2]: df
Out[2]: 
   a
0  0
1  1
2  2
3  3
4  4
5  5
6  6
7  7
8  8
9  9

[10 rows x 1 columns]

In [3]: df.dtypes
Out[3]: 
a    int32
dtype: object

In [4]: df.loc[10] = 99

In [5]: df
Out[5]: 
     a
0    0
1    1
2    2
3    3
4    4
5    5
6    6
7    7
8    8
9    9
10  99

[11 rows x 1 columns]

In [6]: df.dtypes
Out[6]: 
a    float64
dtype: object

@goretkin
Copy link

Is there a way to just assert the type on an enlargement?

I'm trying to avoid this...

In [1]: import pandas as pd

In [2]: names_df = pd.DataFrame(
   ...:             {"id":pd.Series([], dtype='int64')
   ...:              })
   ...: names_df.dtypes
   ...: 
Out[2]: 
id    int64
dtype: object

In [3]: names_df.loc[0, "id"] = 3
   ...: names_df.dtypes
   ...: 
Out[3]: 
id    float64
dtype: object

@jorisvandenbossche
Copy link
Member

You can avoid this by not assigning with enlargement.
Options are populating the dataframe on construction, or using concat

@mroeschke
Copy link
Member

Adding to a non-empty dataframe no longer enlarges (Jeff's example) but the empty case still does.

@mroeschke
Copy link
Member

The first example could use a regression test since it works on master.

In [49]: In [1]: df = DataFrame({'a' : range(10)}, dtype='i4')
    ...:

In [50]: df
Out[50]:
   a
0  0
1  1
2  2
3  3
4  4
5  5
6  6
7  7
8  8
9  9

In [51]: df.dtypes
Out[51]:
a    int32
dtype: object

In [52]: In [4]: df.loc[10] = 99
    ...:

In [53]: df.dtypes
Out[53]:
a    int64
dtype: object

In [56]: pd.__version__
Out[56]: '0.26.0.dev0+627.gef77b5700'

@mroeschke mroeschke added good first issue Needs Tests Unit test(s) needed to prevent regressions and removed Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves labels Oct 22, 2019
@mroeschke mroeschke added Bug Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves and removed Needs Tests Unit test(s) needed to prevent regressions good first issue labels Dec 24, 2019
@mroeschke
Copy link
Member

Noted #30444 (comment): The first example should maintain the int32 typing so this is not completely fixed.

@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves setitem-with-expansion
Projects
None yet
Development

No branches or pull requests

5 participants