Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

from_records should order dict columns similar to constructor and from_dict like pandas < 0.23 #22708

Open
llawall opened this issue Sep 14, 2018 · 3 comments
Labels
Constructors Series/DataFrame/Index/pd.array Constructors Enhancement

Comments

@llawall
Copy link

llawall commented Sep 14, 2018

Code Sample, a copy-pastable example if possible

import pandas as pd

print("Pandas version :  ", pd.__version__)

def print_dict(fn):
    print(fn.__name__)
    print(fn(dict(C=[3], B=[2], A=[1])))

with pd.option_context('display.max_rows', 10, 'display.max_columns', 5):
    print_dict(pd.DataFrame)
    print_dict(pd.DataFrame.from_dict)
    print_dict(pd.DataFrame.from_records)

Problem description

pandas 0.22 gave consistent output when creating a DataFrame passing a dictionary using any of DataFrame constructor, DataFrame.from_dict and DataFrame.from_records. pandas 0.23 changed the behaviour of creating from dictionaries to respect key ordering in Python version 3.6+ for both the DataFrame constructor and DataFrame.from_dict but not using DataFrame.from_records.

Output 0.23

Different ordering of columns depending on construction method

Pandas version :   0.23.4
DataFrame
   C  B  A
0  3  2  1
from_dict
   C  B  A
0  3  2  1
from_records
   A  B  C
0  1  2  3

Output 0.22

Consistent ordering of columns (sorted keys)

Pandas version :   0.22.0
DataFrame
   A  B  C
0  1  2  3
from_dict
   A  B  C
0  1  2  3
from_records
   A  B  C
0  1  2  3

Expected Output (my local build including #22687)

Consistent ordering of columns (dict key order respected)

Pandas version :   0.24.0.dev0+570.g434910bcb
DataFrame
   C  B  A
0  3  2  1
from_dict
   C  B  A
0  3  2  1
from_records
   C  B  A
0  3  2  1
@TomAugspurger
Copy link
Contributor

I'm not quite sure I understand what you mean by "dicts of different orders in records"?

For example, what's the order of

pd.DataFrame.from_records([{'b': 1, 'a': 2}, {'a': 2, 'b': 1}])

@llawall
Copy link
Author

llawall commented Sep 14, 2018

@TomAugspurger My change does not change the current behaviour in that example. The code path in that case is pandas.core.frame#1315 which ultimately calls through to _list_of_dict_to_arrays.

@llawall
Copy link
Author

llawall commented Oct 22, 2018

I've pushed a test for this that shows the changed behaviour making from_records consistent with the DataFrame constructor when only a single dictionary is passed as input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Constructors Series/DataFrame/Index/pd.array Constructors Enhancement
Projects
None yet
Development

No branches or pull requests

4 participants