Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pprint for dict in sorted order or insert order? #74855

Closed
josephsmeng mannequin opened this issue Jun 15, 2017 · 19 comments
Closed

pprint for dict in sorted order or insert order? #74855

josephsmeng mannequin opened this issue Jun 15, 2017 · 19 comments
Labels
3.8 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@josephsmeng
Copy link
Mannequin

josephsmeng mannequin commented Jun 15, 2017

BPO 30670
Nosy @rhettinger, @serhiy-storchaka, @josephsmeng, @pombredanne, @aaronchall, @eamanu, @remilapeyre, @tirkarthi, @ewjoachim
PRs
  • bpo-30670: Add pp function to the pprint module #11769
  • bpo-30670: Add pp function to the pprint module #11769
  • bpo-30670: Add pp function to the pprint module #11769
  • bpo-30670: Fix typo in pprint docs #12500
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-03-22.17:24:37.317>
    created_at = <Date 2017-06-15.02:15:55.228>
    labels = ['3.8', 'type-bug', 'library']
    title = 'pprint for dict in sorted order or insert order?'
    updated_at = <Date 2019-03-23.01:17:06.773>
    user = 'https://github.com/josephsmeng'

    bugs.python.org fields:

    activity = <Date 2019-03-23.01:17:06.773>
    actor = 'xavierguihot'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-03-22.17:24:37.317>
    closer = 'rhettinger'
    components = ['Library (Lib)']
    creation = <Date 2017-06-15.02:15:55.228>
    creator = 'josephsmeng'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 30670
    keywords = ['patch', 'patch', 'patch']
    message_count = 19.0
    messages = ['296056', '296065', '296068', '305440', '305442', '310898', '327904', '334588', '334605', '334689', '334693', '334846', '334854', '334932', '334933', '334936', '334937', '338614', '338615']
    nosy_count = 11.0
    nosy_names = ['rhettinger', 'serhiy.storchaka', 'josephsmeng', 'pombredanne', 'Aaron Hall', 'eamanu', 'remi.lapeyre', 'xtreak', 'Jan-Hein B\xc3\xbchrman', 'Valentin Beyer', 'ewjoachim']
    pr_nums = ['11769', '11769', '11769', '12500']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue30670'
    versions = ['Python 3.8']

    @josephsmeng
    Copy link
    Mannequin Author

    josephsmeng mannequin commented Jun 15, 2017

    start from 3.6+, dict keys are ordered heir creation order.
    the builtin print function works as we expected, but for
    pprint, the keys is sorted. should we using the sorted version
    or just obey the creation order as builtin print?

    @josephsmeng josephsmeng mannequin added 3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jun 15, 2017
    @rhettinger
    Copy link
    Contributor

    Until dict insertion order is guaranteed, it doesn't make sense to change pprint().

    @serhiy-storchaka
    Copy link
    Member

    Concur with Raymond.

    @aaronchall
    Copy link
    Mannequin

    aaronchall mannequin commented Nov 2, 2017

    If/when order is guaranteed (3.7?) we should have a pprint that respects current order,

    -or-

    we should get an improved pprint (maybe named pp or print?) that understands mappings and other abstract data types.

    I had a conversation about pprint at the Python meetup last night. It kinda went like this: https://www.youtube.com/watch?v=NpYEJx7PkWE

    Maybe now's the time for improved behavior?

    @serhiy-storchaka
    Copy link
    Member

    A year ago I proposed to add a method for detecting whether a dict preserves an order and in what sense. [1] But this idea was rejected.

    [1] https://mail.python.org/pipermail/python-dev/2016-October/146597.html

    @pombredanne
    Copy link
    Mannequin

    pombredanne mannequin commented Jan 27, 2018

    IMHO since Guido said that dictionary order is guaranteed going forward [1], the order should now be preserved in 3.6+

    Getting a sorted pprint'ed dict in a Py3.6.1 was a surprise to me coming from Python2.

    The default ordered dict is to me the killer feature of 3 and a major driver to migrate from 2.

    [1] https://mail.python.org/pipermail/python-dev/2017-December/151286.html

    @Jan-HeinBhrman
    Copy link
    Mannequin

    Jan-HeinBhrman mannequin commented Oct 17, 2018

    @terry.reedy - If I understand correctly, this issue was closed by you awaiting what would happen with the dict insertion order after 3.6.

    It has been decided that dict insertion will stay for now and in the future. >>> Should this issue now be reopened?

    And I would favor backward compatibility (key sort order) with an additional keyword for pprint to display it "unsorted" order (from pprint perspective, insertion order for dicts and OrderedDicts, and whatever other order for whatever other dicts).

    @ValentinBeyer
    Copy link
    Mannequin

    ValentinBeyer mannequin commented Jan 30, 2019

    Please reopen this.
    With version 3.7 and guaranteed dict order pprint should not sort by key.

    @josephsmeng
    Copy link
    Mannequin Author

    josephsmeng mannequin commented Jan 31, 2019

    I reopened this issue, and change affected versions to 3.6, 3.7 and 3.8.

    @josephsmeng josephsmeng mannequin added the 3.8 (EOL) end of life label Jan 31, 2019
    @josephsmeng josephsmeng mannequin reopened this Jan 31, 2019
    @remilapeyre
    Copy link
    Mannequin

    remilapeyre mannequin commented Feb 1, 2019

    Hi josephsmeng, dict order preservation is now used in a lot of place in Python. I will post a patch to make pprint use insertion order in a few hours.

    @tirkarthi
    Copy link
    Member

    https://docs.python.org/3/library/pprint.html

    Dictionaries are sorted by key before the display is computed.

    https://mail.python.org/pipermail/python-dev/2017-December/151369.html . There was some discussion in the thread about this being a breaking change and the behavior of pprint. @remi.lapeyre I would suggest waiting for a core dev on approving this change before proceeding with a patch to avoid work being wasted.

    @ewjoachim
    Copy link
    Mannequin

    ewjoachim mannequin commented Feb 4, 2019

    If you stop sorting keys in pprint, then unittest.TestCase.assertDictEquals() (and pytest and such) won't be able to generate a meaningful diff when 2 dicts aren't equal, and it will be much more complicated to understand why a test fails.

    cpython/Lib/unittest/case.py

    Lines 1172 to 1175 in 89427cd

    standardMsg = '%s != %s' % _common_shorten_repr(d1, d2)
    diff = ('\n' + '\n'.join(difflib.ndiff(
    pprint.pformat(d1).splitlines(),
    pprint.pformat(d2).splitlines())))

    (well except if we fix unittest, but then external test runners will need fixing too, and it might not be the only place diff is used on the result of pprint to see how two objects differ)

    @rhettinger
    Copy link
    Contributor

    Consider adding a flag to switch between sorted order and insertion order (defaulting to the current behavior). For convenience, add a short-cut function to call pprint() with the flag set to False.

        def pprint(data, *, sort_dicts=True):
            ...
    
        def pp(data, *args, **kwds):
            pprint(data, *args, sort_dicts=False, **kwds)

    @rhettinger rhettinger removed the 3.7 (EOL) end of life label Feb 5, 2019
    @eamanu
    Copy link
    Mannequin

    eamanu mannequin commented Feb 6, 2019

    Hi!

    def pprint(data, *, sort_dicts=True):
    ...

    def pp(data, *args, **kwds):
    pprint(data, *args, sort_dicts=False, **kwds)

    We could use the parameters sort=? IMHO is better a short parameters name.

    If we assign by default sort_dicts=True, that would have some error on unittest, isn't?

    @tirkarthi
    Copy link
    Member

    pprint only sorts dicts currently so sort_dicts is better than sort that sounds more general. sort_dicts=True is the current behavior so I guess this is backwards compatible way to do this change.

    @eamanu
    Copy link
    Mannequin

    eamanu mannequin commented Feb 6, 2019

    pprint only sorts dicts currently so sort_dicts is better than sort that sounds more general.

    Yes. Because pprint just sort dicts, I think that is better rename the parameter like just sort. But was an opinion, that does not affect to the behavior.

    sort_dicts=True is the current behavior so I guess this is backwards compatible way to do this change.

    I understand it. I was confuse.

    @remilapeyre
    Copy link
    Mannequin

    remilapeyre mannequin commented Feb 6, 2019

    I agree with Karthikeyan, I would expect sort=True to sort all collections, including tuples and lists for example.

    @rhettinger
    Copy link
    Contributor

    New changeset 96831c7 by Raymond Hettinger (Rémi Lapeyre) in branch 'master':
    bpo-30670: Add pp function to the pprint module (GH-11769)
    96831c7

    @rhettinger
    Copy link
    Contributor

    Thanks for the contribution.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants