pprint ignores the compact parameter for dicts

[NicolasHug]

BPO	34798
Nosy	@serhiy-storchaka, @posita, @miss-islington, @tirkarthi, @iritkatriel, @iansedano
PRs	bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… #26967 [3.10] bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… (GH-26967) #26990

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-07-02.10:08:55.398>
created_at = <Date 2018-09-25.14:45:41.804>
labels = ['3.11', 'type-bug', '3.10', 'docs']
title = 'pprint ignores the compact parameter for dicts'
updated_at = <Date 2021-12-11.16:41:58.483>
user = 'https://bugs.python.org/NicolasHug'

bugs.python.org fields:

activity = <Date 2021-12-11.16:41:58.483>
actor = 'iritkatriel'
assignee = 'docs@python'
closed = True
closed_date = <Date 2021-07-02.10:08:55.398>
closer = 'iritkatriel'
components = ['Documentation']
creation = <Date 2018-09-25.14:45:41.804>
creator = 'Nicolas Hug'
dependencies = []
files = []
hgrepos = []
issue_num = 34798
keywords = ['patch']
message_count = 14.0
messages = ['326358', '326379', '326385', '326386', '326387', '326391', '326394', '375805', '396777', '396851', '396852', '404952', '408307', '408317']
nosy_count = 8.0
nosy_names = ['docs@python', 'serhiy.storchaka', 'posita', 'miss-islington', 'xtreak', 'Nicolas Hug', 'iritkatriel', 'iansedano']
pr_nums = ['26967', '26990']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue34798'
versions = ['Python 3.10', 'Python 3.11']

[NicolasHug]

Dict representations that exceed the line width are printed with one line per key-value pair, ignoring the compact=True parameter:

>>> pprint.pprint({i: 0 for i in range(15)}, compact=True)
{0: 0,
 1: 0,
 2: 0,
 3: 0,
 4: 0,
 5: 0,
 6: 0,
 7: 0,
 8: 0,
 9: 0,
 10: 0,
 11: 0,
 12: 0,
 13: 0,
 14: 0}

Expected behavior:

>>> pprint.pprint({i: 0 for i in range(15)}, compact=True)
{0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0,
 12: 0, 13: 0, 14: 0}

Note that lists are correctly compacted, and that dicts that don't exceed line width are printed on a single line, regardless on the compact parameter.

I could try to work on that if needed?

[tirkarthi]

Thanks for the report Nicolas. I looked into the code and it seems that pprint for a dictionary now doesn't really take compact into account. List, sets, tuple and dequeue use _format_items [0] that honors value of compact but dictionary formatting uses _format_dict_items [1] that doesn't without using compact value. Unfortunately, I don't have any links over why dictionary doesn't take compact into account. I would suggest python-ideas [3] to get more feedback about formatting and implementation along with backwards compatibility so that you can proceed further. Your suggestion seems reasonable to me but I don't know if some program is using compact=True for a dictionary without knowing the internals in mind that might break for them. I would wait for others thoughts on this and I think this can be done only on 3.8 and not 3.7 which is in bug fix mode.

[0]

cpython/Lib/pprint.py

Line 350 in fdcb5ae

def _format_items(self, items, stream, indent, allowance, context, level):

[1]

cpython/Lib/pprint.py

Line 333 in fdcb5ae

def _format_dict_items(self, items, stream, indent, allowance, context,

[2] https://mail.python.org/pipermail/python-ideas/

Hope this helps!

[NicolasHug]

Thank you for the feedback!

I'll try the python-ideas mail list. I posted a message on Python-list [1] a few weeks ago but it didn't get much traction.

I'm not sure about what the final solution could be (if any), but I had to hack pprint myself [2] for the scikit-learn project, and what I did was simply copy pasting _format_items into _format_dict_items, with some minimal changes.

[1] https://mail.python.org/pipermail/python-list/2018-September/737082.html
[2]

[NicolasHug]

Sorry:

[2] https://github.com/scikit-learn/scikit-learn/pull/11705/files#diff-f83e8d9362766b385472f1be7fed9482R96

[serhiy-storchaka]

It is on purpose. See bpo-19132.

[tirkarthi]

Thanks much Serhiy for the details!

[NicolasHug]

Thanks for the link,

But I don't see any justification for this behavior*? Why should lists be compacted but not dicts (even when explicitly asked)?

At the very least it should be made clear in the documentation that dicts are not compacted.

• Maybe there's more content in the patch reviews but I am unable to see them

[iritkatriel]

At the very least it should be made clear in the documentation that dicts are not compacted.

According to https://docs.python.org/3/library/pprint.html compact impacts the way that sequences are displayed, and a dict is not a sequence.
So I'm not sure a documentation change is required.

[iritkatriel]

I made PR26967 to break up the paragraph about the constructor params so that it's easier to read, and also added emphasis around the "sequence" point.

[iritkatriel]

New changeset 943e77d by Irit Katriel in branch 'main':
bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… (GH-26967)
943e77d

[iritkatriel]

New changeset 42c2628 by Miss Islington (bot) in branch '3.10':
bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… (GH-26967) (GH-26990)
42c2628

[iansedano]

I came across this and was confused by it too. I also don't understand the justification with not having dicts to be affected by the compact parameter.

If the "compact form" is having separate entries or elements on one line, instead of having each element separated by a new line, then it seems like inconsistent behavior.

**If a dict is short enough, it will appear in "compact form", just like a list.**

If a dict is too long for the width, then each item will appear in "expanded form", also like a list.
However, the actual compact parameter only affects sequence items. Why is this?

There is no reason given in bpo-19132. It does mention a review, but it doesn't seem to be available, or I don't know how to get to it, to understand the reason for that decision.

[posita]

Please consider highlighting that dicts are not included in the documentation. While *technically* true, this ...

compact impacts the way that long sequences (lists, tuples, sets, etc) are formatted. If compact is false (the default) then each item of a sequence will be formatted on a separate line. If compact is true, as many items as will fit within the width will be formatted on each output line.

... has several problems.

First, sequence is a term of art that also has a common meaning. This creates a potential ambiguity in the understanding of the reader. Resolving that ambiguity in this context requires that readers have already internalized that dicts are not Python sequences. Those new to Python may not understand the (to them, subtle) differences between Python's iterables and sequences. Second, the "etc" only expands that ambiguity and invites confusion. Third, the term "items" is strongly associated with dicts and is used several times throughout the paragraph.

This ...

According to https://docs.python.org/3/library/pprint.html compact impacts the way that sequences are displayed, and a dict is not a sequence.

... is unhelpfully pedantic, and ignorant of the needs of the newcomer (a key demographic of documentation). Documentation is a core product surface with a diverse audience. Rather than focus on technical correctness, documentation authors should focus on accessibility. Redundancy is a feature, not a bug. You can't predict how your reader got to that slice of the documentation. Imagine this as an opportunity to educate or reinforce concepts for readers, not as an opportunity to argue from a position of technical superiority.

The fact that there are now four developers who have taken their time to file patches, bugs, and comments is pretty strong signal that confusion exists among the audience and that the documentation is insufficient.

[iritkatriel]

Don't discuss on a closed issue. Create a new one if there is still a problem.