Plistlib outputs empty data tags when deeply nested

[jfortier]

BPO	17353
Nosy	@ronaldoussoren, @ezio-melotti, @serhiy-storchaka
Files	deeply-nested-plist.py issue-17353.txt issue-17353-tests.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ronaldoussoren'
closed_at = <Date 2013-04-23.11:51:27.717>
created_at = <Date 2013-03-04.20:33:35.585>
labels = ['OS-mac', 'easy', 'type-bug', 'library']
title = 'Plistlib outputs empty data tags when deeply nested'
updated_at = <Date 2013-04-23.11:51:27.716>
user = 'https://bugs.python.org/jfortier'

bugs.python.org fields:

activity = <Date 2013-04-23.11:51:27.716>
actor = 'ronaldoussoren'
assignee = 'ronaldoussoren'
closed = True
closed_date = <Date 2013-04-23.11:51:27.717>
closer = 'ronaldoussoren'
components = ['Library (Lib)', 'macOS']
creation = <Date 2013-03-04.20:33:35.585>
creator = 'jfortier'
dependencies = []
files = ['29862', '29863', '29894']
hgrepos = []
issue_num = 17353
keywords = ['easy']
message_count = 13.0
messages = ['183496', '183497', '183509', '183525', '186929', '186941', '186944', '186967', '186981', '186982', '187127', '187153', '187629']
nosy_count = 6.0
nosy_names = ['ronaldoussoren', 'ezio.melotti', 'python-dev', 'serhiy.storchaka', 'jfortier', 'mmilkin']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'needs patch'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17353'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

[jfortier]

To reproduce:
import plistlib
print plistlib.writePlistToString([[[[[[[[[{'test': plistlib.Data('aaaaaa')}]]]]]]]]])

Compare to:
print plistlib.writePlistToString([[[[[[[[{'test': Data('aaaaaa')}]]]]]]]])

It looks like the max line length calculation in the Data class is causing the output to be empty once you get nested far enough.

[jfortier]

My second print should have been:

print plistlib.writePlistToString([[[[[[[[{'test': plistlib.Data('aaaaaa')}]]]]]]]])

[jfortier]

Quick source inspection suggests this is present in the latest version.

[serhiy-storchaka]

We should limit the indentation to be less than the maximal line length.

[mmilkin]

Looks like plistlib.writePlistToString is no loger in the plistlib.

[ezio-melotti]

On 3.x it's called writePlistToBytes, and it seems to have the same issue.

[mmilkin]

Yep you are correct.
>>>
plistlib.readPlistFromBytes(plistlib.writePlistToBytes([[[[[[[[{'test': plistlib.Data(b'aaaaaa')}]]]]]]]]))
[[[[[[[[{'test': Data(b'aaaaaa')}]]]]]]]]
>>> plistlib.readPlistFromBytes(plistlib.writePlistToBytes([[[[[[[[[{'test': plistlib.Data(b'aaaaaa')}]]]]]]]]]))
[[[[[[[[[{'test': Data(b'')}]]]]]]]]]

I presume we want to raise a ValueError if the indentation is more than the maximal line length.

[ezio-melotti]

If possible it would be better to still generate a valid output, possibly by limiting the indentation or removing it after a certain point.
However I don't know if there are any restrictions that should be followed while generating a plist (What happens if the line is too long? Is the indentation required?).

[ronaldoussoren]

I agree that plistlib shouldn't raise an exception for data that can represented as a valid plist file.

I've checked that the Cocoa class for generating plist files will happily create a plist file when the data is nested 100 levels deep. In that case NSData values generate lines of 12 characters long.

An unrelated issue: PlistWriter.writeValue should treat bytes instances the same as Data instances in Python 3. That would be a (small) feature enhencement, and hence can only be done for Python 3.4.

[ronaldoussoren]

The attach patch should fix the issue (but there needs to be a unittest as well).

[mmilkin]

Adding tests.

[ronaldoussoren]

The tests look good, thanks for writing them.

[python-dev]

New changeset a389700fbc74 by Ronald Oussoren in branch '2.7':
Ensure that plistlib doesn't corrupt deeply nested datastructures
http://hg.python.org/cpython/rev/a389700fbc74

New changeset 6a8cb4875ac6 by Ronald Oussoren in branch '3.3':
Ensure that plistlib doesn't corrupt deeply nested datastructures
http://hg.python.org/cpython/rev/6a8cb4875ac6

New changeset e0c0bcd60033 by Ronald Oussoren in branch 'default':
(3.3->default) Ensure that plistlib doesn't corrupt deeply nested datastructures
http://hg.python.org/cpython/rev/e0c0bcd60033