plistlib unable to load iOS7 Safari History.plist

[slosleuth]

BPO	21538
Nosy	@ronaldoussoren, @ned-deily, @serhiy-storchaka
Files	History.plist plistlib_read_refs_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-05-23.14:09:46.147>
created_at = <Date 2014-05-19.22:08:52.507>
labels = ['type-bug', 'library']
title = 'plistlib unable to load iOS7 Safari History.plist'
updated_at = <Date 2014-05-23.14:09:46.146>
user = 'https://bugs.python.org/slosleuth'

bugs.python.org fields:

activity = <Date 2014-05-23.14:09:46.146>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-05-23.14:09:46.147>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2014-05-19.22:08:52.507>
creator = 'slo.sleuth'
dependencies = []
files = ['35298', '35300']
hgrepos = []
issue_num = 21538
keywords = ['patch']
message_count = 13.0
messages = ['218821', '218824', '218825', '218827', '218834', '218835', '218838', '218850', '218854', '218855', '218878', '218968', '218971']
nosy_count = 5.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'python-dev', 'serhiy.storchaka', 'slo.sleuth']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21538'
versions = ['Python 3.4', 'Python 3.5']

[slosleuth]

plistlib has key error when attempting to load the iOS7
var/mobile/Applications/com.apple.mobilesafari/Library/Safari/History.plist. No detected issues with other iOS7 plists.

In [8]: with open('cases/vitale_test_iphone/unback/AppDomain-com.apple.mobilesafari/Library/Safari/History.plist', 'rb') as f:
...: plist = plistlib.lo
plistlib.load plistlib.loads
...: plist = plistlib.load(f)
...:
---------------------------------------------------------------------------

KeyError                                  Traceback (most recent call last)
<ipython-input-8-7824ac03848c> in <module>()
      1 with open('cases/vitale_test_iphone/unback/AppDomain-com.apple.mobilesafari/Library/Safari/History.plist', 'rb') as f:
----> 2     plist = plistlib.load(f)
      3 

/usr/lib/python3.4/plistlib.py in load(fp, fmt, use_builtin_types, dict_type)
993 p = _FORMATS[fmt]'parser'
994
--> 995 return p.parse(fp)
996
997

/usr/lib/python3.4/plistlib.py in parse(self, fp)
620 ) = struct.unpack('>6xBBQQQ', trailer)
621 self._fp.seek(offset_table_offset)
--> 622 offset_format = '>' + _BINARY_FORMAT[offset_size] * num_objects
623 self._ref_format = _BINARY_FORMAT[self._ref_size]
624 self._object_offsets = struct.unpack(

KeyError: 3

[ned-deily]

Can you attach a zipped version of the failing plist or at least identify what data in it is causing the exception? Presumably that plist is user-specific, i.e. contains browser history.

[slosleuth]

I have shared the file for which I have permission, but neither of two
History.plists in my possession will load, both with the same traceback.

Thank you.

On Mon, May 19, 2014 at 4:18 PM, Ned Deily <report@bugs.python.org> wrote:

Ned Deily added the comment:

Can you attach a zipped version of the failing plist or at least identify
what data in it is causing the exception? Presumably that plist is
user-specific, i.e. contains browser history.

----------
nosy: +ned.deily

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue21538\>

---

[ned-deily]

John, a quick look at the file shows that it contains browsing history information that might be personally identifiable. I don't think that is appropriate to store on a public web site like this one so I've deleted the file. Is there some way to provide a simpler file with non-personal history?

[slosleuth]

Ned, I understand. I did get permission to post that history, and as I
feared, the sanitized History.plist I'm attaching now doesn't have the
loading issue that the first did. Possibly a URL character is throwing the
exception?

Can you suggest where I can put a print statement in the library or some
other way to post to get you meaningful debug information? I don't have
any experience with pdb, and a verbose python execution is pretty lengthy.
As I said before, I have two more extensive History.plist files from two
different devices that won't load.

On Mon, May 19, 2014 at 5:51 PM, Ned Deily <report@bugs.python.org> wrote:

Ned Deily added the comment:

John, a quick look at the file shows that it contains browsing history
information that might be personally identifiable. I don't think that is
appropriate to store on a public web site like this one so I've deleted the
file. Is there some way to provide a simpler file with non-personal
history?

----------
nosy: +ned.deily

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue21538\>

---

[serhiy-storchaka]

I think this patch should fix the issue. But tests needed.

[serhiy-storchaka]

And here is a patch with test.

[slosleuth]

Serhiy, I applied the patch and can confirm this is a fix (three previous
failing loads of History.plists from three different devices now load
successfully). Thank you, I would not likely have identified the issue
myself.

On Mon, May 19, 2014 at 11:08 PM, Serhiy Storchaka
<report@bugs.python.org>wrote:

Changes by Serhiy Storchaka <storchaka@gmail.com>:

Removed file: http://bugs.python.org/file35299/plistlib_read_refs.patch

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue21538\>

---

[slosleuth]

Patch plistlib_read_refs_2.patch corrects load error for iOS Safari History.plist.

[serhiy-storchaka]

Issue shouldn't be closed as resolved until patch is committed.

Ronald, is it good to you?

[ronaldoussoren]

The patch looks good.

[python-dev]

New changeset f0452bc62cc3 by Serhiy Storchaka in branch '3.4':
Issue bpo-21538: The plistlib module now supports loading of binary plist files
http://hg.python.org/cpython/rev/f0452bc62cc3

New changeset b2c5d0cba5fd by Serhiy Storchaka in branch 'default':
Issue bpo-21538: The plistlib module now supports loading of binary plist files
http://hg.python.org/cpython/rev/b2c5d0cba5fd

[serhiy-storchaka]

Thank you John for your report.