marshal.load() reads entire remaining file instead of just next value

[mattchaput]

BPO	14447
Nosy	@bitdancer
Superseder	bpo-12291: file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-03-30.01:25:24.790>
created_at = <Date 2012-03-29.20:10:45.931>
labels = ['type-bug', 'expert-IO']
title = 'marshal.load() reads entire remaining file instead of just next value'
updated_at = <Date 2012-03-30.01:25:24.788>
user = 'https://bugs.python.org/mattchaput'

bugs.python.org fields:

activity = <Date 2012-03-30.01:25:24.788>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2012-03-30.01:25:24.790>
closer = 'r.david.murray'
components = ['IO']
creation = <Date 2012-03-29.20:10:45.931>
creator = 'mattchaput'
dependencies = []
files = []
hgrepos = []
issue_num = 14447
keywords = []
message_count = 4.0
messages = ['157093', '157096', '157099', '157117']
nosy_count = 3.0
nosy_names = ['r.david.murray', 'mattchaput', 'weirdink13']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '12291'
type = 'behavior'
url = 'https://bugs.python.org/issue14447'
versions = ['Python 3.1']

[mattchaput]

In Python 3.2, if you write several values to a file with multiple calls to marshal.dump(), and then try to read them back, the first marshal.load() returns the first value, but reads to the end of the file, so subsequent calls to marshal.load() raise an EOFError.

E.g.:

  import marshal
  f = open("test", "wb")
  marshal.dump(("hello", 1), f)
  marshal.dump(("there", 2), f)
  marshal.dump(("friend", 3), f)
  f.close()
  f = open("test", "rb")
  print(marshal.load(f))  # ('hello', 1)
  print(marshal.load(f))  # ERROR

This page seems to indicate this was also a bug in Python 3.1: http://www.velocityreviews.com/forums/t728526-python-3-1-2-and-marshal.html

[weirdink13]

You are correct.
I got:
Python 3.1.2 (release31-maint, Dec  9 2011, 20:50:50) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import marshall
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named marshall
>>> import marshal
>>> f = open("test", "wb")
>>> marshal.dump(("hello", 1), f)
20
>>> marshal.dump(("there", 2), f)
20
>>> marshal.dump(("friend", 3), f)
21
>>> f.close()
>>> f = open("test", "rb")
>>> print(marshal.load(f))
('hello', 1)
>>> print(marshal.load(f))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
EOFError: EOF read where object expected
>>>

[weirdink13]

The previous test was on linux mint 10 (Julia) with python 3.1.2
here is the same test on windows XP with python 3.2.2

Python 3.2.2 (default, Sep  4 2011, 09:51:08) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import marshal
>>> f = open('t', 'wb')
>>> marshal.dump(('skd', 1), f)
18
>>> marshal.dump(('slkd', 2), f)
19
>>> marshal.dump('lkdss', 3), f)
SyntaxError: invalid syntax
>>> marshal.dump(('lskda', 3), f)
20
>>> f.close()
>>> f = open('t', 'rb')
>>> print(marshal.load(f))
('skd', 1)
>>> print(marshal.load(f))
('slkd', 2)
>>> print(marshal.load(f))
('lskda', 3)
>>> print(marshal.load(f))
Traceback (most recent call last):
  File "<pyshell#11>", line 1, in <module>
    print(marshal.load(f))
EOFError: EOF read where object expected
>>> 

As you can see, this problem appearently does not apply to 3.2.2

[bitdancer]

This is a duplicate of bpo-12291. 3.1 is in security-fix only mode.