xdrlib fails to detect overflow (struct bug?)

[lloyd]

BPO	1523
Nosy	@mdickinson, @devdanzin
Files	xdr.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/mdickinson'
closed_at = <Date 2009-07-07.15:11:32.292>
created_at = <Date 2007-11-29.17:15:11.995>
labels = ['extension-modules', 'type-bug']
title = 'xdrlib fails to detect overflow (struct bug?)'
updated_at = <Date 2009-07-07.15:11:32.285>
user = 'https://bugs.python.org/lloyd'

bugs.python.org fields:

activity = <Date 2009-07-07.15:11:32.285>
actor = 'mark.dickinson'
assignee = 'mark.dickinson'
closed = True
closed_date = <Date 2009-07-07.15:11:32.292>
closer = 'mark.dickinson'
components = ['Extension Modules']
creation = <Date 2007-11-29.17:15:11.995>
creator = 'lloyd'
dependencies = []
files = ['8825']
hgrepos = []
issue_num = 1523
keywords = []
message_count = 5.0
messages = ['57939', '87709', '87710', '90151', '90237']
nosy_count = 3.0
nosy_names = ['mark.dickinson', 'ajaksu2', 'lloyd']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'test needed'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1523'
versions = ['Python 2.6', 'Python 2.7']

[lloyd]

The XDR format requires integers to fit into 4 byte values. However (at
least on x86-64) xdrlib will silently accept (and truncate) values
larger than this (up to 2**64-1). Taking a (very brief) look at the
xdrlib code, it appears this is actually a problem in the struct module,
but I don't have the time (or interest) to trace through the _struct
module code.

An example on an x86-64 machine (Python 2.4.3) of encoding 2**32 (which
will not fit in an XDR field) being silently truncated:

$ ./xdr.py 
4294967296 -> 00000000 -> 0

An example of struct itself not detecting overflow:

>>> struct.pack("!I", 2**32)
'\x00\x00\x00\x00'

struct.pack will only throw an overflow error if a value >= 2**64 is
used, even if it is encoding into a field that is much smaller.

[devdanzin]

On trunk, I get the described struct behavior with " DeprecationWarning:
struct integer overflow masking is deprecated". The xdr.py script gives:
18446744073709551615 -> ffffffff -> 4294967295

On py3k, both raise "struct.error: argument out of range".

Tested on Linux ia32.

[mdickinson]

I'll take a look.

[mdickinson]

I think there's nothing we can do about this in 2.6: there's probably code
out there that depends on the overflow behaviour.

However, 2.6 should have been issuing DeprecationWarnings for overflow
handling, so it would be safe to remove the overflow wrapping for 2.7.

[mdickinson]

The deprecated overflow wrapping has been removed from trunk in r73891.
(It was already removed in py3k a while ago.)

I'd really like to remove the deprecated float coercion from trunk too,
but since the warnings weren't functioning properly in 2.6, this probably
has to wait until 2.8.