struct.unpack weird behavior with "bi" (byte then integer)

[Manux]

BPO	6924
Nosy	@mdickinson

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/mdickinson'
closed_at = <Date 2009-09-16.22:05:53.292>
created_at = <Date 2009-09-16.21:53:18.025>
labels = ['type-bug', 'library']
title = 'struct.unpack weird behavior with "bi" (byte then integer)'
updated_at = <Date 2009-09-16.22:05:53.290>
user = 'https://bugs.python.org/Manux'

bugs.python.org fields:

activity = <Date 2009-09-16.22:05:53.290>
actor = 'mark.dickinson'
assignee = 'mark.dickinson'
closed = True
closed_date = <Date 2009-09-16.22:05:53.292>
closer = 'mark.dickinson'
components = ['Library (Lib)']
creation = <Date 2009-09-16.21:53:18.025>
creator = 'Manux'
dependencies = []
files = []
hgrepos = []
issue_num = 6924
keywords = []
message_count = 2.0
messages = ['92724', '92725']
nosy_count = 2.0
nosy_names = ['mark.dickinson', 'Manux']
pr_nums = []
priority = 'normal'
resolution = 'works for me'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue6924'
versions = ['Python 2.6']

[Manux]

Using the following command in Python 2.6.1:

>>> struct.unpack("BI","12345")
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    struct.unpack("BI","12345")
error: unpack requires a string argument of length 8

I get this error message. What confused me was that doing
>>> struct.unpack("IB","12345")
(875770417, 53)
Worked just fine.

I have found out that this only happens using the native byte
order("@"), which is the default.
For Example:
>>> struct.unpack("!BI","12345")
(49, 842216501)
Works, and all other variants, =, <, > (native standard,little endian,
and small endian) also do.

I haven't found anything about that in the documentation.

Also, the requested 3 other bytes arent event used:
>>> struct.unpack("I","abcd")
(1684234849,) # see the big number starting with 16
>>> ord("x")
120
>>> struct.unpack("BI","xabcd") # we get the error
Traceback (most recent call last):
  File "<pyshell#7>", line 1, in <module>
    struct.unpack("BI","xabcd")
error: unpack requires a string argument of length 8
>>> struct.unpack("BI","xabcdefg")
(120, 1734763876) # not the same here
>>> struct.unpack("BI","xabcabcd")
(120, 1684234849) # same here
>>> struct.unpack("BI","x___abcd")
(120, 1684234849) # same again

[mdickinson]

I think this is expected behaviour: the key point is that structs can
include padding bytes. From the documentation:

"By default, C numbers are represented in the machine’s native format and
byte order, and properly aligned by skipping pad bytes if necessary
(according to the rules used by the C compiler)."

'Native' struct formats include padding, while 'standard' formats don't.

So a native struct with format 'BI' has one byte for the 'B', followed by
3 padding bytes, followed by four bytes for the 'I'. This exactly matches
the way a C struct of the form {char c; int x;} would be organized in
memory on that machine.