Add integer formatting code for fixed-width signed arithmetic (2's complement)

[rhettinger]

BPO	30571
Nosy	@rhettinger, @pitrou, @ericvsmith, @skrah, @serhiy-storchaka, @lisroach, @csabella

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/lisroach'
closed_at = None
created_at = <Date 2017-06-05.04:03:26.700>
labels = ['interpreter-core', 'type-feature', '3.7']
title = "Add integer formatting code for fixed-width signed arithmetic (2's complement)"
updated_at = <Date 2017-06-13.03:25:21.157>
user = 'https://github.com/rhettinger'

bugs.python.org fields:

activity = <Date 2017-06-13.03:25:21.157>
actor = 'lisroach'
assignee = 'lisroach'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2017-06-05.04:03:26.700>
creator = 'rhettinger'
dependencies = []
files = []
hgrepos = []
issue_num = 30571
keywords = []
message_count = 8.0
messages = ['295162', '295749', '295752', '295753', '295754', '295793', '295805', '295843']
nosy_count = 8.0
nosy_names = ['rhettinger', 'talin', 'pitrou', 'eric.smith', 'skrah', 'serhiy.storchaka', 'lisroach', 'cheryl.sabella']
pr_nums = []
priority = 'low'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue30571'
versions = ['Python 3.7']

[rhettinger]

The current 'b' formatting code in inconvenient for emulating, demonstrating, and teaching two's complement arithmetic. The problem is that negative inputs are always formatted with a minus sign. I propose that some formatting code be provided for fixed-width display where the leading bit is a sign bit.

For example, if code were a capital 'B' and the total width were 8-bits:

    >>> x = -12
    >>> format(12, '08B')
    '11110100'

Currently, to achieve the same effect, one of the following is used:

    >>> format(x if x >= 0 else x + 2**8, '08b')
    '11110100'

or

    >>> format(x & (2**8 - 1), '08b')
    '11110100'

For values outside the valid range, perhaps a ValueError could be raised:

    >>> format(-200, '08B')
    Traceback (most recent call last):
    ...
    ValueError:  Expected value in range -128 <= x < 128, not -200

I'm not sure what the right code should be. The idea of capital 'B' is attractive, but we already have a different relationship between 'X' and 'x'. There could also be a modifier symbol such as '!' in '!8b'.

[mdickinson]

-1 from me. Using format(x % 2**8, '08b') seems both short enough to be easy to use, and long enough to remind one that there's something a little bit unnatural going on here, given that two's complement isn't something that makes sense for arbitrary-sized integers.

[serhiy-storchaka]

Concur with Mark. Similar issues already were raised multiple times on mailing lists and the conclusion is that explicit wrapping integers to specific range is better. Different behavior for integers out of range is needed in different applications.

[rhettinger]

This is a recurring need in my teaching of Python to hardware engineers. The whole point of having a binary format code is to show which bits are set. For negative values, that need is not being served by the current option. And it makes it awkward when modeling the effects of bitwise operations on signed numbers.

I can understand a -1 if you think this is fundamentally broken, but if you're just saying that you've never personally needed this or casually dismissing the awkwardness of the usual workarounds, then it seems like a meaningless vote that blows-off my proposal which is based on real-world classroom experiences.

The idea is that modifier (such as "!" would require a width argument (the signed representation only make sense in fixed width concepts as every assembly language programmer knows). Its presence would also imply the "0".

Serhiy, I believe you've either read different maillist posts than I have or that you're misinterpreting this as a proposal for a fixed width integer type with automatic wrap-around and truncation (full emulation of a register). This proposal is for display formatting only. IMO, it remedies a deficiency where the current option is more for our convenience (not wanting to deal with the width) rather than for the user's convenience where the user wants to see which bits are set rather than seeing a minus sign (which is both obvious and useless).

Also, when a senior coredev presents a proposal, I expect that it will be greeted with a little more open mindedness and not instantly shot down as if I have no idea what I'm talking about.

[serhiy-storchaka]

The idea is that modifier (such as "!" would require a width argument (the signed representation only make sense in fixed width concepts as every assembly language programmer knows). Its presence would also imply the "0".

I would use the precision for this. Truncating the number of digits in integer representation is similar to truncating the number of characters for strings.

But actually I don't think we should add such feature in the core. It has very limited application. This can be implemented as a third-part library.

Actually I think the stdlib needs something like bitarray and/or bitset. The specially purposed types that allow testing and modifying separate bits and bit ranges.

[skrah]

To expand on what Mark said: If the proposed format code implicitly
gives these results ...

>>> format(12 % 2**8, '08b')
'00001100'
>>> format(-12 % 2**8, '08b')
'11110100'

..., would students not expect that these two operations have the same
result?

>>> 0b11110100 | 0b00001100
252
>>> 
>>> -12 | 12
-4

[pitrou]

The format code space is already crowded, and it isn't immediately obvious that '08B' will format the 2's complement representation of the number, as opposed to '08b' which prepends a sign bit. Not to mention that, as Raymond remarked, we cannot use the same convention for hex formatting where 'X' already has a different meaning.

Instead, how about adding an optional argument to bin()? And then we can also do the same for hex().

By the way, I do find annoying the default behaviour of prepending a sign bit on hexadecimal and binary displays (but I never use binary displays, so my annoyance is really with hexadecimal displays).

[lisroach]

I can't see the drawback of being able to format two's complement easily, isn't it more common to represent negative binary numbers in two's complement form? I find the - sign a little odd myself.

I agree with the idea of using "!" with the precision to represent width.