PEP 515-style formatting with underscores does not seem to work for Decimal

[sndrtj]

BPO	45708
Nosy	@tim-one, @rhettinger, @mdickinson, @ericvsmith, @tiran, @serhiy-storchaka, @sobolevn, @sndrtj
PRs	bpo-45708: Support underscore separators when formatting Decimal objects #29438

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-11-04.09:01:14.767>
labels = ['type-feature', 'library', '3.11']
title = 'PEP 515-style formatting with underscores does not seem to work for Decimal'
updated_at = <Date 2021-11-07.16:29:51.064>
user = 'https://github.com/sndrtj'

bugs.python.org fields:

activity = <Date 2021-11-07.16:29:51.064>
actor = 'eric.smith'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-11-04.09:01:14.767>
creator = 'sndrtj'
dependencies = []
files = []
hgrepos = []
issue_num = 45708
keywords = ['patch']
message_count = 14.0
messages = ['405667', '405679', '405683', '405684', '405685', '405724', '405739', '405791', '405795', '405853', '405854', '405855', '405895', '405907']
nosy_count = 8.0
nosy_names = ['tim.peters', 'rhettinger', 'mark.dickinson', 'eric.smith', 'christian.heimes', 'serhiy.storchaka', 'sobolevn', 'sndrtj']
pr_nums = ['29438']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45708'
versions = ['Python 3.11']

[sndrtj]

Hello,

It appears that Decimal does not support PEP-515 style formatting with underscores as thousands separators.

>>> from decimal import Decimal
>>> f"{Decimal('5000'):,}"
'5,000'
>>> f"{Decimal('5000'):_}"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid format string

This does work for all the other types mentioned in PEP-515

>>> f"{5000:_}"
'5_000'
>>> f"{5000.0:_}"
'5_000.0'
>>> f"{complex(5000, 1):_}"
'(5_000+1j)'

I have tried this on python 3.8, 3.9 and 3.10 on a Mac.

[sobolevn]

It is not hard to fix (

cpython/Modules/_decimal/libmpdec/io.c

Lines 857 to 863 in 2c045bd

	/* thousands separator */
	if (*cp == ',') {
	spec->dot = ".";
	spec->sep = ",";
	spec->grouping = "\003\003";
	cp++;
	}

):

    /* thousands separator */
    if (*cp == ',' || *cp == '_') {
        spec->dot = ".";
        if (*cp == ',') {
            spec->sep = ",";
        } else {
            spec->sep = "_";
        }
        spec->grouping = "\003\003";
        cp++;
    }

But, there's some context to it: https://bugs.python.org/issue44267 / https://mail.python.org/pipermail/python-dev/2016-March/143556.html

I am going to research this further.

[mdickinson]

I think the two main reasons that applied to not implementing the parsing part of PEP-515 for the Decimal type (speed, compliance with the IBM specification) don't apply to the formatting side.

We do need to think about the implications of making local changes to our copy of the externally-maintained libmpdec library, though.

Changing Python versions: this is a new feature, so could only go into Python 3.11.