Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

undefined parsing behavior with the old style string formatting #73754

Closed
JerryDimitrov mannequin opened this issue Feb 15, 2017 · 9 comments
Closed

undefined parsing behavior with the old style string formatting #73754

JerryDimitrov mannequin opened this issue Feb 15, 2017 · 9 comments
Labels
3.7 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@JerryDimitrov
Copy link
Mannequin

JerryDimitrov mannequin commented Feb 15, 2017

BPO 29568
Nosy @rhettinger, @larryhastings, @benjaminp, @ned-deily, @bitdancer, @serhiy-storchaka, @zhangyangyu
PRs
  • bpo-29568: Disable any characters between two percents for escaped pe… #513
  • Files
  • format-percent.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2017-03-08.03:51:57.349>
    created_at = <Date 2017-02-15.14:53:02.770>
    labels = ['interpreter-core', 'type-bug', '3.7']
    title = 'undefined parsing behavior with the old style string formatting'
    updated_at = <Date 2017-03-24.22:43:13.732>
    user = 'https://bugs.python.org/JerryDimitrov'

    bugs.python.org fields:

    activity = <Date 2017-03-24.22:43:13.732>
    actor = 'xiang.zhang'
    assignee = 'none'
    closed = True
    closed_date = <Date 2017-03-08.03:51:57.349>
    closer = 'xiang.zhang'
    components = ['Interpreter Core']
    creation = <Date 2017-02-15.14:53:02.770>
    creator = 'Jerry Dimitrov'
    dependencies = []
    files = ['46640']
    hgrepos = []
    issue_num = 29568
    keywords = ['patch']
    message_count = 9.0
    messages = ['287857', '287859', '287862', '287864', '287871', '287872', '287890', '287916', '290276']
    nosy_count = 8.0
    nosy_names = ['rhettinger', 'larry', 'benjamin.peterson', 'ned.deily', 'r.david.murray', 'serhiy.storchaka', 'xiang.zhang', 'Jerry Dimitrov']
    pr_nums = ['513']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue29568'
    versions = ['Python 3.7']

    @JerryDimitrov
    Copy link
    Mannequin Author

    JerryDimitrov mannequin commented Feb 15, 2017

    Hello everyone,
    This is my first bug report to the python project, so please excuse me if the metadata for this particular issue is not 100% accurate.

    Today I noticed (with the help from couple of people in IRC) a strange behavior in the python string formatting functionality.

    Consider the following code snippets:

    '%(a)s %(b)' % {'a': '1', 'b': '2'}
    
    # result:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: incomplete format
    
    '%(a) %(b)s' % {'a': '1', 'b': '2'}
    
    # result:
    '%(b)s'
    
    # expected result:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: incomplete format
    

    It seems that there is some kind of inconsistent (undefined) behavior, during the parsing of the type character for the formatted string (tested across all major python 2.x/3.x versions).

    According to the documentation for string formatting and the relevant PEPs, there is no additional info about this particular case.

    I want to say thank you to Yhg1s, JustASlacker, Jerub and lz1irq for discovering this 'bug/feature' and the additional information about it.

    Please let me know if this is a bug, since I am not 100% sure if this is the case.
    Thanks in advance for your time!

    Best Regards,
    Jerry

    @JerryDimitrov JerryDimitrov mannequin added 3.7 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Feb 15, 2017
    @serhiy-storchaka
    Copy link
    Member

    In this example characters between percents are parsed as for any conversion specifier but ignored and '%(a) %' is formatted to '%'. '(a)' specifies a mapping key, ' ' is a conversion flag, the second '%' is a conversion type.

    >>> '%(a) d' % {'a': 123}
    ' 123'
    >>> '%(a) %' % {'a': 123}
    '%'

    This behavior is explicable but looks weird and errorprone. I'm not sure about changing behavior in maintained branches (technically this may be not a bug), but I think it is worth to make this an error in the developed branch.

    @bitdancer
    Copy link
    Member

    I don't think we should change the behavior in maintenance branches. I take it what you want to do is make it so that if the parser ends up thinking it is seeing '%%' but there is stuff between the two %s, that's an error?

    @serhiy-storchaka
    Copy link
    Member

    Yes, it is what I want. Current behavior is just an artefact of the implementation. I have doubts that any other implementation of printf-like formatting has such behavior.

    Bytes formatting starves from the same issue.

    >>> b'% %' % {}
    b'%'

    But there are differences:

    >>> '%12%' % ()
    '%'
    >>> b'%12%' % ()
    b'           %'

    @zhangyangyu
    Copy link
    Member

    The documentation[1] explicitly states using % to do string format could be error-prone and recommends using str.format(). So +1 on no change in maintenance branches.

    [1] https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting

    @zhangyangyu
    Copy link
    Member

    Or even not fix it in develop branch.

    @serhiy-storchaka
    Copy link
    Member

    Here is a patch.

    @JerryDimitrov
    Copy link
    Mannequin Author

    JerryDimitrov mannequin commented Feb 16, 2017

    Thanks for the documentation reference. Can we at least add this link reference [1] as a note or something like that into those documentation sections: [2] https://docs.python.org/2/library/stdtypes.html#string-formatting and [3] https://docs.python.org/3/library/string.html#string-formatting

    I couldn't fine the printf specific documentation at first, so this is why I created the report.

    Regards,
    Jerry

    @zhangyangyu
    Copy link
    Member

    New changeset 9f8ad3f by Xiang Zhang (Serhiy Storchaka) in branch 'master':
    bpo-29568: Disable any characters between two percents for escaped percent "%%" in the format string for classic string formatting. (GH-513)
    9f8ad3f

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants