Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codeop: Show warnings once during _maybe_compile #84984

Closed
csabella opened this issue May 28, 2020 · 11 comments
Closed

Codeop: Show warnings once during _maybe_compile #84984

csabella opened this issue May 28, 2020 · 11 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@csabella
Copy link
Contributor

BPO 40807
Nosy @terryjreedy, @vstinner, @serhiy-storchaka, @Carreau, @csabella, @miss-islington
PRs
  • bpo-40807: Show warnings once from codeop._maybe_compile #20486
  • [3.9] bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486) #20639
  • [3.8] bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486) #20640
  • [3.7] bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486) #20641
  • [3.8] bpo-40807: Backport test_codeop change [GH-19670] #20646
  • [3.7] bpo-40807: Backport test_codeop change [GH-19670] #20647
  • [3.8] bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486) #20672
  • [3.7] bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486) #20673
  • bpo-41520: codeop no longer ignores SyntaxWarning #21838
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-06-09.13:12:26.101>
    created_at = <Date 2020-05-28.13:55:08.174>
    labels = ['type-bug', '3.8', '3.9', '3.10', '3.7', 'library']
    title = 'Codeop: Show warnings once during _maybe_compile'
    updated_at = <Date 2020-08-12.17:54:48.490>
    user = 'https://github.com/csabella'

    bugs.python.org fields:

    activity = <Date 2020-08-12.17:54:48.490>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-06-09.13:12:26.101>
    closer = 'terry.reedy'
    components = ['Library (Lib)']
    creation = <Date 2020-05-28.13:55:08.174>
    creator = 'cheryl.sabella'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 40807
    keywords = ['patch']
    message_count = 11.0
    messages = ['370208', '370722', '370724', '370738', '370739', '370748', '370750', '370751', '370814', '371102', '375155']
    nosy_count = 6.0
    nosy_names = ['terry.reedy', 'vstinner', 'serhiy.storchaka', 'mbussonn', 'cheryl.sabella', 'miss-islington']
    pr_nums = ['20486', '20639', '20640', '20641', '20646', '20647', '20672', '20673', '21838']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue40807'
    versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

    @csabella
    Copy link
    Contributor Author

    When calling codeop._maybe_compile, compile is run three times. If the code being compiled causes a warning message, the warning is generated each time that compile is called, thus (possibly) showing the message three times.

    See msg370163 and that issue for context.

    @csabella csabella added type-feature A feature request or enhancement 3.10 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir labels May 28, 2020
    @terryjreedy
    Copy link
    Member

    The purpose of code and codeop is to imitate interactive python in python. Calling compile() thrice is an implementation detail. Exposing that is a bug. On bpo-37824, Serhiy reported 3 'IDLE' bugs for 3.7 to 3.9: tripled DeprecationWarning (now joined by SyntaxWarning), printing to console (if available) instead of Shell, and an exit exception. This issue is continuation of the triplicate issue, given that the bug is not in IDLE. As a bug, I think it should still be backported.

    I looked more carefully at the behavior of the standard REPL with different statement inputs. Before executing, if the statement is complete, it may raise a compile exception, usually SyntaxError, or emit one or more a warnings, once for each instance. For multiline statements, DeprecationWarnings are emitted immediately after the line with the deprecated code while SyntaxWarnings are emitted, if there is no syntax error, just before execution. SyntaxWarnings for a given slice of code are not repeated. However, at least some of these details have to be enforced in the UI code that repeatedly calls compile or _maybe_compile as lines are added, not in compile, and should not be in _maybe_compile. Both have no memory or previous calls. (I don't think that IDLE necessarily *must* do the same either.)

    Looking at compile itself, it appears that warnings are not emitted if there is an exception. But I am puzzled at this in 3.9 and master.

    >>> if '\e' is 1:
    <stdin>:1: DeprecationWarning: invalid escape sequence \e
    ...
    versus
    >>> compile('if '\e' is 1:\n', '', 'single')
      File "<stdin>", line 1
    SyntaxError: unexpected character after line continuation character

    Something must cause "'\e'" to be a warning instead of an error.

    This is enough to experiment with your patch applied.

    @terryjreedy terryjreedy added 3.7 (EOL) end of life 3.8 only security fixes labels Jun 4, 2020
    @terryjreedy terryjreedy changed the title CODEOP: Show warnings once during _maybe_compile Codeop: Show warnings once during _maybe_compile Jun 4, 2020
    @terryjreedy terryjreedy added type-bug An unexpected behavior, bug, or error 3.7 (EOL) end of life 3.8 only security fixes and removed type-feature A feature request or enhancement labels Jun 4, 2020
    @terryjreedy terryjreedy changed the title CODEOP: Show warnings once during _maybe_compile Codeop: Show warnings once during _maybe_compile Jun 4, 2020
    @terryjreedy terryjreedy added type-bug An unexpected behavior, bug, or error and removed type-feature A feature request or enhancement labels Jun 4, 2020
    @terryjreedy
    Copy link
    Member

    I am not sure if 3 compiles are always needed. At one time, compile required a final '\n' to not raise, but that is no longer true. And as near as I can tell, 'code without final newline\n' and 'code without final newline\n\n' always compile the same. Certainly, Shell does not need to compile without a final newline. But we can look at this in a separate issue. If IDLE's requirement is different from code.II's, I believe it could pass its own compile function.

    @terryjreedy
    Copy link
    Member

    New changeset 052d3fc by Cheryl Sabella in branch 'master':
    bpo-40807: Show warnings once from codeop._maybe_compile (bpo-20486)
    052d3fc

    @miss-islington
    Copy link
    Contributor

    New changeset 3dfe549 by Miss Islington (bot) in branch '3.9':
    bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486)
    3dfe549

    @terryjreedy
    Copy link
    Member

    New changeset a5d6aba by Terry Jan Reedy in branch '3.8':
    [3.8] bpo-40807: Backport test_codeop change [GH-19670]
    a5d6aba

    @terryjreedy
    Copy link
    Member

    New changeset 12d3061 by Miss Islington (bot) in branch '3.7':
    [3.8] bpo-40807: Backport test_codeop change [GH-19670]
    12d3061

    @terryjreedy
    Copy link
    Member

    This is a spinoff of bpo-37824.

    My above comments about compile are wrong. The example nests '' within ''. And, with freshly compiled master, I found an example where Deprecation warning, but not SyntaxWarning, accompanies SyntaxError, and which requires trailing \n.

    >>> compile("if'\e' is 1: 1", '', 'single')
    DeprecationWarning...
    SyntaxError: unexpected EOF while parsing
    >>> compile("if'\e' is 1: 1\n", '', 'single')
    DeprecationWarning...
    SyntaxWarning...
    <code object ...>
    
    Interactively,
    >>> if'\e' is 1: 1
    <stdin>:1: DeprecationWarning: invalid escape sequence \e
    ... 
    <stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
    >>>

    In IDLE, DeprecationWarning is repeated, but that is another issue.

    The secondary prompts seems like a bug, The statement is complete and entering anything but a comment is a SyntaxError. (A comment results in another ... prompt.)

    @miss-islington
    Copy link
    Contributor

    New changeset c067183 by Miss Islington (bot) in branch '3.8':
    bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486)
    c067183

    @terryjreedy
    Copy link
    Member

    The 3.7 backport is needed but required a revised example. The changeset is 4b378ac.

    PRs 20486, 20639, 20672, and 20673 are the main patch. PRs 20646 and 20647 for 3.8 and 3.7 backport a test_codeop import change made elsewhere and not backported. The alternative would have been changing our backport to the older naming convention.

    @Carreau
    Copy link
    Mannequin

    Carreau mannequin commented Aug 10, 2020

    I think that might have introduce https://bugs.python.org/issue41520 where now warnings.simplefilter('error', SyntaxWarning) is silently ignored...

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants