test_check_c_globals "crashed" and then "FutureWarning: Possible nested set at position 12"

[iritkatriel]

BPO	44786
Nosy	@vstinner, @ericsnowcurrently, @serhiy-storchaka, @miss-islington, @iritkatriel
PRs	bpo-44786: Fix a warning in RE in c-analyzer #28351 [3.10] bpo-44786: Fix a warning in RE in c-analyzer (GH-28351) #28353

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-15.16:44:21.659>
created_at = <Date 2021-07-30.18:27:57.153>
labels = ['type-bug', 'tests', '3.10', '3.11']
title = 'test_check_c_globals "crashed" and then "FutureWarning: Possible nested set at position 12"'
updated_at = <Date 2021-09-15.16:44:21.658>
user = 'https://github.com/iritkatriel'

bugs.python.org fields:

activity = <Date 2021-09-15.16:44:21.658>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-15.16:44:21.659>
closer = 'serhiy.storchaka'
components = ['Tests']
creation = <Date 2021-07-30.18:27:57.153>
creator = 'iritkatriel'
dependencies = []
files = []
hgrepos = []
issue_num = 44786
keywords = ['patch']
message_count = 5.0
messages = ['398600', '401827', '401841', '401842', '401853']
nosy_count = 5.0
nosy_names = ['vstinner', 'eric.snow', 'serhiy.storchaka', 'miss-islington', 'iritkatriel']
pr_nums = ['28351', '28353']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44786'
versions = ['Python 3.10', 'Python 3.11']

[iritkatriel]

% ./python.exe -E -We -m test -v test_check_c_globals

...

test test_check_c_globals crashed -- Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython-1/Lib/test/libregrtest/runtest.py", line 335, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/test/libregrtest/runtest.py", line 280, in _runtest_inner2
    the_module = importlib.import_module(abstest)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1061, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1038, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1015, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 689, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 894, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/iritkatriel/src/cpython-1/Lib/test/test_check_c_globals.py", line 6, in <module>
    from cpython.__main__ import main
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Tools/c-analyzer/cpython/__main__.py", line 18, in <module>
    from c_parser.info import KIND
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Tools/c-analyzer/c_parser/__init__.py", line 1, in <module>
    from .parser import parse as _parse
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Tools/c-analyzer/c_parser/parser/__init__.py", line 119, in <module>
    from ..info import ParsedItem
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Tools/c-analyzer/c_parser/info.py", line 10, in <module>
    import c_common.tables as _tables
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Tools/c-analyzer/c_common/tables.py", line 236, in <module>
    _COLSPEC_RE = re.compile(textwrap.dedent(r'''
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/re.py", line 229, in compile
    return _compile(pattern, flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/re.py", line 281, in _compile
    p = sre_compile.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/sre_parse.py", line 834, in _parse
    p = _parse_sub(source, state, sub_verbose, nested + 1)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/sre_parse.py", line 540, in _parse
    warnings.warn(
    ^^^^^^^^^^^^^^
FutureWarning: Possible nested set at position 12

test_check_c_globals failed (uncaught exception)

[vstinner]

The warning comes from this regex: Tools/c-analyzer/c_common/tables.py", line 236

_COLSPEC_RE = re.compile(textwrap.dedent(r'''
    ^
    (?:
        [[]
        (
            (?: [^\s\]] [^\]]* )?
            [^\s\]]
        )  # <label>
        []]
    )?
    ( \w+ )  # <field>
    (?:
        (?:
            :
            ( [<^>] )  # <align>
            ( \d+ )  # <width1>
        )
        |
        (?:
            (?:
                :
                ( \d+ )  # <width2>
            )?
            (?:
                :
                ( .*? )  # <fmt>
            )?
        )
    )?
    $
'''), re.VERBOSE)

[serhiy-storchaka]

Yes, [[] looks as the start of a character set containing nested sets (e.g. "[[a-z]--[p-q]]"). This feature is supported by regex and can be added in future versions of re, breaking the code which contain such combinations.

[serhiy-storchaka]

New changeset 1a9ef57 by Serhiy Storchaka in branch 'main':
bpo-44786: Fix a warning in RE in c-analyzer (GH-28351)
1a9ef57

[serhiy-storchaka]

New changeset 89966f5 by Miss Islington (bot) in branch '3.10':
bpo-44786: Fix a warning in RE in c-analyzer (GH-28351) (GH-28353)
89966f5