gh-90095: Make .pdbrc work properly and add some reasonable tests #110496

gaogaotiantian · 2023-10-07T05:36:28Z

pdb -c and .pdbrc works in a very weird way now. cmd.Cmd has cmdqueue which is perfect for loading user defined commands yet we decided to make our own path to hack it. This makes the special commands like commands or ;; fail because they rely on some cmd.Cmd mechanics.

This PR utilizes cmdqueue (and fixed where it was used wrong) which fixed the problem mentioned in #90095. With this implementation, the commands in .pdbrc and passed in with -c are treated almost exactly the same as the commands the users type in.

Two existing tests related to .pdbrc are removed as ... they were not testing they thought they were.

test_read_pdbrc_with_ascii_encoding simply does not make sense - it basically tests that system with ascii encoding can't print unicode - it has almost nothing to do with pdb. It has nothing to do with enforcing .pdbrc to be utf-8.
test_readrc_kwarg passes only because the commands in .pdbrc does not execute - I don't understand what's the point of the test.

Reasonable tests are added with the PR to test .pdbrc. The testing framework is improved a bit to make writing .pdbrc tests easier.

Issue: Pdb can't use the commands through -c or .pdbrc files #90095

Delengowski · 2024-03-09T17:19:28Z

I'd really like to echo that this become included. I myself wanted to take a crack this and am excited to see a PR open for it. Defining commands in .pdbrc would be a great use.

Lib/test/test_pdb.py

Lib/pdb.py

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>

gaogaotiantian · 2024-03-11T21:58:20Z

Do we consider this as a bug @iritkatriel ? (backport?)

iritkatriel · 2024-03-11T22:20:44Z

Could backporting break working code?

Delengowski · 2024-03-11T22:50:41Z

@iritkatriel thank you reviewing this and getting it merged. @gaogaotiantian thank you for taking up this implementation.

bedevere-bot · 2024-03-11T22:52:43Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x has failed when building commit 44f9a84.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/345/builds/7401) and take a look at the build logs.
Check if the failure is related to this commit (44f9a84) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/345/builds/7401

Failed tests:

test_tokenize
test_io
test.test_concurrent_futures.test_thread_pool

Failed subtests:

test_normalization - test.test_unicodedata.NormalizationTest.test_normalization
test_choices_algorithms - test.test_random.MersenneTwister_TestBasicOps.test_choices_algorithms
test_bug_9025 - test.test_random.MersenneTwister_TestBasicOps.test_bug_9025
test_framing_many_objects - test.test_pickletools.OptimizedPickleTests.test_framing_many_objects
test_threads - test.test_io.PyBufferedReaderTest.test_threads
test_sample_counts_equivalence - test.test_random.MersenneTwister_TestBasicOps.test_sample_counts_equivalence

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 882, in test_choices_algorithms
    c = self.gen.choices(range(n), cum_weights=range(1, n+1), k=10000)
        ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 492, in choices
    return [population[bisect(cum_weights, random() * total, 0, hi)]
                                           ~~~~~~^^
TypeError: 'Random' object is not callable


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_io.py", line 1574, in test_threads
    random.shuffle(l)
    ~~~~~~~~~~~~~~^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 360, in shuffle
    j = randbelow(i + 1)
        ~~~~~~~~~^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 251, in _randbelow_with_getrandbits
    while r >= n:
          ^^^^^^
TypeError: '>=' not supported between instances of 'Random' and 'Random'


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 409, in test_bug_9025
    k = sum(randrange(6755399441055744) % 3 == 2 for i in range(n))
        ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 409, in <genexpr>
    k = sum(randrange(6755399441055744) % 3 == 2 for i in range(n))
            ~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 313, in randrange
    raise ValueError("empty range for randrange()")
ValueError: empty range for randrange()


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_io.py", line 1574, in test_threads
    random.shuffle(l)
    ~~~~~~~~~~~~~~^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 360, in shuffle
    j = randbelow(i + 1)
        ~~~~~~~~~^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 252, in _randbelow_with_getrandbits
    r = getrandbits(k)
        ~~~~~~~~~~~^^^
TypeError: 'Random' object is not callable


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/pickletester.py", line 2617, in test_framing_many_objects
    self.check_frame_opcodes(pickled)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/pickletester.py", line 2566, in check_frame_opcodes
    for op, arg, pos in pickletools.genops(pickled):
    ...<27 lines>...
            frame_end = pos + 9 + arg
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/pickletools.py", line 2285, in _genops
    raise ValueError("at position %s, opcode %r unknown" % (
                     "<unknown>" if pos is None else pos,
                     code))
ValueError: at position 141121, opcode b'M' unknown


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
                     "__main__", mod_spec)
                     ^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/worker.py", line 6, in <module>
    from test.support import os_helper, Py_DEBUG
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 15, in <module>
    import textwrap
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/textwrap.py", line 416, in <module>
    _whitespace_only_re = re.compile('^[ \t]+$', re.MULTILINE)
                          ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/__init__.py", line 289, in compile
    return _compile(pattern, flags)
           ~~~~~~~~^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/__init__.py", line 350, in _compile
    p = _compiler.compile(pattern, flags)
        ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_compiler.py", line 743, in compile
    p = _parser.parse(p, flags)
        ~~~~~~~~~~~~~^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 980, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 459, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       not nested and not items))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 886, in _parse
    op, av = subpattern[i]
    ^^^^^^
ValueError: not enough values to unpack (expected 2, got 0)


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 963, in test_sample_counts_equivalence
    s1 = sample(colors, counts=counts, k=k)
         ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 429, in sample
    selections = self.sample(range(total), k=k)
                 ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 445, in sample
    result[i] = pool[j]
                ~~~~^^^
TypeError: list indices must be integers or slices, not Random


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_unicodedata.py", line 393, in test_normalization
    self.run_normalization_tests(testdata)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_unicodedata.py", line 426, in run_normalization_tests
    self.assertTrue(c5 ==  NFD(c4) ==  NFD(c5), line)
                                       ~~~^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 690, in __call__
    return self.run(*args, **kwds)
           ~~~~~~~~^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 609, in run
    result.startTest(self)
    ^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'startTest'

bedevere-bot · 2024-03-11T23:28:10Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 MacOS M1 Refleaks NoGIL 3.x has failed when building commit 44f9a84.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1368/builds/460) and take a look at the build logs.
Check if the failure is related to this commit (44f9a84) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1368/builds/460

Failed tests:

test.test_asyncio.test_server

Failed subtests:

test_abort_clients - test.test_asyncio.test_server.TestServer2.test_abort_clients

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/async_case.py", line 93, in _callTestMethod
    if self._callMaybeAsync(method) is not None:
       ~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/async_case.py", line 115, in _callMaybeAsync
    return self._asyncioRunner.run(
           ~~~~~~~~~~~~~~~~~~~~~~~^
        func(*args, **kwargs),
        ^^^^^^^^^^^^^^^^^^^^^^
        context=self._asyncioTestContext,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/asyncio/base_events.py", line 721, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/test_asyncio/test_server.py", line 231, in test_abort_clients
    s_sock = s_wr.get_extra_info('socket')
             ^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get_extra_info'

gaogaotiantian · 2024-03-12T00:42:20Z

Could backporting break working code?

I don't think so. This change only affects .pdbrc and -c. The changes it made will not break any reasonable code. I can't think of any case that can be broken even if it works in a weird way.

miss-islington-app · 2024-03-12T09:02:42Z

Thanks @gaogaotiantian for the PR, and @iritkatriel for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington-app · 2024-03-12T09:02:42Z

Thanks @gaogaotiantian for the PR, and @iritkatriel for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington-app · 2024-03-12T09:02:46Z

Sorry, @gaogaotiantian and @iritkatriel, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 44f9a84b67c97c94f0d581ffd63b24b73fb79610 3.11

miss-islington-app · 2024-03-12T09:03:12Z

Sorry, @gaogaotiantian and @iritkatriel, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 44f9a84b67c97c94f0d581ffd63b24b73fb79610 3.12

bedevere-app · 2024-03-12T15:49:34Z

GH-116660 is a backport of this pull request to the 3.11 branch.

…ble tests (pythonGH-110496) (cherry picked from commit 44f9a84) Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>

bedevere-app · 2024-03-12T15:58:43Z

GH-116661 is a backport of this pull request to the 3.12 branch.

#116661) [3.12] gh-90095: Make .pdbrc work properly and add some reasonable tests (GH-110496) (cherry picked from commit 44f9a84)

#116660) [3.11] gh-90095: Make .pdbrc work properly and add some reasonable tests (GH-110496) (cherry picked from commit 44f9a84)

…ts (python#110496)

gaogaotiantian added 2 commits October 6, 2023 22:19

Make .pdbrc work properly and add some reasonable tests

fdd36d2

Add comment test

3f3cd7d

bedevere-app bot added the awaiting review label Oct 7, 2023

bedevere-app bot mentioned this pull request Oct 7, 2023

Pdb can't use the commands through -c or .pdbrc files #90095

Closed

gaogaotiantian requested review from methane and iritkatriel October 7, 2023 05:37

gaogaotiantian and others added 3 commits October 6, 2023 23:14

Fix homesave issue

33225ee

📜🤖 Added by blurb_it.

2e1bc86

Merge branch 'main' into pdb-rc-commands

78f953a

brandtbucher self-assigned this Oct 26, 2023

brandtbucher added type-bug An unexpected behavior, bug, or error stdlib Python modules in the Lib dir labels Oct 26, 2023

gaogaotiantian added 3 commits February 8, 2024 13:43

Merge branch 'main' into pdb-rc-commands

6761844

Revert emplty line change

e66eb76

Fix a bare return

f07e770

iritkatriel reviewed Mar 10, 2024

View reviewed changes

Lib/test/test_pdb.py Show resolved Hide resolved

Lib/test/test_pdb.py Show resolved Hide resolved

Lib/pdb.py Show resolved Hide resolved

gaogaotiantian and others added 2 commits March 10, 2024 17:34

Clean up logic for remove_home

0b1c0e8

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>

Check homesave to restore HOME

75d1361

iritkatriel approved these changes Mar 11, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting review labels Mar 11, 2024

Merge branch 'main' into pdb-rc-commands

a5187a5

iritkatriel merged commit 44f9a84 into python:main Mar 11, 2024
32 checks passed

bedevere-app bot removed the awaiting merge label Mar 11, 2024

gaogaotiantian deleted the pdb-rc-commands branch March 11, 2024 21:56

iritkatriel added needs backport to 3.11 only security fixes needs backport to 3.12 bug and security fixes labels Mar 12, 2024

miss-islington-app bot assigned iritkatriel and unassigned brandtbucher Mar 12, 2024

gaogaotiantian mentioned this pull request Mar 12, 2024

[3.11] gh-90095: Make .pdbrc work properly and add some reasonable te… #116660

Merged

bedevere-app bot removed the needs backport to 3.11 only security fixes label Mar 12, 2024

gaogaotiantian mentioned this pull request Mar 12, 2024

[3.12] gh-90095: Make .pdbrc work properly and add some reasonable te… #116661

Merged

bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Mar 12, 2024

iritkatriel pushed a commit that referenced this pull request Mar 12, 2024

[3.12] gh-90095: Make .pdbrc work properly and add some reasonable te… (

d19c2eb

#116661) [3.12] gh-90095: Make .pdbrc work properly and add some reasonable tests (GH-110496) (cherry picked from commit 44f9a84)

iritkatriel pushed a commit that referenced this pull request Mar 12, 2024

[3.11] gh-90095: Make .pdbrc work properly and add some reasonable te… (

b17d347

#116660) [3.11] gh-90095: Make .pdbrc work properly and add some reasonable tests (GH-110496) (cherry picked from commit 44f9a84)

adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024

pythongh-90095: Make .pdbrc work properly and add some reasonable tes…

3582718

…ts (python#110496)

diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024

pythongh-90095: Make .pdbrc work properly and add some reasonable tes…

10b2399

…ts (python#110496)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-90095: Make .pdbrc work properly and add some reasonable tests #110496

gh-90095: Make .pdbrc work properly and add some reasonable tests #110496

gaogaotiantian commented Oct 7, 2023 •

edited by bedevere-app bot

Delengowski commented Mar 9, 2024

gaogaotiantian commented Mar 11, 2024

iritkatriel commented Mar 11, 2024

Delengowski commented Mar 11, 2024

bedevere-bot commented Mar 11, 2024

bedevere-bot commented Mar 11, 2024

gaogaotiantian commented Mar 12, 2024

miss-islington-app bot commented Mar 12, 2024

miss-islington-app bot commented Mar 12, 2024

miss-islington-app bot commented Mar 12, 2024

miss-islington-app bot commented Mar 12, 2024

bedevere-app bot commented Mar 12, 2024

bedevere-app bot commented Mar 12, 2024

gh-90095: Make .pdbrc work properly and add some reasonable tests #110496

gh-90095: Make .pdbrc work properly and add some reasonable tests #110496

Conversation

gaogaotiantian commented Oct 7, 2023 • edited by bedevere-app bot

Delengowski commented Mar 9, 2024

gaogaotiantian commented Mar 11, 2024

iritkatriel commented Mar 11, 2024

Delengowski commented Mar 11, 2024

bedevere-bot commented Mar 11, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 11, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

gaogaotiantian commented Mar 12, 2024

miss-islington-app bot commented Mar 12, 2024

miss-islington-app bot commented Mar 12, 2024

miss-islington-app bot commented Mar 12, 2024

miss-islington-app bot commented Mar 12, 2024

bedevere-app bot commented Mar 12, 2024

bedevere-app bot commented Mar 12, 2024

gaogaotiantian commented Oct 7, 2023 •

edited by bedevere-app bot