Description
Bug report
Bug description:
I am reporting this bug on behalf of user Bradipo
in #python
on the Libera.chat IRC network. The user was trying to run a pytest
quite with pdb
and experiencing a confusing ImportError
associated with the -m
mark expression they were passing in.
It seems that the changes in 73ccfa2 (#109165) have changed the semantics of -m
(and -c
) so that any program being run with the debugger may not accept these arguments.
It is also worth mentioning that the traceback shown in the below reproduction does not manifest in the release build, which makes this problem harder to identify as there is no provenance for the error.
Reproduction
$ cat echo.py
import sys
print(sys.argv)
$ ./python -m pdb -m echo -m XYZ # simulating running `pytest -m XYZ` with the debugger
Traceback (most recent call last):
File "/home/jgerity/repos/cpython/Lib/pdb.py", line 166, in check
self._details
File "/home/jgerity/repos/cpython/Lib/functools.py", line 1014, in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "/home/jgerity/repos/cpython/Lib/pdb.py", line 174, in _details
return runpy._get_module_details(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen runpy>", line 142, in _get_module_details
ImportError: No module named XYZ
$ ./python -c "import sys; print(sys.version)"
3.13.0a0 (bisect/bad:73ccfa28c5e, Jun 13 2025, 18:26:14) [GCC 12.3.0]
Expected behavior:
$ ./python -m pdb -m echo -m XYZ # pdb prompt available at startup, -m in sys.argv
> /home/jgerity/repos/cpython/echo.py(1)<module>()
-> import sys
(Pdb) continue
['echo', '-m', 'XYZ']
The program finished and will be restarted
> /home/jgerity/repos/cpython/echo.py(1)<module>()
-> import sys
(Pdb) quit
$ ./python -c "import sys; print(sys.version)" # AKA 73ccfa2~1
3.13.0a0 (tags/v3.12.0b1-1758-g3e8fcb7df74:3e8fcb7df74, Jun 13 2025, 18:32:54) [GCC 12.3.0]
with thanks to @nedbat for the excellent suggestion to use echo.py
to cut pytest
out of the reproduction.
CPython versions tested on:
3.13, 3.12
Operating systems tested on:
Linux
Activity
SnoopJ commentedon Jun 13, 2025
I should note that this problem does also manifest for
./python -m pdb echo.py -m XYZ
gaogaotiantian commentedon Jun 13, 2025
Okay so this is a side effect of moving to
argparse
. To be honest I think this is a bug (that we probably can't fix) ofargparse
. The fundamental issue here is that the user passes two-m
arguments topdb
, and expects the second one to be automatically considered as the argument to the program being debugged. Even though this is kind of user-intuitive,argparse
does not deal with it well. When it is fed two-m
, it will just use the latter one (instead of raising an error).We can probably do something for it to re-select the first one, we already have some special logic for it.
The second case
./python -m pdb echo.py -m XYZ
is a bit different - it's actually more ambiguous. I agree that it's not backwards compatible with 3.12, but I'm not entirely convinced that this is supposed to work.I think a general rule of thumb is - if you are trying to pass an ambiguous argument (that both pdb and the script/module can parse), you should separate those with
--
. So./python -m pdb echo.py -- -m XYZ
is a proper way to debug.SnoopJ commentedon Jun 13, 2025
Hrm, I could have sworn I tried adding
--
between the module and the args during our discussion on IRC and it didn't work. I must have been doing something wrong, that does resolve this problem.I can see potential for a workaround by injecting
--
frompdb
but maybe that is too much of a kludge. Maybe this is in the direction you mean when you talk about "re-select the first one"?gaogaotiantian commentedon Jun 13, 2025
The effort would be when
-m
exists, do a lot of manual parse to make it intuitive (and backward compatible). The existing CLI arguments ofpdb
is relatively simple now so we can kind of do this, but I'm not sure if that's very maintainable in the future. I think it's safest for the user to explicitly separate the arguments with--
. However, at this point, I'll see if I can work something out. I don't want to re-write a parser.It should work work with
-m echo
-./python -m pdb -m echo -- -m XYZ
SnoopJ commentedon Jun 13, 2025
Understood. I'll write up a docs PR to add a warning to the
pdb
docs to at least give users a fighting chance to know that this sharp edge has been introduced. Edit: see #135520As an alternative to the maintenance burden that a "true fix" would create, do you think it's feasible to try and detect the case of multiple
-m
before calling intoargparse
and issue a suitable warning/error? This seems like it could be a workable compromise position that balances the complexity of the fix against the relatively unmovable nature of the problem.gaogaotiantian commentedon Jun 13, 2025
Let me take a look at the problem and see if we can find a reasonable fix - as we already have some manual parsing in
pdb
now. If we can make this work with understandable code, we can do that.SnoopJ commentedon Jun 13, 2025
Thanks. In the meantime, here's a failing test of the
-m echo
form.