regression in pdb output between 2.7 and 3.5

[dhellmann]

BPO	26053
Nosy	@gvanrossum, @vstinner, @taleinat, @dhellmann, @xdegaye, @miss-islington, @iritkatriel
PRs	bpo-26053: Fix args echoed by pdb run command #22033 bpo-26053: Correct assertion in pdb test #25139 [3.9] bpo-26053: Fix args echoed by pdb run command  #25149 [3.8] [3.9] bpo-26053: Fix args echoed by pdb run command (GH-25149) #25150
Files	pdb_run.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-04-02.11:39:55.108>
created_at = <Date 2016-01-08.23:43:41.249>
labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
title = 'regression in pdb output between 2.7 and 3.5'
updated_at = <Date 2021-04-02.11:39:55.108>
user = 'https://github.com/dhellmann'

bugs.python.org fields:

activity = <Date 2021-04-02.11:39:55.108>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2021-04-02.11:39:55.108>
closer = 'iritkatriel'
components = ['Library (Lib)']
creation = <Date 2016-01-08.23:43:41.249>
creator = 'doughellmann'
dependencies = []
files = ['41538']
hgrepos = []
issue_num = 26053
keywords = ['patch', '3.5regression']
message_count = 14.0
messages = ['257785', '257786', '376164', '380404', '380405', '380407', '389994', '390006', '390026', '390040', '390043', '390051', '390052', '390053']
nosy_count = 7.0
nosy_names = ['gvanrossum', 'vstinner', 'taleinat', 'doughellmann', 'xdegaye', 'miss-islington', 'iritkatriel']
pr_nums = ['22033', '25139', '25149', '25150']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26053'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[dhellmann]

Under python 2.7 using the "run" command within pdb and passing it arguments causes those arguments to be printed out. Under 3.5, this is no longer true.

$ python2.7 -m pdb pdb_run.py
> /Users/dhellmann/Dropbox/PyMOTW/Python3/pymotw-3/source/pdb/pdb_run.py(7)<module>()
-> import sys
(Pdb) c
('Command-line args:', ['pdb_run.py'])
The program finished and will be restarted
> /Users/dhellmann/Dropbox/PyMOTW/Python3/pymotw-3/source/pdb/pdb_run.py(7)<module>()
-> import sys
(Pdb) run a b c "this is a long argument"
Restarting pdb_run.py with arguments:
	a b c this is a long argument
> /Users/dhellmann/Dropbox/PyMOTW/Python3/pymotw-3/source/pdb/pdb_run.py(7)<module>()
-> import sys



$ python3.5 -m pdb pdb_run.py
> /Users/dhellmann/Dropbox/PyMOTW/Python3/pymotw-3/source/pdb/pdb_run.py(7)<module>()
-> import sys
(Pdb) c
Command-line args: ['pdb_run.py']
The program finished and will be restarted
> /Users/dhellmann/Dropbox/PyMOTW/Python3/pymotw-3/source/pdb/pdb_run.py(7)<module>()
-> import sys
(Pdb) run a b c "this is a long argument"
Restarting pdb_run.py with arguments:
	pdb_run.py
> /Users/dhellmann/Dropbox/PyMOTW/Python3/pymotw-3/source/pdb/pdb_run.py(7)<module>()
-> import sys

It looks like the issue is in the pdb main loop. Under 2.7 the restart logic has:

except Restart:
    print "Restarting", mainpyfile, "with arguments:"
    print "\t" + " ".join(sys.argv[1:])

but under 3.5 that was changed to:

except Restart:
    print("Restarting", mainpyfile, "with arguments:")
    print("\t" + " ".join(args))

The do_run() method has already reset sys.argv before throwing Restart, so to print the correct arguments sys.argv[1:] should be used.

[dhellmann]

I should also mention that I haven't tested early versions of 3.x to see where exactly the regression was introduced.

[iritkatriel]

The regression was introduced here: e023091

in main() it prints " ".join(sys.argv[1:]) instead of " ".join(args).

In do_run (line 1029) sys.argv gets updated with the arg for that particular "run" command: sys.argv = shlex.split(arg)

The quickest fix would be to change main back to print sys.argv rather than arg. But I'm a little uneasy with a program modifying sys.argv (I wouldn't do it). So I'm wondering if that should be changed. What do you think?

[taleinat]

I'm also uneasy about the sys.argv mangling.

It seem to me that it would be safer, and more explicit, to pass the desired arguments in the Restart exception instance.

[iritkatriel]

I like the idea of adding it to the Restart exception. Should we do this refactor in the same PR as fixing the regression or keep them separate?

[taleinat]

Since it's a different implementation approach, I'd prefer a separate PR.

Whether we merge the existing one depends on whether we'd like to backport a fix for this to 3.9 and 3.8.

[gvanrossum]

New changeset 652bfde by Irit Katriel in branch 'master':
bpo-26053: Fix args echoed by pdb run command (bpo-22033)
652bfde

[vstinner]

New changeset bd4ab8e by Irit Katriel in branch 'master':
bpo-26053: Fix test_pdb.test_issue26053() (GH-25139)
bd4ab8e

[gvanrossum]

Do you need my help here?

[iritkatriel]

It’s fixed now. The tests failed when we merged this old PR. I guess they passed a few months ago but something changed in the meantime.
I don’t know if there’s something to do in the CI to prevent this (expire the test run after a while?)

If we want to backport then we need both PRs.

[vstinner]

If we want to backport then we need both PRs.

Commit 652bfde is a bugfix and this issue is marked as Python 3.8-3.10, so yeah, a backport is worth it.

You can create a PR for Python 3.9 combining the two commits. I like to use "git cherry-pick -x <sha1>" manually for that. And then, use the label to backport this PR to Python 3.8.

[vstinner]

New changeset 7ad56e2 by Irit Katriel in branch '3.9':
[3.9] bpo-26053: Fix args echoed by pdb run command (GH-25149)
7ad56e2

[miss-islington]

New changeset 2049bb2 by Miss Islington (bot) in branch '3.8':
[3.9] bpo-26053: Fix args echoed by pdb run command (GH-25149)
2049bb2

[iritkatriel]

Thanks, Victor.

I created a workflow issue for the CI question:
python/core-workflow#393