pdb: fix for 1326406 (import __main__ pdb failure)

[isandler]

BPO	1429539
Nosy	@birkenfeld, @abalkin
Files	patch_main.v1 patch_main.v2 patch_main.v3

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-11-07.23:15:41.960>
created_at = <Date 2006-02-11.03:34:09.000>
labels = ['library']
title = 'pdb: fix for  1326406 (import  __main__ pdb failure)'
updated_at = <Date 2008-11-07.23:15:41.960>
user = 'https://bugs.python.org/isandler'

bugs.python.org fields:

activity = <Date 2008-11-07.23:15:41.960>
actor = 'belopolsky'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2006-02-11.03:34:09.000>
creator = 'isandler'
dependencies = []
files = ['6994', '6995', '6996']
hgrepos = []
issue_num = 1429539
keywords = ['patch']
message_count = 10.0
messages = ['49447', '49448', '49449', '49450', '49451', '49452', '49453', '49454', '49455', '75622']
nosy_count = 5.0
nosy_names = ['nobody', 'georg.brandl', 'isandler', 'belopolsky', 'jakamkon']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1429539'
versions = []

[isandler]

The patch allows pdb to debug program which
import from __main__

[jakamkon]

Logged In: YES
user_id=1491175

I think that exposing pdb's namespaces for debugged code is
dangerous.When debugged code have this kind of access he can
dynamic change pdb's behaviour without your control:

y.py:
die = """\
def destroy(x,y):
print 'Iam crashing your HOME and deleting your FILES'

Pdb.__dict__['do_break'] = destroy # pdb's break = destroy
"""
x.py:
# innocently looking code;)
import y
exec(y.puff)
print "X"

with your patch:
$ python2.5 -m pdb x.py
> /home/xyz/python/x.py(1)<module>()
-> import y
(Pdb) Pdb.__dict__['do_break']
<function do_break at 0xb7cafdf4>
(Pdb) break
(Pdb) n
> /home/xyz/python/x.py(2)<module>()
-> exec(y.puff)
(Pdb) n
> /home/xyz/python/x.py(3)<module>()
-> print "X"
(Pdb) Pdb.__dict__['do_break']
<function destroy at 0xb7cb81b4>
(Pdb) break
Iam crashing your HOME and deleting your FILES

I think that this patch can't be accepted due to above
reason.According to my advanced reaserch;) ( find Lib/ -name
'*.py' -exec grep 'from __main__ import' {} -ls \; ) 'from
__main__' is rare case so maybe it will be reasonable to
simply handle ImportError and print something like
'** 'from __main__ import' not supported' message.What do
you think?

[isandler]

Logged In: YES
user_id=971153

I do see your point (In fact it was me who submitted the
patch bpo-896011 which separated pdb namespace from the
program's -- and thus broke imports from __main__ ;-))..

I do want to bring a couple of points:

1. I don't think it matters whether a program can
   intentionally interfere with pdb...Even when pdb's namespace
   is separated, it's easy for the program to interfere with
   debugger.. (Or delete your home directory for that matter)

2. Importing from __main_ may not be common in the std lib,
   but that's simply because stdlib doesn't contain that many
   executable hence there are very few places where there is
   __main__ to import from.

google search for "from __main__ import" results in about 1M
hits.

3. Just for the record, profile module does not separate its
   namespace from programs's either...

So, basically, it boils down to this: what's worse breaking
imports from __main__ or risking accidental interference
between pdb and the program (e.g if your program redefines a
help symbol)...

As a middle ground it might be a good idea to expand the
patch to reduce pdb's dependency on module global symbols
and thus reducing the risk of interference.

What do you think?

[nobody]

Logged In: NO

1. Could you give some code examples for that?
   2,3. Did you notice that google search for "from __main__
   import" give hits similar to:
   t = Timer("test()", "from __main__ import test")
   in most situations?
   I think it's hard to value uses of "from..." based on google
   search or similar method.Maybe we shoud ask on python-list
   what are the others opinions?

As a middle ground it might be a good idea to expand the
patch to reduce pdb's dependency on module global symbols
I'am interesting how would you do that?

[jakamkon]

Logged In: YES
user_id=1491175

Sorry I forget to login in;)The comment below is from me.

[isandler]

Logged In: YES
user_id=971153

1. Could you give some code examples for that?

Do you mean examples of intentional interference with
debugger? Well, you could just traverse the stack and check
whether the program runs under debugger and then do anything
you want... But why do you think intentional interference
would ever be an issue? After all python is not a language
to write debugger-resistant applications ;-)

Anyway, here are some examples of unintentional interference:

1. If you need a custom version of std module, you can
   modify sys.path and then import the module.. Which works by
   itself. But if pdb is loaded first and imports the module,
   then it does not work...

2. Similar problem with any application which changes
   sys.stdout/sys.stdin (there is actually a SF bug for that)

3. Also I don't see how pdb in its current form can control
   any program which needs a full-screen control of the terminal...

4. Any program which tries to do any magic with stack and
   assumes that top level stack frame is the main application
   will not work under pdb (where top level stack frame is pdb)

---------------------------------------------------
And there is a whole separate bunch of intereference issues
when pdb restarts the program.

---------------------------------------------------

When a program does run in pdb's namespace (as would be the
case if this patch is applied), pdb could save copies of all
module global symbols which it needs and thus become immune
to the accidental overwriting of those symbols in the main
program...

There could be a better way...

[isandler]

Logged In: YES
user_id=971153

I'm attaching an alternative patch: the program stil runs in
__main__ namespace, but pdb gets imported first:

  import pdb
  pdb.main()

So the main program cann't accidentally stomp on pdb
internals (e.g by doing help=None)

(there is still a bit of namespace pollution in the main
program)

[isandler]

Logged In: YES
user_id=971153

Another iteration of the patch

Now __main__ namespace is explicitly initialized before the
program (re)starts. The new patch should both support
imports from __main__ AND separate program's and pdb's
namespaces..

As a side effect, __file__ will now be set correctly in the
main program

[birkenfeld]

Thanks for the patch, committed as rev. 54363.

[abalkin]

I am not sure whether it is appropriate to comment on a closed issue,
but if the tracker will take this comment, hopefully it will find the
right audience.

I have recently stumbled on this bug running python 2.5. While tracking
the problem down, I've noticed that some of the script running logic in
pdb and runpy is replicated in some aspects and slightly different in
others.

Thus both pdb and runpy pop sys.argv[0], but (after this patch) pdb
cleans __main__.__dict__ and repopulates it by executing the script
while runpy replaces sys.modules['__main__'] with a fresh module and
uses that module's namespace to run the script in.

Furthemore, runpy injects __loader__ in the "main" namespace while pdb
does not.

While I cannot point out any specific problems (other than inability to
debug applications in zipped packages with python -m pdb app.zip), I
believe pdb should use the same logic and preferably the same code as
runpy.

Finally a nit: why does _runscript use run("execfile(filename)") instead
of runcall(execfile, filename, gloabals_, locals_)?