Add gdb7 hooks to make it easier to debug Python #52280
Comments
|
gdb 7 can be extended with Python code, allowing the writing of domain-specific pretty-printers and commands. I've been working on gdb 7 hooks to make it easier to debug python itself, as mentioned here: I'm attaching a patch for merger with "trunk". My hope is to be the maintainer of this work, although I do not yet have SVN commit rights (see http://mail.python.org/pipermail/python-committers/2010-February/000711.html ) The code is fully compatible with the existing "Misc/gdbinit" macros - you can freely use both as needed. = Two versions of Python = As I understand things, gdb only supports embedding Python 2 at the moment. This code is thus targeting that version of Python. So far, I've attempted to keep this code so that it will run when the "inferior process" is either Python 2 or Python 3. I could vary this code in the py3k branch if desired. The code would then track the "inferior process" version of Python, so that code to debug Python 3 would live in the py3k branch. That code would still be written for Python 2, though. = Makefile changes = The Makefile installs the gdb hooks to -gdb.py relative to the built python (even if you're not using them), which some may find irritating. It needs to do this for the test_gdb.py selftest to work (and for the gdb hooks to be usable if you're actually using them to debug your build of Python). Should I write a gdb configuration test to check for the availability of gdb built with python? I've added the file to .hgignore and .bzrignore. IIRC, a similar thing can be done to the SVN metadata (I don't think this is expressable as a patch, though). Alternatively, I could wire up the gdb tests to load the file from its location in the source tree. However, I intend for this code to be installed to a location alongside the build Python, so that it can be automatically detected and used by gdb. Typically this means copying it to the path of the ELF file with a -gdb.py file. In my RPM builds I add an extra copy, locating it relative to the location of the stripped DWARF data (e.g. /usr/lib/debug/usr/lib64/libpython26.so.1.0-gdb.py) = Selftests = The tests take about 14 seconds to run on my box: ---------------------------------------------------------------------- OK real 0m13.599s = Platform support = = Legal stuff = Earlier versions of this code were licensed under the LGPL 2.1 I'm in the process of doing the PSF Contributor Agreement paperwork (as an individual); waiting to get my hands on a fax machine. My employer, Red Hat, has agreed for me to retain copyright on all contributions I make to Python. |
|
(I faxed in my contributor agreement to the PSF on 2010-03-03) |
|
Maybe I'm using it incorrectly, but my first attempt to use it failed: gdb) b PyEval_EvalCodeEx Breakpoint 1, PyEval_EvalCodeEx (co=0xb7d86928, globals=Traceback (most recent call last):
File "/home/martin/work/27/python-gdb.py", line 574, in to_string
return stringify(proxyval)
File "/home/martin/work/27/python-gdb.py", line 526, in stringify
return repr(val)
File "/home/martin/work/27/python-gdb.py", line 271, in __repr__
for arg, val in self.attrdict.iteritems()])
AttributeError: 'FakeRepr' object has no attribute 'iteritems'
, locals=<unknown at remote 0x0>, args=0xb7d7eb00, argcount=1, kws=0x0, kwcount=0, defs=0x827c9b0, defcount=1, closure=<unknown at remote 0x0>)
at Python/ceval.c:3020
3020 register PyObject *retval = NULL;This was with the Python 2.7 trunk. |
|
Attribution question: would it be ok if all mentioning of your name in the patch was removed accept for the one in Misc/ACKS? I'm concerned that statements of authorship may become incorrect over time (if people start contributing to it), yet nobody will dare to remove your copyright notice (for example). |
|
Martin: thanks for reviewing this. Re msg100537: sorry about the inauspicious start. I've added some bulletproofing for the case you discovered, and added two new unit tests. Re msg100538: OK. I've removed my name and the copyright notice in all places apart from the Misc/ACKS file. There are still some comments in the code where I use the first person singular. I'm attaching an updated patch against trunk |
|
I find the printing of frame objects confusing: 3033 f = PyFrame_New(tstate, co, globals, locals); I didn't recognize that this is actually the output of the Frame; I recommend something like Frame %x, for file .... Also, it prints NULL PyObject* as "<unknown at remote 0x0>". I think null pointers should be special cased, and just be printed as 0x0. Also, what is the "remote" keyword? Aren't all pointers remote in this application? I'd hope that local Python objects never show up. |
|
Thanks for reviewing the patch. I've changed the pretty-printing of NULL pointers to "0x0" as suggested, and I've updated frame printing. Frames are now printed like this ...so that in a gdb backtrace you might see: Is this what you had in mind? As for the use of the word "remote",this is mostly for my own sanity, but I suspect it is needed. My rationale for this is that there are two address spaces at play: the local address space with the gdb process, and the remote one within the inferior process. It's possible for addresses with the gdb process to be printed for any object with a NULL tp_repr in its class, and if it were, it would be confusing which address space is it. For example, if within gdb I run this command: the hexadecimal value that's printed is an address within gdb's address space, that of the python object wrapping the gdb frame information. I did deliberately model the FakeRepr representation on the output of PyObject_Repr when tp_repr is NULL:
PyString_FromFormat("<%s object at %p>", Py_TYPE(v)->tp_name, v);
adding the "remote" word to disambiguate things.Is that OK? It seemed to me like the best compromise of readability and lack of ambiguity. I'm attaching an updated version of the patch (version 3), and am about to attach a diff against the last version (which was version 2) There are some other substantial changes in the patch:
I split out the test cases into multiple classes, invoking them all. All pass on my system, taking under 20 seconds ("Ran 28 tests in 17.770s") |
|
Attaching diff from v2 to v3 |
|
I'm attaching a new version of the patch (v4), against svn trunk (r79422) Changes since v3:
|
|
(adding diff from v3 to v4, for ease of review) |
|
I'm attaching a new version of the patch (v5), against svn trunk (r79517) I've been testing these hooks by using "gdb attach" to attach to real-world python programs. When doing this with earlier versions of the hooks, gdb would pause for many seconds when printing some values (e.g. during a backtrace). The issue was that "proxyval" scrapes the entire graph of python objects from the inferior process into gdb, and then prints it. As well as being slow, this can be many pages of textual output. To counter this, I've rewritten the pretty-printers to use a stream-based approach. The various PyObjectPtr subclasses have a "write_repr" method that writes the represenation to a file-like object whilst they recurse. I supply a file-like object "TruncatedStringIO", which raises an exception after a limit is reached: by default 1K of string data. (some simple subclasses simply reuse repr(self.proxyval()) for this, but objects that can follow cross-references have their own write_repr implementation). I've also rewritten frame handling: I eliminated the FrameInfo class, moving its functionality to the PyFrameObjectPtr class, and introduced a new "Frame" class to wrap gdb.Frame. This allowed a big simplification of the extension commands, and fixed various bugs (involving inlining and optimization) Within the py-up/py-down and py-bt commands, the code now prefixes Python frame printing with C-level frame indexes, to better tie together the C-level and Python-level views of the stack. I've added a couple of new commands:
Other changes since v4:
All tests pass on my box (Fedora 12 x86_64 with gdb-7.0.1-33.fc12.x86_64): "Ran 45 tests in 17.439s" |
|
Adding diff from v4 to v5, for ease of review. For my reference, md5sum of v5's hooks: |
|
Thanks for the patch. I have committed it as r79548, with two modifications:
Please understand that this commit went to the trunk, which will be 2.7. After 2.7b1, no new features are acceptable, so you can only perform bug fixes on the code that is now committed. Please create new issues for any such bug fixes. Once the tests all pass, I'll merge this into 3.2. |
|
A nitpick: on OS X, the gdb script ends up being called: python.exe-gdb.py Is this intentional? If it is, then I'll add this filename to the svn:ignore property. (And also to make distclean, I guess. Is python-gdb.py currently deleted by a 'make distclean'?) |
That was unexpected, but re-reading Makefile.pre.in I see where it's coming from. Please do add it to svn:ignore. Looks like I forgot to add the removal of the file to the "distclean" target. Sorry about that. Presumably adding: Another nit I spotted: the buildbot is "unexpectedly" skipping test_gdb on Win32 (with "gdb not found in path"). Looking at _expectations in Lib/test/regrtest.py it strikes me that test_gdb is likely to be skipped on every configuration other than on "linux2") - should I wire that in with a decorator within test_gdb.py? To my knowledge, OS X doesn't ship with gdb 7 - though presumably someone could build their own copy, linking with the system Python, and if so, the python.exe-gdb.py file would then be of use in debugging the freshly-built ./python.exe |
|
Added python.exe-gdb.py to svn:ignore in r79616. |
That sounds right. On my OS X 10.6.3 machine, the system gdb is: Mark-Dickinsons-MacBook-Pro:py3k dickinsm$ gdb --version I have the impression that few people have got gdb 7 working on OS X yet, but I might be wrong. Certainly macports doesn't seem to offer it yet. I expect that'll change, though. [About make distclean]
Yes, that seems to work. Thanks! Added in r79617. |
|
Could I request that the python-gdb.py script not be put in the top level directory? It's a little annoying to have another executable script starting with "python" when you're trying to test the "python" executable. |
|
That is tricky to do. gdb will consider it automatically only if it is called <exename>-gdb.py. I agree that there should be a better solution, though. David, is it possible to somehow hook this into .gdbinit, with an arbitrary path? Benjamin, would a .gdbinit in the top-level build directory still be annoying? |
|
http://sourceware.org/gdb/current/onlinedocs/gdb/Auto_002dloading.html says "If this file exists and is readable, gdb will evaluate it as a Python script." So maybe it doesn't need to be executable? |
|
2010/4/3 Martin v. Löwis <report@bugs.python.org>:
That would be fine with me. |
|
I have now changed Makefile.pre.in to not install python-gdb.py as a script anymore; this seems to work fine. People will still need to remove there existing python-gdb.py (or make clean) to see this change. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: