CPython 2.7.5 hacked to support finer-grained tracing and debugging
Python C Objective-C Assembly Shell TeX Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
Python-2.7.5
screenshots
tests
.gitignore
README.md

README.md

Py2crazy

Py2crazy is a modified version of the official Python interpreter (CPython 2.7.5) that supports finer-grained tracing and debugging.

It implements the following four features:

  1. Each Python bytecode instruction maps to a range of line and column numbers in the portion of the source code that compiled into that instruction.

  2. Debugger applications (such as those built upon bdb) call the trace function at each executed bytecode rather than at each executed source line.

  3. Peephole optimizations and opcode prediction macros are disabled so that source code matches more closely with bytecode. Doing so makes stepping through executed bytecodes appear more intuitive, since steps aren't "magically" skipped.

  4. The frame object exposes a new f_valuestack field, which is a list containing the current values on the expression stack used by the Python bytecode VM. This field allows debugging and tracing tools to inspect intermediate results of expression evaluation.

Why would anyone do this?

I created Py2crazy to support finer-grained expression-level tracing in Online Python Tutor. This wiki page discusses some of the design rationale. To illustrate,

(For a more intricate example, here's recursive factorial visualized with regular Python and Py2crazy.)

Precise line and column info in bytecodes

Here's an illustration of the first (and most significant) feature. If you compile this code

x = 5
y = 13
if (x + 5 > 7) and (y - 3 == 10):
    print 'You win'

with regular Python 2.7.5 and disassemble it (python -m dis), you get roughly the following bytecode:

compiled with Python

Note that each bytecode instruction maps to one line of source code.

In contrast, if you compile this code with Py2crazy and disassemble, you can see that each bytecode maps not only to a line, but also to a precise range of columns within that line (highlighted in yellow):

compiled with Py2crazy

This level of detail makes it possible to create much more fine-grained tracing and debugging tools, such as expression-level stepping for Online Python Tutor.

How do you view line and column number info?

Compile Py2crazy:

cd Python-2.7.5/
./configure
make

and then run the special "Super Disassembler" module (Python-2.7.5/Lib/super_dis.py) on a Python source file:

Py2crazy/Python-2.7.5/python -m super_dis test.py

Most terminals support colors, so you should be able to see the yellow highlights.

To programmatically access this data, import super_dis and hook into the proper functions from your code.

(Note that super_dis.py works only with Py2crazy, not with regular Python.)

How does Py2crazy debugger stepping differ from regular Python stepping?

Normally, the debugger interface (bdb) registers a tracing function into the regular CPython interpreter and steps through the target program roughly one line at a time.

However, when run in Py2crazy, bdb steps one bytecode instruction at a time, which provides much finer-grained tracing.

To see the difference, run pdb (the standard Python debugger built upon bdb) on a test file in both regular Python

python -m pdb test.py

and Py2crazy:

Py2crazy/Python-2.7.5/python -m pdb test.py

What did you change in CPython 2.7.5?

Check out the Git repo and run

git diff d36dfc8ffaf5337adb96bd582e0733fe2ffe3f02

to see diffs against a fresh Python 2.7.5 source distribution.

Caveat: Although you might find some ideas in Py2crazy to be useful, its design is ultimately driven by pedagogical goals, not by industrial-strength debugging goals. For instance, run-time efficiency wasn't a concern.

Footer

Created on 2013-07-03 by Philip Guo (philip@pgbovine.net)

Special thanks to Ned Batchelder for inspiring this project, and for providing technical guidance.