Py2crazy is a modified version of the official Python interpreter (CPython 2.7.5) that supports finer-grained tracing and debugging.
It implements the following four features:
Each Python bytecode instruction maps to a range of line and column numbers in the portion of the source code that compiled into that instruction.
Debugger applications (such as those built upon
bdb) call the trace function at each executed bytecode rather than at each executed source line.
Peephole optimizations and opcode prediction macros are disabled so that source code matches more closely with bytecode. Doing so makes stepping through executed bytecodes appear more intuitive, since steps aren't "magically" skipped.
The frame object exposes a new
f_valuestackfield, which is a list containing the current values on the expression stack used by the Python bytecode VM. This field allows debugging and tracing tools to inspect intermediate results of expression evaluation.
Why would anyone do this?
Online Python Tutor with Py2crazy provides fine-grained expression-level stepping,
while regular Python provides only line-level stepping (like in an ordinary debugger).
Precise line and column info in bytecodes
Here's an illustration of the first (and most significant) feature. If you compile this code
x = 5 y = 13 if (x + 5 > 7) and (y - 3 == 10): print 'You win'
with regular Python 2.7.5 and disassemble it (
python -m dis), you get roughly the following bytecode:
Note that each bytecode instruction maps to one line of source code.
In contrast, if you compile this code with Py2crazy and disassemble, you can see that each bytecode maps not only to a line, but also to a precise range of columns within that line (highlighted in yellow):
This level of detail makes it possible to create much more fine-grained tracing and debugging tools, such as expression-level stepping for Online Python Tutor.
How do you view line and column number info?
cd Python-2.7.5/ ./configure make
and then run the special "Super Disassembler" module (Python-2.7.5/Lib/super_dis.py) on a Python source file:
Py2crazy/Python-2.7.5/python -m super_dis test.py
Most terminals support colors, so you should be able to see the yellow highlights.
To programmatically access this data,
import super_dis and hook into the proper functions from your code.
super_dis.py works only with Py2crazy, not with regular Python.)
How does Py2crazy debugger stepping differ from regular Python stepping?
Normally, the debugger interface (
bdb) registers a tracing function into the
regular CPython interpreter and steps through the target program roughly
one line at a time.
However, when run in Py2crazy,
bdb steps one bytecode
instruction at a time, which provides much finer-grained tracing.
To see the difference, run
pdb (the standard Python debugger built upon
on a test file in both regular Python
python -m pdb test.py
Py2crazy/Python-2.7.5/python -m pdb test.py
What did you change in CPython 2.7.5?
Check out the Git repo and run
git diff d36dfc8ffaf5337adb96bd582e0733fe2ffe3f02
to see diffs against a fresh Python 2.7.5 source distribution.
Caveat: Although you might find some ideas in Py2crazy to be useful, its design is ultimately driven by pedagogical goals, not by industrial-strength debugging goals. For instance, run-time efficiency wasn't a concern.
Special thanks to Ned Batchelder for inspiring this project, and for providing technical guidance.