Quickening interpreter

[markshannon]

This is a simple idea, but very powerful.

Once a code object has been executed a few times, say 8, create a new bytecode array for the instructions.
The new array is not exposed to the rest of the VM and definitely not to the C or Python APIs.

Because it is private, it can be freely modified at runtime, which allows a number of really nice features:

• Super-instructions can be added without complicating the compiler and associated tools, like dis.
• Adaptive (self-modifying) instructions are possible. All sorts of fun can be had here 😄
• Zero overhead debugging. A break-point can be inserted into the private array, at zero runtime cost until the breakpoint is hit.
• No-trace versions of instructions can be implemented that avoid testing use_tracing when it is known that tracing is off.
  • We only need to do LOAD_FAST, STORE_FAST, LOAD_CONST, POP_TOP, as they account for approx 45% of all instructions.
  • Super instructions composed only of the above are also implicitly "no-trace".

The quickened bytecode should have the same 2 byte format as normal bytecode.
That way only one interpreter is required and it avoids the complexity of switching between interpreters.

[gvanrossum]

Agreed. However, I'd like to expose this (in some read-only form) to the Python API so that dis can see it. This would be useful for debugging the optimizer.

[gvanrossum]

At some point this will require fixing up jump targets.

[ericsnowcurrently]

• Zero overhead debugging. A break-point can be inserted into the private array, at zero runtime cost until the breakpoint is hit.

The same is true for Python tracing/profiling, right?

That way only one interpreter is required and it avoids the complexity of switching between interpreters.

❤️

[markshannon]

At some point this will require fixing up jump targets.

No. Jump targets will remain the same. That's one of the reasons for not changing the bytecode format.

[markshannon]

• Zero overhead debugging. A break-point can be inserted into the private array, at zero runtime cost until the breakpoint is hit.

The same is true for Python tracing/profiling, right?

No, Python tracing/profiling is really expensive. The additional cost to add a breakpoint may be zero, but the total cost is large.
Quickening would allow breakpoints with near-zero total cost.

[markshannon]

Dumping the quickened code in some form might be useful for debugging, but I think that collecting stats is more useful.
We want to know if the optimizations are working effectively, not just correctly.

Adding it to dis makes it part of the public interface, which is IMO a bad idea.
We want to be able to change the behavior radically without backward compatibility concerns.

[ztane]

BTW Google has some really ugly hacks for inserting breakpoints into byte/wordcode of unmodified CPythons in google-python-cloud-debugger, would be super to have the same thing officially in the CPython API.