Skip to content

Conversation

@DinoV
Copy link
Contributor

@DinoV DinoV commented Dec 11, 2025

#105727 introduced the ability to have different types of executables. During the discussion of that there was mention of potentially adding support for JITs to be able to plug in as well. There's a similar disucssion of the idea over here: faster-cpython/ideas#575

This extends the existing set of executors so that PEP-523 JITs can hook into the interpreter frames and create their own "lightweight" frames which live on the stack with some minimal amount of initialization.

Currently the initialization required is probably more than idea:

  • in order for frames to appear complete they need to eagerly initialize instr_ptr
  • Some things assume f_funcobj is initialized
  • Because the frames are currently FRAME_OWNED_BY_THREAD various things look at stackpointer.

Presumably the next step after this PR would be to add a new frame owner which is complete but doesn't require these additional fields to be initialized.

The JIT executors are created with a fixed object that includes a callback to reify the frame, a code object, and an optional piece of state. The reifier is called in various places where the JIT needs to populate the frame - for example accessing globals, builtins, or instr_ptr. Currently the API just provides a single callback with no information on what's required but you could imagine we could have a finer grained control if it was desired - but reification of the frame is generally going to happen in the slow path so simplicity seems the best here.

The code object isn't strictly necessary but because profilers and other external introspection tools frequently want a minimal number of hops to determine a functions identity it's tracked here instead of making them head back to the function object and then read the code from there.

@DinoV DinoV requested a review from pablogsal December 11, 2025 01:13
@DinoV DinoV force-pushed the jit_exec branch 5 times, most recently from 9b66f20 to c9178d6 Compare December 11, 2025 18:39
@DinoV DinoV changed the title gh-xxxxx: Add new type of executable to support external JITs gh-142598: Add new type of executable to support external JITs Dec 11, 2025
@DinoV DinoV marked this pull request as ready for review December 11, 2025 22:04
@DinoV DinoV force-pushed the jit_exec branch 7 times, most recently from 01e3184 to 959ef86 Compare December 12, 2025 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant