Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-112354: Initial implementation of warm up on exits and trace-stitching #114142

Merged
merged 50 commits into from
Feb 20, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
1975b4c
Cold exits: Work in progress.
markshannon Jan 10, 2024
9fb97f7
Merge branch 'main' into cold-exits
markshannon Jan 11, 2024
f9aa235
Optimize on side exits
markshannon Jan 11, 2024
1288258
Merge branch 'main' into cold-exits
markshannon Jan 11, 2024
7c6267a
Modify internal interfaces
markshannon Jan 11, 2024
55c48e8
Merge branch 'main' into cold-exits
markshannon Jan 11, 2024
8b3c2e0
Jump to next executor without updating current_executor.
markshannon Jan 11, 2024
92a3b61
Support cycle GC for executors.
markshannon Jan 12, 2024
e3def48
Give cold exits their own class, to fix GC handling of exits
markshannon Jan 16, 2024
87e544b
Generate table of cold exits
markshannon Jan 16, 2024
2172d68
Treat EXIT_TRACE as a side exit
markshannon Jan 16, 2024
4448793
Treat most common guard failures as side exits
markshannon Jan 16, 2024
c70f12f
Tweak generated tble to help C analyzer
markshannon Jan 16, 2024
d73fe0a
Add some documentation about the tier 2 engine
markshannon Jan 16, 2024
5c8f0bd
Fix constness and rename hotness
markshannon Jan 17, 2024
140486b
Add new static objects to ignored file.
markshannon Jan 17, 2024
3362c93
Address review comments
markshannon Jan 18, 2024
63fe653
Transfer executor on thread-state and othe minor changes to be more j…
markshannon Feb 8, 2024
b0991a7
Merge branch 'main' into cold-exits
markshannon Feb 8, 2024
625bce2
Get side exits to build with jit enabled.
markshannon Feb 9, 2024
e191fd7
Initialize cold exits dynamically on demand
markshannon Feb 9, 2024
941a14c
Tidy tier 2 code a bit
markshannon Feb 9, 2024
cfd3285
Add Brandt's fixes
markshannon Feb 9, 2024
1025495
Free the correct amount of memory
markshannon Feb 9, 2024
171dad7
Merge branch 'main' into cold-exits
markshannon Feb 9, 2024
e6ca3fe
Remove unreachable code
markshannon Feb 9, 2024
308b2a7
Merge branch 'main' into cold-exits
markshannon Feb 9, 2024
518143e
Clear executors attached to exits when clearing executors
markshannon Feb 9, 2024
9d8cab8
Merge branch 'main' into cold-exits
markshannon Feb 9, 2024
bf07dad
Merge branch 'main' into cold-exits
markshannon Feb 9, 2024
19b6b84
Keep c-analyzer happy
markshannon Feb 9, 2024
c959e8f
Merge branch 'main' into cold-exits
markshannon Feb 14, 2024
f393ba5
Use threshold for side exits
markshannon Feb 14, 2024
bd66b01
Statically allocate cold exits
markshannon Feb 14, 2024
fe75484
Handle errors in JIT compile
markshannon Feb 14, 2024
3d0110c
Merge branch 'main' into cold-exits
markshannon Feb 14, 2024
de93130
Fix possible leak
markshannon Feb 14, 2024
77a6740
Fix refleak transfering from JIT to tier 1
markshannon Feb 14, 2024
0a61d29
Check that only one of EXIT_IF and DEOPT_IF is present
markshannon Feb 14, 2024
b3e306d
Address review comments
markshannon Feb 14, 2024
8f3aa33
Make exit_index 32 bits to avoid endianness issues in JIT
markshannon Feb 14, 2024
7c84967
Run black
markshannon Feb 15, 2024
8ee6710
Address code review
markshannon Feb 15, 2024
f37d7fc
Update comment
markshannon Feb 15, 2024
1f8967d
Address review comments
markshannon Feb 15, 2024
8e4c601
Fix compiler warning
markshannon Feb 15, 2024
4eb2cfc
Address review comments
markshannon Feb 15, 2024
ebe804f
Add missing brace
markshannon Feb 15, 2024
c38d4e8
Address review comments
markshannon Feb 15, 2024
830eb4e
Keep c-analyzer quiet
markshannon Feb 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
21 changes: 19 additions & 2 deletions Include/cpython/optimizer.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,29 @@ typedef struct {
uint64_t operand; // A cache entry
} _PyUOpInstruction;

typedef struct _exit_data {
uint32_t target;
markshannon marked this conversation as resolved.
Show resolved Hide resolved
int16_t temperature;
struct _PyExecutorObject *executor;
} _PyExitData;

typedef struct _PyExecutorObject {
PyObject_VAR_HEAD
const _PyUOpInstruction *trace;
_PyVMData vm_data; /* Used by the VM, but opaque to the optimizer */
_PyUOpInstruction trace[1];
uint32_t exit_count;
uint32_t code_size;
_PyExitData exits[1];
} _PyExecutorObject;

typedef struct _cold_exit {
_PyExecutorObject base;
_PyUOpInstruction uop;
} _PyColdExitObject;


markshannon marked this conversation as resolved.
Show resolved Hide resolved
extern _PyColdExitObject Py_FatalErrorExecutor;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I understand what this is for ("pretend we came from here and then do the tier 2 -> tier 2 switcheroo") this should probably have a different name, emphasizing what it's used for (maybe "dummy" something) rather that what it would do if executed (because it never will be, unless we have a bug).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've renamed it Py_NeverExecutedExecutor


typedef struct _PyOptimizerObject _PyOptimizerObject;

/* Should return > 0 if a new executor is created. O if no executor is produced and < 0 if an error occurred. */
Expand Down Expand Up @@ -72,7 +89,7 @@ PyAPI_FUNC(_PyOptimizerObject *) PyUnstable_GetOptimizer(void);
PyAPI_FUNC(_PyExecutorObject *) PyUnstable_GetExecutor(PyCodeObject *code, int offset);

int
_PyOptimizer_Optimize(struct _PyInterpreterFrame *frame, _Py_CODEUNIT *start, PyObject **stack_pointer);
_PyOptimizer_Optimize(struct _PyInterpreterFrame *frame, _Py_CODEUNIT *start, PyObject **stack_pointer, _PyExecutorObject **exec_ptr);

extern _PyOptimizerObject _PyOptimizer_Default;

Expand Down
54 changes: 28 additions & 26 deletions Include/internal/pycore_opcode_metadata.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions Include/internal/pycore_uop_ids.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 21 additions & 15 deletions Include/internal/pycore_uop_metadata.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.