gh-109039: Branch prediction for Tier 2 interpreter #109038

gvanrossum · 2023-09-06T23:42:19Z

Add cache entries to bytecodes.c and update them (but don't use them yet)
Make tests pass
Use cache entries for branch prediction
Add new tests
Initialize cache entries to 0x5555 (0b_0101_0101_0101_0101)
Buildbots
Benchmark

Issue: Branch prediction design for Tier 2 (uops) interpreter #109039

Tools/cases_generator/generate_cases.py

iritkatriel · 2023-09-07T10:19:02Z

Python/bytecodes.c

+            #if ENABLE_SPECIALIZATION
+            next_instr->cache = (next_instr->cache << 1) | flag;
+            #endif
+            JUMPBY(oparg * flag);


Don't you need also a SKIP_OVER the cache?

I'm guessing that could cause the assert(frame->prev_instr == instr); in _Py_call_instrumentation_jump to fail for the instrumented jumps.

That skip over the cache is already generated -- see the corresponding code in generated_cases.c.h.

You mean the "next_instr += 1;"? In all other cases there is an explicit SKIP_OVER(INLINE_CACHE_ENTRIES_...) in bytecodes.c.

Those SKIP_OVER() calls are always followed by a DISPATCH() call (or maybe a goto).

I see. Can we make the code generator emit SKIP_OVER(X) instead of next_instr += x;?

We can, though IIRC Mark at some point objected to emitting macros. So I'd rather keep the status quo.

@markshannon What is the reason not to emit macros?

A reason to emit them is so that they are implemented in one place, so if their implementation changes you only change there. Do we want to change the code generator (and to remember that we need to) every time the implementation of a macro like SKIP_OVER changes?

Honestly I don't expect SKIP_OVER() to ever change. In hand-written code the macro expresses the intent better. But in generated code it just obscures what happens. I had to go to some lengths to change PEEK() and POKE() calls in the generated code to using stack_pointer[x] instead; I don't want to go back. If you still disagree, try engaging @markshannon.

If you still disagree,

More like trying to understand than disagreeing.

try engaging @markshannon.

Yes I directed my previous comment to him.

Python/bytecodes.c

This is needed so branch prediction can work.

Alas, this goes untested (how to test it?). In INSTRUMENTED_POP_JUMP_IF_NOT_NONE, rename flag to nflag to emphasise that its sense is reversed (this is the only op that jumps if the flag is false, because there's no Py_IsNotNone() function). (Alternatively, we could have changed the sense of the flag, but that would have been more work.)

gvanrossum · 2023-09-08T00:45:43Z

Benchmark is running, will post results here. I think I've addressed all actionable review comments. Please review.

gvanrossum · 2023-09-08T03:54:41Z

Benchmark is neutral(*), which is a good thing (it means adding a cache entry to the branch instructions didn't slow anything down).

(*) Or possibly some benchmarks crashed. I'm going to run buildbots to be sure.

bedevere-bot · 2023-09-08T04:45:55Z

🤖 New build scheduled with the buildbot fleet by @gvanrossum for commit 1850988 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

iritkatriel · 2023-09-08T06:14:13Z

Tools/cases_generator/generate_cases.py

+                    family_member_names.update(family.members)
+                for instr in self.instrs.values():
+                    if (
+                        instr.name not in family_member_names


Why do we need to exclude family members from this table?

That's tradition -- the table only contains the data for family heads and is always consulted after looking up the deoptimized opcode in _PyOpcode_Deopt.

exclude family members from this table?

That's tradition

At first glance I thought this was some reference to a hypothetical family in the midst of conflict. 🙂

gvanrossum · 2023-09-08T15:57:58Z

Hm. Many buildbots fail on test_sys_settrace, but I can't (yet) reproduce it. Must be about build flags.

This time by disabling the optimizer.

AlexWaygood · 2023-09-08T16:24:44Z

Hm. Many buildbots fail on test_sys_settrace, but I can't (yet) reproduce it. Must be about build flags.

Unlikely that it's to do with this PR, as it's already happening on main -- see #109052 and #109143

gvanrossum · 2023-09-08T16:36:34Z

Unlikely that it's to do with this PR, as it's already happening on main -- see #109052 and #109143

Thanks, I'll not worry about it then.

Buildbots are beginning to lose their value for me. :-(

gvanrossum · 2023-09-11T17:31:29Z

I'll fix the conflict, then merge this.

gvanrossum · 2023-09-11T17:56:03Z

(Sorry, several reviewers got a review request because I changed the bytecode magic number. Please ignore.)

markshannon · 2023-10-30T10:58:18Z

This introduced a regression in branch and jump monitoring, as the target is off by one.
The line numbers in test_monitoring should be unchanged from 3.12.

gvanrossum · 2023-10-30T18:04:09Z

This introduced a regression in branch and jump monitoring, as the target is off by one. The line numbers in test_monitoring should be unchanged from 3.12.

Okay, can you give me a hint on what went wrong? I haven't been following how the instrumentation works in detail, and I have no idea which bits are being tested by the tests I modified, or what I should fix. (If it's involved, please open a new issue and CC me.)

markshannon · 2023-10-31T09:35:23Z

It is fixed in #111486, so nothing to worry about. Just putting it here for the record.

gvanrossum added 2 commits September 6, 2023 16:35

inst() and macro() may need cache size metadata

44db701

Add cache entry to *POP_JUMP_IF_* instructions

ff29ab3

gvanrossum changed the title ~~Branch prediction for Tier 2 interpreter~~ gh-109039: Branch prediction for Tier 2 interpreter Sep 6, 2023

bedevere-bot mentioned this pull request Sep 6, 2023

Branch prediction design for Tier 2 (uops) interpreter #109039

Closed

gvanrossum added the skip news label Sep 6, 2023

gvanrossum added 2 commits September 6, 2023 17:40

Fix test_dis (also fixes test_peepholer)

ebc91a2

Fix test_monitoring

072bb38

gvanrossum mentioned this pull request Sep 7, 2023

Stitching it all together faster-cpython/ideas#621

Open

Include pycore_bitutils.h in instrumentation.c

896ae53

iritkatriel reviewed Sep 7, 2023

View reviewed changes

Tools/cases_generator/generate_cases.py Show resolved Hide resolved

iritkatriel reviewed Sep 7, 2023

View reviewed changes

markshannon reviewed Sep 7, 2023

View reviewed changes

Python/bytecodes.c Show resolved Hide resolved

gvanrossum added 8 commits September 7, 2023 10:14

Follow likely jumps in trace

0eb5b90

Merge remote-tracking branch 'upstream/main' into count-branches

25bfb3d

Require 16 iterations before optimizing

a9c0805

This is needed so branch prediction can work.

Fix existing uops tests

7dfb94c

Add test for branch prediction

73eb60f

Initialize POP_JUMP_IF* counters to 0x5555

fbd322a

Simplify writing of _PyOpcode_Caches

1850988

gvanrossum marked this pull request as ready for review September 8, 2023 00:44

bedevere-bot added the awaiting core review label Sep 8, 2023

gvanrossum added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 8, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 8, 2023

iritkatriel reviewed Sep 8, 2023

View reviewed changes

Fix test_huntrleaks under -Xuops

4f1684c

This time by disabling the optimizer.

Fix test_dis under -Xuops

cb2cf12

Merge branch 'main' into count-branches

d74670c

gvanrossum enabled auto-merge (squash) September 11, 2023 17:36

gvanrossum disabled auto-merge September 11, 2023 17:45

Update magic number

41463a5

gvanrossum requested review from brettcannon, ericsnowcurrently, ncoghlan and warsaw as code owners September 11, 2023 17:54

gvanrossum enabled auto-merge (squash) September 11, 2023 17:54

gvanrossum removed request for brettcannon, warsaw, ncoghlan and ericsnowcurrently September 11, 2023 17:55

gvanrossum merged commit bcce5e2 into python:main Sep 11, 2023
23 of 24 checks passed

bedevere-app bot removed the awaiting core review label Sep 11, 2023

gvanrossum deleted the count-branches branch September 11, 2023 18:21

markshannon mentioned this pull request Sep 12, 2023

Adds stats for the tier 2 optimizer #109329

Closed

markshannon mentioned this pull request Oct 30, 2023

GH-111485: Increment next_instr consistently at the start of the instruction. #111486

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-109039: Branch prediction for Tier 2 interpreter #109038

gh-109039: Branch prediction for Tier 2 interpreter #109038

gvanrossum commented Sep 6, 2023 •

edited

iritkatriel Sep 7, 2023

gvanrossum Sep 7, 2023

iritkatriel Sep 7, 2023

gvanrossum Sep 7, 2023

iritkatriel Sep 7, 2023

gvanrossum Sep 7, 2023

iritkatriel Sep 7, 2023

gvanrossum Sep 7, 2023

iritkatriel Sep 7, 2023

gvanrossum commented Sep 8, 2023

gvanrossum commented Sep 8, 2023 •

edited

bedevere-bot commented Sep 8, 2023

iritkatriel Sep 8, 2023

gvanrossum Sep 8, 2023

ericsnowcurrently Sep 13, 2023

gvanrossum commented Sep 8, 2023

AlexWaygood commented Sep 8, 2023 •

edited

gvanrossum commented Sep 8, 2023

gvanrossum commented Sep 11, 2023

gvanrossum commented Sep 11, 2023

markshannon commented Oct 30, 2023

gvanrossum commented Oct 30, 2023

markshannon commented Oct 31, 2023

gh-109039: Branch prediction for Tier 2 interpreter #109038

gh-109039: Branch prediction for Tier 2 interpreter #109038

Conversation

gvanrossum commented Sep 6, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gvanrossum commented Sep 8, 2023

gvanrossum commented Sep 8, 2023 • edited

bedevere-bot commented Sep 8, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gvanrossum commented Sep 8, 2023

AlexWaygood commented Sep 8, 2023 • edited

gvanrossum commented Sep 8, 2023

gvanrossum commented Sep 11, 2023

gvanrossum commented Sep 11, 2023

markshannon commented Oct 30, 2023

gvanrossum commented Oct 30, 2023

markshannon commented Oct 31, 2023

gvanrossum commented Sep 6, 2023 •

edited

gvanrossum commented Sep 8, 2023 •

edited

AlexWaygood commented Sep 8, 2023 •

edited