Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-107265: Fix code_richcompare for ENTER_EXECUTOR case #108165

Merged
merged 4 commits into from
Aug 21, 2023

Conversation

corona10
Copy link
Member

@corona10 corona10 commented Aug 20, 2023

@corona10 corona10 changed the title gh-107265: Fix code_richcompare to lookup actual opcode when ENTER_EX… gh-107265: Fix code_richcompare for ENTER_EXECUTOR case Aug 20, 2023
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG, but could you add a test for this? You could add it to test_capi/test_misc.py in TestOptimizerAPI.

@corona10
Copy link
Member Author

corona10 commented Aug 21, 2023

LG, but could you add a test for this? You could add it to test_capi/test_misc.py in TestOptimizerAPI.

Thanks, I added the test, and if I understand the intention correctly.
co_instr.op.arg and cp_instr.op.arg should also be updated.
If not, comparing co_instr.cache == cp_instr.cache will be failed anyway.

Please let me know if I understand wrongly.

@corona10
Copy link
Member Author

corona10 commented Aug 21, 2023

pydoc test failure is unrelated to this PR, I checked it locally and asked to Discord too

Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the arg must also be restored, since it is overwritten (with the index of the executor to be used in this code object's array of executors). So it looks like writing the test was useful!

@corona10 corona10 enabled auto-merge (squash) August 21, 2023 05:03
@corona10 corona10 merged commit 4fdf3fd into python:main Aug 21, 2023
20 checks passed
@corona10 corona10 deleted the gh-107265 branch August 21, 2023 05:50
if (co_instr.op.code == ENTER_EXECUTOR) {
const int exec_index = co_instr.op.arg;
_PyExecutorObject *exec = co->co_executors->executors[exec_index];
co_instr.op.code = exec->vm_data.opcode;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing two code objects must not modify the code object.

The interaction between executor, the specializer and instrumentation is subtle and likely to break without care.
This will leak executors, as it flip-flops between JUMP_BACKWARDS and ENTER_EXECUTOR, or worse if an optimizer assumes that a single instruction will only be seen once.

if (cp_instr.op.code == ENTER_EXECUTOR) {
const int exec_index = cp_instr.op.arg;
_PyExecutorObject *exec = cp->co_executors->executors[exec_index];
cp_instr.op.code = exec->vm_data.opcode;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise

@markshannon
Copy link
Member

Please revert this. It will leak, and may not be safe.

@markshannon
Copy link
Member

#101346

@gvanrossum
Copy link
Member

@corona10 We can fix this in the hash PR you are working on.

@corona10
Copy link
Member Author

@corona10 We can fix this in the hash PR you are working on.

Okay got it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants