Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-24340: Fix estimation of the code stack size. #5076

Merged
merged 10 commits into from
Jan 9, 2018

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Jan 1, 2018

Tests based on patch by Antoine Pitrou (GH #2827).

https://bugs.python.org/issue24340

@serhiy-storchaka serhiy-storchaka added the type-bug An unexpected behavior, bug, or error label Jan 1, 2018
@serhiy-storchaka serhiy-storchaka requested review from pitrou and a team January 1, 2018 20:06
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Jan 1, 2018
@pitrou
Copy link
Member

pitrou commented Jan 1, 2018

I'd rather see https://bugs.python.org/issue17611 finished than review another bytecode PR.

@serhiy-storchaka
Copy link
Member Author

I prefer to fix separate issues by separate commits. Fixing this issue will make patches for bpo-17611 simpler. Currently they contain changes for fixing this issue (not completely).

@serhiy-storchaka
Copy link
Member Author

In particularly this could limit changes in PyCompile_OpcodeStackEffect by only changes related to new and changed opcodes.

@pitrou
Copy link
Member

pitrou commented Jan 1, 2018

OTOH reviewing two bytecode and compiler PRs is more work than reviewing a single one. And I would like to see how Mark's approach works with this.

@serhiy-storchaka
Copy link
Member Author

Isn't reviewing reviewing two separate PRs for two issues easier than reviewing the single PR that mixes fixes of two issues in the same code? Or rather reviewing three PRs every of them fixing the same two issues by slightly different code?

@pitrou
Copy link
Member

pitrou commented Jan 2, 2018

Well, those three PRs will have to be regenerated because of conflicts after this PR gets merged.

In any case, if you want to go forward with this, I would like to see more comments (especially on opcodes with a variable stack effect).

@serhiy-storchaka
Copy link
Member Author

In this PR I use slightly different approach than in #5006, and it looks more correct to me. Now I better understand this code. I'll update #5006 accordingly.

@serhiy-storchaka
Copy link
Member Author

What information do you want be added?

Python/compile.c Outdated
maxdepth = target_depth;
instr->i_opcode == SETUP_EXCEPT)
{
depth = depth - 6;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not obvious why that is. SETUP_FINALLY and SETUP_EXCEPT merely call PyFrame_BlockSetup. Can you add a comment?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyFrame_BlockSetup doesn't affect the stack of values. The stack effect of these opcodes is 0 in normal flow.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add comments for all this? (why -6?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+6 (jump) - 6 = 0 (not jump)

Do you want to duplicate comments from PyCompile_OpcodeStackEffect()? Or just add a reference to PyCompile_OpcodeStackEffect()?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicating comments is fine. The file is large anyway.

Copy link
Member Author

@serhiy-storchaka serhiy-storchaka Jan 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case I'm going to rewrite this code in bpo-32455 to something like:

new_depth = depth + PyCompile_OpcodeStackEffectEx(opcode, oparg, 0);
if (new_depth > maxdepth)
    maxdepth = new_depth;
if (isjump) {
    target_depth = depth + PyCompile_OpcodeStackEffectEx(opcode, oparg, 1);
    maxdepth = stackdepth_walk(c, instr->i_target, target_depth, maxdepth);
}
depth = new_depth;

And all comments will be in a single place.

Python/compile.c Outdated
else if (instr->i_opcode == SETUP_WITH ||
instr->i_opcode == SETUP_ASYNC_WITH)
{
depth = depth - 5;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above. See comments in PyCompile_OpcodeStackEffect().

Python/compile.c Outdated
if (instr->i_opcode == FOR_ITER) {
target_depth = depth-2;
target_depth = depth - 2;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why. At best, FOR_ITER calls STACKADJ(-1) (when the iterator is exhausted).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See PyCompile_OpcodeStackEffect(). The effect is +1 in normal flow and -1 when jumps.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So where does -2 come from?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 - 2 = -1

@brettcannon brettcannon removed the request for review from a team January 3, 2018 19:31
@serhiy-storchaka
Copy link
Member Author

Could you please take a look again @pitrou? All stack effects are now defined in a single place.

@pitrou
Copy link
Member

pitrou commented Jan 9, 2018

This looks ok to me. I can't guarantee that the compiler changes for async for and try...except are right, but the test suite validates them.

@serhiy-storchaka serhiy-storchaka merged commit d4864c6 into python:master Jan 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants