`JUMP_FORWARD` decompilation error

[Berbe]

Description

# uncompyle6 version 3.9.0a1
# Python bytecode version base 2.7 (62211)
# Decompiled from: Python 3.9.2 (default, Feb 28 2021, 17:03:44)
# [GCC 10.2.1 20210110]
# Embedded file name: test.5.KO.py
# Compiled at: 2022-10-01 00:49:19


class Test:

    def test--- This code section failed: ---

 L.   3         0  SETUP_LOOP           47  'to 50'
                3  LOAD_FAST             1  'a'
                6  LOAD_ATTR             0  'values'
                9  CALL_FUNCTION_0       0  None
               12  GET_ITER
               13  FOR_ITER             33  'to 49'
               16  STORE_FAST            2  'b'

 L.   4        19  LOAD_FAST             2  'b'
               22  POP_JUMP_IF_FALSE    13  'to 13'

 L.   5        25  LOAD_FAST             2  'b'
               28  STORE_FAST            1  'a'
               31  JUMP_FORWARD          3  'to 37'

 L.   7        34  CONTINUE             13  'to 13'
             37_0  COME_FROM            31  '31'

 L.   8        37  LOAD_FAST             2  'b'
               40  POP_JUMP_IF_FALSE    13  'to 13'

 L.   9        43  CONTINUE             13  'to 13'
               46  JUMP_BACK            13  'to 13'
               49  POP_BLOCK
             50_0  COME_FROM             0  '0'

Parse error at or near `LOAD_FAST' instruction at offset 37

How to Reproduce

Code

class Test():
    def test(self):
        for b in a.values():
            if b:
                a = b
            else:
                continue
            if b:
                pass

Output Given

Output

-- Stacks of completed symbols:
START ::= |- stmts .
_come_froms ::= \e__come_froms . COME_FROM
_ifstmts_jump ::= \e_c_stmts_opt . JUMP_FORWARD come_froms
_ifstmts_jump ::= c_stmts_opt . JUMP_FORWARD come_froms
_ifstmts_jump ::= c_stmts_opt JUMP_FORWARD . come_froms
_stmts ::= _stmts . stmt
_stmts ::= stmt .
and ::= expr . JUMP_IF_FALSE_OR_POP expr COME_FROM
and ::= expr . jmp_false expr \e_come_from_opt
and ::= expr . jmp_false expr come_from_opt
and ::= expr jmp_false . expr \e_come_from_opt
and ::= expr jmp_false . expr come_from_opt
and ::= expr jmp_false expr . come_from_opt
and ::= expr jmp_false expr \e_come_from_opt .
assert ::= assert_expr . jmp_true LOAD_ASSERT RAISE_VARARGS_1
assert2 ::= assert_expr . jmp_true LOAD_ASSERT expr CALL_FUNCTION_1 RAISE_VARARGS_1
assert_expr ::= assert_expr_and .
assert_expr ::= expr .
assert_expr_and ::= assert_expr . jmp_false expr
assert_expr_and ::= assert_expr jmp_false . expr
assert_expr_and ::= assert_expr jmp_false expr .
assert_expr_or ::= assert_expr . jmp_true expr
assign ::= expr . DUP_TOP designList
assign ::= expr . store
assign ::= expr store .
assign2 ::= expr . expr ROT_TWO store store
assign3 ::= expr . expr expr ROT_THREE ROT_TWO store store store
attribute ::= expr . GET_ITER
attribute ::= expr . LOAD_ATTR
attribute ::= expr GET_ITER .
attribute ::= expr LOAD_ATTR .
aug_assign1 ::= expr . expr inplace_op ROT_FOUR STORE_SLICE+3
aug_assign1 ::= expr . expr inplace_op ROT_THREE STORE_SLICE+1
aug_assign1 ::= expr . expr inplace_op ROT_THREE STORE_SLICE+2
aug_assign1 ::= expr . expr inplace_op ROT_THREE STORE_SUBSCR
aug_assign1 ::= expr . expr inplace_op ROT_TWO STORE_SLICE+0
aug_assign1 ::= expr . expr inplace_op store
aug_assign2 ::= expr . DUP_TOP LOAD_ATTR expr inplace_op ROT_TWO STORE_ATTR
bin_op ::= expr . expr binary_operator
c_stmts ::= _stmts .
c_stmts ::= _stmts . lastc_stmt
c_stmts_opt ::= c_stmts .
call ::= expr . CALL_FUNCTION_0
call ::= expr CALL_FUNCTION_0 .
call_stmt ::= expr . POP_TOP
classdefdeco1 ::= expr . classdefdeco1 CALL_FUNCTION_1
classdefdeco1 ::= expr . classdefdeco2 CALL_FUNCTION_1
come_froms ::= COME_FROM .
come_froms ::= come_froms . COME_FROM
compare_chained ::= expr . compare_chained1 ROT_TWO POP_TOP \e__come_froms
compare_chained ::= expr . compare_chained1 ROT_TWO POP_TOP _come_froms
compare_single ::= expr . expr COMPARE_OP
continue ::= CONTINUE .
continues ::= _stmts . lastl_stmt continue
continues ::= continue .
continues ::= lastl_stmt . continue
delete ::= expr . DELETE_ATTR
delete_subscript ::= expr . expr DELETE_SUBSCR
else_suite ::= suite_stmts .
expr ::= LOAD_FAST .
expr ::= attribute .
expr ::= call .
expr ::= get_iter .
expr_jitop ::= expr . JUMP_IF_TRUE_OR_POP
expr_jt ::= expr . jmp_true
for ::= SETUP_LOOP . expr for_iter store for_block POP_BLOCK \e__come_froms
for ::= SETUP_LOOP . expr for_iter store for_block POP_BLOCK _come_froms
for ::= SETUP_LOOP expr . for_iter store for_block POP_BLOCK \e__come_froms
for ::= SETUP_LOOP expr . for_iter store for_block POP_BLOCK _come_froms
for ::= SETUP_LOOP expr for_iter . store for_block POP_BLOCK \e__come_froms
for ::= SETUP_LOOP expr for_iter . store for_block POP_BLOCK _come_froms
for ::= SETUP_LOOP expr for_iter store . for_block POP_BLOCK \e__come_froms
for ::= SETUP_LOOP expr for_iter store . for_block POP_BLOCK _come_froms
for_block ::= \e_l_stmts_opt . JUMP_ABSOLUTE JUMP_BACK JUMP_BACK
for_block ::= \e_l_stmts_opt . JUMP_BACK
for_block ::= \e_l_stmts_opt . _come_froms JUMP_BACK
for_block ::= \e_l_stmts_opt \e__come_froms . JUMP_BACK
for_block ::= l_stmts_opt . JUMP_ABSOLUTE JUMP_BACK JUMP_BACK
for_block ::= l_stmts_opt . JUMP_BACK
for_block ::= l_stmts_opt . _come_froms JUMP_BACK
for_block ::= l_stmts_opt \e__come_froms . JUMP_BACK
for_iter ::= GET_ITER . COME_FROM FOR_ITER
for_iter ::= GET_ITER . FOR_ITER
for_iter ::= GET_ITER FOR_ITER .
forelsestmt ::= SETUP_LOOP . expr for_iter store for_block POP_BLOCK else_suite \e__come_froms
forelsestmt ::= SETUP_LOOP . expr for_iter store for_block POP_BLOCK else_suite _come_froms
forelsestmt ::= SETUP_LOOP expr . for_iter store for_block POP_BLOCK else_suite \e__come_froms
forelsestmt ::= SETUP_LOOP expr . for_iter store for_block POP_BLOCK else_suite _come_froms
forelsestmt ::= SETUP_LOOP expr for_iter . store for_block POP_BLOCK else_suite \e__come_froms
forelsestmt ::= SETUP_LOOP expr for_iter . store for_block POP_BLOCK else_suite _come_froms
forelsestmt ::= SETUP_LOOP expr for_iter store . for_block POP_BLOCK else_suite \e__come_froms
forelsestmt ::= SETUP_LOOP expr for_iter store . for_block POP_BLOCK else_suite _come_froms
genexpr_func ::= LOAD_FAST . FOR_ITER store comp_iter JUMP_BACK
get_iter ::= expr . GET_ITER
get_iter ::= expr GET_ITER .
if_exp ::= expr . jmp_false expr JUMP_ABSOLUTE expr
if_exp ::= expr . jmp_false expr JUMP_FORWARD expr COME_FROM
if_exp ::= expr jmp_false . expr JUMP_ABSOLUTE expr
if_exp ::= expr jmp_false . expr JUMP_FORWARD expr COME_FROM
if_exp ::= expr jmp_false expr . JUMP_ABSOLUTE expr
if_exp ::= expr jmp_false expr . JUMP_FORWARD expr COME_FROM
if_exp_lambda ::= expr . jmp_false expr return_if_lambda return_stmt_lambda LAMBDA_MARKER
if_exp_lambda ::= expr jmp_false . expr return_if_lambda return_stmt_lambda LAMBDA_MARKER
if_exp_lambda ::= expr jmp_false expr . return_if_lambda return_stmt_lambda LAMBDA_MARKER
if_exp_not ::= expr . jmp_true expr _jump expr COME_FROM
if_exp_not_lambda ::= expr . jmp_true expr return_if_lambda return_stmt_lambda LAMBDA_MARKER
if_exp_true ::= expr . JUMP_FORWARD expr COME_FROM
ifelsestmt ::= testexpr . c_stmts_opt JUMP_FORWARD else_suite come_froms
ifelsestmt ::= testexpr \e_c_stmts_opt . JUMP_FORWARD else_suite come_froms
ifelsestmt ::= testexpr c_stmts_opt . JUMP_FORWARD else_suite come_froms
ifelsestmt ::= testexpr c_stmts_opt JUMP_FORWARD . else_suite come_froms
ifelsestmt ::= testexpr c_stmts_opt JUMP_FORWARD else_suite . come_froms
ifelsestmt ::= testexpr c_stmts_opt JUMP_FORWARD else_suite come_froms .
ifelsestmtl ::= testexpr . c_stmts_opt CONTINUE else_suitel
ifelsestmtl ::= testexpr . c_stmts_opt JUMP_BACK else_suitel
ifelsestmtl ::= testexpr \e_c_stmts_opt . CONTINUE else_suitel
ifelsestmtl ::= testexpr \e_c_stmts_opt . JUMP_BACK else_suitel
ifelsestmtl ::= testexpr c_stmts_opt . CONTINUE else_suitel
ifelsestmtl ::= testexpr c_stmts_opt . JUMP_BACK else_suitel
ifelsestmtr ::= testexpr . return_if_stmts COME_FROM returns
iflaststmtl ::= testexpr . c_stmts
iflaststmtl ::= testexpr . c_stmts_opt JUMP_BACK
iflaststmtl ::= testexpr \e_c_stmts_opt . JUMP_BACK
iflaststmtl ::= testexpr c_stmts .
iflaststmtl ::= testexpr c_stmts_opt . JUMP_BACK
ifstmt ::= testexpr . _ifstmts_jump
ifstmt ::= testexpr . return_if_stmts COME_FROM
ifstmt ::= testexpr . return_stmts COME_FROM
jmp_false ::= POP_JUMP_IF_FALSE .
l_stmts ::= lastl_stmt .
l_stmts_opt ::= l_stmts .
lastl_stmt ::= iflaststmtl .
mkfuncdeco ::= expr . mkfuncdeco CALL_FUNCTION_1
mkfuncdeco ::= expr . mkfuncdeco0 CALL_FUNCTION_1
print_items_nl_stmt ::= expr . PRINT_ITEM \e_print_items_opt PRINT_NEWLINE_CONT
print_items_nl_stmt ::= expr . PRINT_ITEM print_items_opt PRINT_NEWLINE_CONT
print_items_stmt ::= expr . PRINT_ITEM \e_print_items_opt
print_items_stmt ::= expr . PRINT_ITEM print_items_opt
print_nl_to ::= expr . PRINT_NEWLINE_TO
print_to ::= expr . print_to_items POP_TOP
print_to_nl ::= expr . print_to_items PRINT_NEWLINE_TO
raise_stmt1 ::= expr . RAISE_VARARGS_1
raise_stmt2 ::= expr . expr RAISE_VARARGS_2
raise_stmt3 ::= expr . expr expr RAISE_VARARGS_3
ret_and ::= expr . JUMP_IF_FALSE_OR_POP return_expr_or_cond COME_FROM
ret_or ::= expr . JUMP_IF_TRUE_OR_POP return_expr_or_cond COME_FROM
return ::= return_expr . RETURN_VALUE
return_expr ::= expr .
return_expr_lambda ::= return_expr . RETURN_VALUE_LAMBDA
return_expr_lambda ::= return_expr . RETURN_VALUE_LAMBDA LAMBDA_MARKER
return_if_stmt ::= return_expr . RETURN_END_IF
return_if_stmts ::= _stmts . return_if_stmt
return_stmts ::= _stmts . return_stmt
returns ::= _stmts . return
slice0 ::= expr . DUP_TOP SLICE+0
slice0 ::= expr . SLICE+0
slice1 ::= expr . expr DUP_TOPX_2 SLICE+1
slice1 ::= expr . expr SLICE+1
slice2 ::= expr . expr DUP_TOPX_2 SLICE+2
slice2 ::= expr . expr SLICE+2
slice3 ::= expr . expr expr DUP_TOPX_3 SLICE+3
slice3 ::= expr . expr expr SLICE+3
stmt ::= assign .
stmt ::= continue .
store ::= STORE_FAST .
subscript ::= expr . expr BINARY_SUBSCR
subscript2 ::= expr . expr DUP_TOPX_2 BINARY_SUBSCR
suite_stmts ::= _stmts .
suite_stmts ::= continues .
testexpr ::= testfalse .
testfalse ::= expr . jmp_false
testfalse ::= expr jmp_false .
testtrue ::= expr . jmp_true
unary_convert ::= expr . UNARY_CONVERT
unary_not ::= expr . UNARY_NOT
unary_op ::= expr . unary_operator
while1elsestmt ::= SETUP_LOOP . l_stmts JUMP_BACK POP_BLOCK else_suitel COME_FROM
while1elsestmt ::= SETUP_LOOP . l_stmts JUMP_BACK else_suitel COME_FROM
while1stmt ::= SETUP_LOOP . l_stmts_opt CONTINUE COME_FROM
while1stmt ::= SETUP_LOOP . l_stmts_opt JUMP_BACK COME_FROM
while1stmt ::= SETUP_LOOP . l_stmts_opt JUMP_BACK POP_BLOCK COME_FROM
while1stmt ::= SETUP_LOOP . returns COME_FROM
while1stmt ::= SETUP_LOOP . returns pb_come_from
while1stmt ::= SETUP_LOOP \e_l_stmts_opt . CONTINUE COME_FROM
while1stmt ::= SETUP_LOOP \e_l_stmts_opt . JUMP_BACK COME_FROM
while1stmt ::= SETUP_LOOP \e_l_stmts_opt . JUMP_BACK POP_BLOCK COME_FROM
whileelsestmt ::= SETUP_LOOP . testexpr \e_l_stmts_opt JUMP_BACK POP_BLOCK else_suitel COME_FROM
whileelsestmt ::= SETUP_LOOP . testexpr l_stmts_opt JUMP_BACK POP_BLOCK else_suitel COME_FROM
whilestmt ::= SETUP_LOOP . testexpr \e_l_stmts_opt JUMP_BACK JUMP_BACK POP_BLOCK \e__come_froms
whilestmt ::= SETUP_LOOP . testexpr \e_l_stmts_opt JUMP_BACK JUMP_BACK POP_BLOCK _come_froms
whilestmt ::= SETUP_LOOP . testexpr \e_l_stmts_opt JUMP_BACK POP_BLOCK \e__come_froms
whilestmt ::= SETUP_LOOP . testexpr \e_l_stmts_opt JUMP_BACK POP_BLOCK _come_froms
whilestmt ::= SETUP_LOOP . testexpr l_stmts_opt JUMP_BACK JUMP_BACK POP_BLOCK \e__come_froms
whilestmt ::= SETUP_LOOP . testexpr l_stmts_opt JUMP_BACK JUMP_BACK POP_BLOCK _come_froms
whilestmt ::= SETUP_LOOP . testexpr l_stmts_opt JUMP_BACK POP_BLOCK \e__come_froms
whilestmt ::= SETUP_LOOP . testexpr l_stmts_opt JUMP_BACK POP_BLOCK _come_froms
whilestmt ::= SETUP_LOOP . testexpr returns \e__come_froms POP_BLOCK COME_FROM
whilestmt ::= SETUP_LOOP . testexpr returns _come_froms POP_BLOCK COME_FROM
with ::= expr . SETUP_WITH POP_TOP \e_suite_stmts_opt POP_BLOCK LOAD_CONST COME_FROM_WITH WITH_CLEANUP END_FINALLY
with ::= expr . SETUP_WITH POP_TOP suite_stmts_opt POP_BLOCK LOAD_CONST COME_FROM_WITH WITH_CLEANUP END_FINALLY
withasstmt ::= expr . SETUP_WITH store \e_suite_stmts_opt POP_BLOCK LOAD_CONST COME_FROM_WITH WITH_CLEANUP END_FINALLY
withasstmt ::= expr . SETUP_WITH store suite_stmts_opt POP_BLOCK LOAD_CONST COME_FROM_WITH WITH_CLEANUP END_FINALLY
yield ::= expr . YIELD_VALUE
Instruction context:
->
 L.   8        37  LOAD_FAST             2  'b'
                  40  POP_JUMP_IF_FALSE    13  'to 13'


# file test.5.KO.pyc
# Deparsing stopped due to parse error
test.5.KO.pyc --
# decompile failed

Additional information

If useful payload (here LOAD_FAST) is replaced with nothing (pass), the decompilation works.
For instance, this code is fine:

Code

class Test():
    def test(self):
        for b in a.values():
            if b:
                pass # No payload
            else:
                continue
            if b:
                pass

which disassembles in:

Disassembly

  3:           0 SETUP_LOOP           (to 44)
               3 LOAD_GLOBAL          (a)
               6 LOAD_ATTR            (values)
               9 CALL_FUNCTION        (0 positional, 0 named)
              12 GET_ITER
         >>   13 FOR_ITER             (to 43)
              16 STORE_FAST           (b)

  4:          19 LOAD_FAST            (b)
              22 POP_JUMP_IF_FALSE    (to 13)

  5:          25 JUMP_FORWARD         (to 31)

  7:          28 JUMP_ABSOLUTE        (to 13)

  8:     >>   31 LOAD_FAST            (b)
              34 POP_JUMP_IF_FALSE    (to 13)

  9:          37 JUMP_ABSOLUTE        (to 13)
              40 JUMP_ABSOLUTE        (to 13)
         >>   43 POP_BLOCK
         >>   44 LOAD_CONST           (None)
              47 RETURN_VALUE

[rocky]

I believe a fix to this is similar to the approach used in #408 where a new parse rule for "for-body" is needed along with possibly an expanded reduce rule to ensure this grammar rule targets only "for-body" kinds of things.

Given this information, maybe you or someone else can try to come up with a solution?

[Berbe]

I'm willing to try that!

I first need to understand what's wrong for the decompiler there: Is it the CONTINUE instruction replacing JUMP_ABSOLUTE to continue the loop which wreaks havoc?
I do not find any CONTINUE instruction in the docs, but rather a CONTINUE_LOOP

[Berbe]

Ah. I think I understand; correct me if I'm wrong.

In #408, you added a signature for for_loop having this signature:

for_block ::= \e_l_stmts_opt . JUMP_ABSOLUTE JUMP_BACK JUMP_BACK

What would be needed here is to address this kind of signature:

for_block ::= \e_l_stmts_opt . CONTINUE JUMP_BACK

[rocky]

Ah. I think I understand; correct me if I'm wrong.

In #408, you added a signature for for_loop having this signature:

for_block ::= \e_l_stmts_opt . JUMP_ABSOLUTE JUMP_BACK JUMP_BACK

Note that \e_ prefix is printed by the parser during execution. It indicates that in matching, this nonterminal matches the empty set of tokens. In other words, you won't find a rule that starts with \e_. Likewise the "." is something shown in tracing to show the match position and is not part of the rule either.

What would be needed here is to address this kind of signature:

for_block ::= \e_l_stmts_opt . CONTINUE JUMP_BACK

Yes, with the caveats mentioned above, I think so. But I haven't been following things too closely.

I do not find any CONTINUE instruction in the docs, but rather a CONTINUE_LOOP

(Re)read https://github.com/rocky/python-uncompyle6/wiki/How-does-this-code-work%3F#token-name-specialization and more generally how the decompiler works.

[Berbe]

I guess we're misled here, despite resemblance to #408.

Grammar display shows the following:

L.  4:  19-37  ifelsestmt ::= testexpr c_stmts_opt JUMP_FORWARD else_suite come_froms (14)
Reduce ifelsestmt invalid by check

matching:

 L.   4        19  LOAD_FAST             2  'b'
               22  POP_JUMP_IF_FALSE    13  'to 13'

 L.   5        25  LOAD_FAST             2  'b'
               28  STORE_FAST            1  'a'
               31  JUMP_FORWARD          3  'to 37'

 L.   7        34  CONTINUE             13  'to 13'
             37_0  COME_FROM            31  '31'

The problem is triggered on these lines :

python-uncompyle6/uncompyle6/parsers/reducecheck/ifelsestmt.py

Lines 242 to 243 in ae46ccf

	if first_offset > jmp_target:
	return True

Since CONTINUE refers to a location before the start of the if ... then ... else ... block, it is considered invalid. I reckon one would need to check being inside a loop... or just let CONTINUE instructions breaking out freely?

[rocky]

The nonterminals ifelsestmtl and ifelsestmtc should allow 'CONTINUE' while a plain ifelsesmt should not. So that test I guess needs to get made more complicated here.

Control flow handling is a big problem. Privately I have been slowing working on coming up with something that deals with this in a more precise way. (But that comes at a price - needing to understand even more concepts like dominator regions).

[Berbe]

IIUC, what you're saying is this bit of code:

            if b:
                a = b
            else:
                continue

shall have been recognised as ifelsestmtl or ifelsestmtc, but certainly not as ifelsestmt?

It seems the "offending" match comes from:

python-uncompyle6/uncompyle6/parsers/reducecheck/ifelsestmt.py

Lines 65 to 74 in 2264ccb

	(
	"ifelsestmt",
	(
	"testexpr",
	"c_stmts_opt",
	"JUMP_FORWARD",
	"else_suite",
	"come_froms",
	),
	),

---

The thing that strikes me most is that the structure of instructions as disassembled makes little-to-no sense.
This part is all that is required to implement the specified logic:

 L.   4        19  LOAD_FAST             2  'b'
               22  POP_JUMP_IF_FALSE    13  'to 13'

 L.   5        25  LOAD_FAST             2  'b'
               28  STORE_FAST            1  'a'

This part, however, looks like the last part of a traditional if-then-else structure, but serves no purpose:

               31  JUMP_FORWARD          3  'to 37'

 L.   7        34  CONTINUE             13  'to 13'
             37_0  COME_FROM            31  '31'

What do you think about returning False from ifelsestmt checks when a CONTINUE instruction matches the target of a POP_JUMP_IF_FALSE in such a sequence?

[rocky]

IIUC, what you're saying is this bit of code:

            if b:
                a = b
            else:
                continue

shall have been recognised as ifelsestmtl or ifelsestmtc, but certainly not as ifelsestmt?

Yes, that is correct.

pyle6/blob/2264ccb1d571ab75660271e196049d63aad8791f/uncompyle6/parsers/reducecheck/ifelsestmt.py#L65-L74

The thing that strikes me most is that the structure of instructions as disassembled makes little-to-no sense. This part is all that is required to implement the specified logic:

 L.   4        19  LOAD_FAST             2  'b'
               22  POP_JUMP_IF_FALSE    13  'to 13'

 L.   5        25  LOAD_FAST             2  'b'
               28  STORE_FAST            1  'a'

This part, however, looks like the last part of a traditional if-then-else structure, but serves no purpose:

               31  JUMP_FORWARD          3  'to 37'

 L.   7        34  CONTINUE             13  'to 13'
             37_0  COME_FROM            31  '31'

The folks who wrote the code generation in Python 2.7 aren't interested in your opinion of how the code would best be written. Nor is that going to change things either. From the decompilation side the only thing that matters is whether the code as it is produced matches the semantics of particular Python code.

In certain cases, especially among older code generation, there was erroneous token classification in the decompiler. For example if the token name were "BREAK out of loop" that would be wrong since the jumps is to the beginning of a loop. Fortunately that is not the situation above.

What do you think about returning False from ifelsestmt checks when a CONTINUE instruction matches the target of a POP_JUMP_IF_FALSE in such a sequence?

My experience is that you need to be as precise as possible. Vagueness just fixes one problem but breaks another. "continue" is valid only inside ifelsestmt{c,l} constructs.

[Berbe]

The folks who wrote the code generation in Python 2.7 aren't interested in your opinion of how the code would best be written. Nor is that going to change things either.

How surprising. 😏

My point was to merely seperate was being used, and what was useless, as in "has no use"/"cannot be used"/"is never used". The 3 instructions fron the second block I highlighted is dead, unreachable code.
I have no knowledge of the inner workings of the compiler having been used, but my thought was: Wasn't there a problem with the compiler in the first place?

What do you think about returning False from ifelsestmt checks when a CONTINUE instruction matches the target of a POP_JUMP_IF_FALSE in such a sequence?

My experience is that you need to be as precise as possible. Vagueness just fixes one problem but breaks another. "continue" is valid only inside ifelsestmt{c,l} constructs.

Without any doubt, but the question rather was: Is it the correct location & way of addressing the problem at hand?

[rocky]

The folks who wrote the code generation in Python 2.7 aren't interested in your opinion of how the code would best be written. Nor is that going to change things either.

How surprising. smirk

My point was to merely seperate was being used, and what was useless, as in "has no use"/"cannot be used"/"is never used". The 3 instructions fron the second block I highlighted is dead, unreachable code. I have no knowledge of the inner workings of the compiler having been used, but my thought was: Wasn't there a problem with the compiler in the first place?

If you think of it from the perspective of how Python bytecode compiler works, things make more sense. There may be some initial instructions produced from a template and then later peephole optimization passes can modify these. Remnants of the original code might still be around, like dead pieces of DNA we have in our genes. Just as in genetics, in some cases a pattern-matching decompiler can use these useless DNA fragments to decide which of two equivalent Python expressions was originally used. But sometimes, as here, it is just dead code.

My understanding of code generation looking at it purely from the bytecode end, is that removing dead Python bytecode hasn't been much of a concern. There is be fair bit of dead code in persisting over many Python releases.

What do you think about returning False from ifelsestmt checks when a CONTINUE instruction matches the target of a POP_JUMP_IF_FALSE in such a sequence?

My experience is that you need to be as precise as possible. Vagueness just fixes one problem but breaks another. "continue" is valid only inside ifelsestmt{c,l} constructs.

Without any doubt, but the question rather was: Is it the correct location & way of addressing the problem at hand?

Parse rules tend to be overly broad and they have to be because currently in the tokens there is very limited information. This is true especially when it comes to control flow. In particular, target offsets are not part of tokens, so they not matched on. Furthermore it would be inconceivable to do this.

However, there is a little bit of this kind of classification by indicating that there is a loop jump (which is always a backward jump in the current way that Python produces bytecode), versus a forward jump. This is nowhere near enough information though.

So the reduction tests are charged with determining whether the control-flow rule is plausible based on where the jumps go to.

In human language what we want to test say is a backward or loop jump from the end of an "else" clause before the beginning of the "if" test is okay only if we are trying to match or reduce an "if" statement of the kind that can occur only inside a loop construct.

That part is necessary. Will that be sufficient? I don't know.

If all of this sounds a little vague, well, yes, that is the nature of this problem, short of understanding Python's code generator in detail and all of the implications it has. (And to be honest, I suspect the programmers who wrote the code generation did this with a bit of trial and error as well).

[Berbe]

Well, here is an attempt: #413
I made it as precise as possible, maybe not reusing all the variables helping understanding, but at least I dug as far as possible.