Lexical closures (my sub) have broken scoping under recursion

[mauke]

Description
When lexical subs (my sub) that close over outer lexicals are used in subs that are recursive, weird things happen. It looks like Perl doesn't realize that the inner sub is a closure and has to be allocated at runtime (once per level of recursion). Instead Perl just reuses the same inner sub, overwriting its bindings.

Steps to Reproduce

#!/usr/bin/env perl
use strict;
use warnings;
use feature 'lexical_subs';

sub outer {
    my ($x) = @_;
    my sub inner {
        my ($label) = @_;
        print "inner($label): \$x = $x at ", \$x, "\n";
    }
    print "outer($x): \\&inner = ", \&inner, "\n";
    inner "before";
    outer($x - 1) if $x > 0;
    inner "after";
}

print "Attempt 1:\n";
outer 2;

print "\n", "Attempt 2:\n";
outer 2;
__END__

Output:

Attempt 1:
outer(2): \&inner = CODE(0x55c73e8332b0)
inner(before): $x = 2 at SCALAR(0x55c73e87c180)
outer(1): \&inner = CODE(0x55c73e8332b0)
inner(before): $x = 1 at SCALAR(0x55c73e8121a0)
outer(0): \&inner = CODE(0x55c73e8332b0)
inner(before): $x = 0 at SCALAR(0x55c73e833370)
inner(after): $x = 0 at SCALAR(0x55c73e833370)
inner(after): $x = 0 at SCALAR(0x55c73e833370)
inner(after): $x = 0 at SCALAR(0x55c73e833370)

Attempt 2:
outer(2): \&inner = CODE(0x55c73e8332b0)
inner(before): $x = 2 at SCALAR(0x55c73e87c180)
outer(1): \&inner = CODE(0x55c73e833340)
inner(before): $x = 1 at SCALAR(0x55c73e8121a0)
outer(0): \&inner = CODE(0x55c73e833640)
inner(before): $x = 0 at SCALAR(0x55c73e833388)
inner(after): $x = 0 at SCALAR(0x55c73e833388)
inner(after): $x = 1 at SCALAR(0x55c73e8121a0)
inner(after): $x = 2 at SCALAR(0x55c73e87c180)

Expected behavior
"Attempt 1" is the broken case. Note in particular that all three levels of outer recursion seem to share the same inner sub (\&inner = CODE(0x55c73e8332b0)) and that the inner(after) calls at each level of outer recursion see the same $x variable (with value 0), namely the one from the innermost (outer(0)) level of recursion.

"Attempt 2" is the expected behavior (I don't know why it behaves differently when called a second time). Here every level of outer has a different inner sub, and inner at each level of recursion sees a different $x variable, both before and after the recursive call to outer.

This bug is reproducible in all perl versions I've tried: 5.39.6, 5.38.2, 5.38.0, 5.36.0, 5.34.1, 5.30.3, 5.26.1, 5.20.3.

(Thanks to vms14 from libera.chat #perl for finding this!)

[troglodyne]

Certainly odd behavior. That said, scopes generally ought to inherit variables from the parent scope unless you override that via declaration in the inner scope. If you want $x to behave like you desire it to (as in attempt 2), pass it in to inner and define it in that scope when unpacking @_ and it will always behave like you expect it to.

Given how long this has existed, I suspect patching this will have unintended consequences. This is probably a load bearing bug to some crazy script out there. Maybe better candidate for a perlcritic rule? To be fair, last I checked tail call recursion isn't encouraged in PBP.

[mauke]

For comparison, the equivalent code using anonymous subs and lexical variables works as expected:

#!/usr/bin/env perl
use strict;
use warnings;
#use feature 'lexical_subs';

sub outer {
    my ($x) = @_;
    my $inner = sub {
        my ($label) = @_;
        print "inner($label): \$x = $x at ", \$x, "\n";
    };
    print "outer($x): \$inner = ", $inner, "\n";
    $inner->("before");
    outer($x - 1) if $x > 0;
    $inner->("after");
}

print "Attempt 1:\n";
outer 2;

print "\n", "Attempt 2:\n";
outer 2;
__END__

Output:

Attempt 1:
outer(2): $inner = CODE(0x558051973408)
inner(before): $x = 2 at SCALAR(0x5580519a1ec0)
outer(1): $inner = CODE(0x5580519d9e30)
inner(before): $x = 1 at SCALAR(0x5580519d9e78)
outer(0): $inner = CODE(0x5580519d9b30)
inner(before): $x = 0 at SCALAR(0x5580519d9b60)
inner(after): $x = 0 at SCALAR(0x5580519d9b60)
inner(after): $x = 1 at SCALAR(0x5580519d9e78)
inner(after): $x = 2 at SCALAR(0x5580519a1ec0)

Attempt 2:
outer(2): $inner = CODE(0x5580519a1320)
inner(before): $x = 2 at SCALAR(0x558051973408)
outer(1): $inner = CODE(0x5580519d9cf8)
inner(before): $x = 1 at SCALAR(0x5580519d9e30)
outer(0): $inner = CODE(0x5580519d9a58)
inner(before): $x = 0 at SCALAR(0x5580519d9b30)
inner(after): $x = 0 at SCALAR(0x5580519d9b30)
inner(after): $x = 1 at SCALAR(0x5580519d9e30)
inner(after): $x = 2 at SCALAR(0x558051973408)

[tonycoz]

Does #21580 fix this?

[mauke]

@tonycoz It does!