Lexical subs used for AUTOLOAD

[p5pRT]

Migrated from rt.perl.org#120441 (status was 'resolved')

Searchable as RT120441$

[p5pRT]

From @cpansprout

perlsub says​:

  The fully qualified name
  of the original subroutine magically appears in the global $AUTOLOAD
  variable of the same package as the C<AUTOLOAD> routine.

That means the $AUTOLOAD used is the one in the same package where
the sub is defined, not necessarily the same package as the *AUTOLOAD
glob where it was found.

Since lexical subs are not defined in any package, what should lexical
AUTOLOAD subs do?

This is what they actually do​:

$ ./perl -Ilib -XMfeature=​:all -e 'my sub foo {} *AUTOLOAD = \&foo; bar()'
Segmentation fault​: 11

My initial thought was to use the package of the *AUTOLOAD glob. But
what if the same originally-lexical sub is used for multiple packages?
Where is it supposed to look?

Lexical subs just happen to record which package they were declared
in, more as an implementation detail than intentionally. In fact, I
was hoping at one point to get rid of that. However, it seems that
that might be what we need to use here. That would allow​:

my sub AUTOLOAD_to_be {
  my $subname = our $AUTOLOAD;
  ...
}
*{$some_package."​::AUTOLOAD"} = \&AUTOLOAD_to_be;

--

Father Chrysostomos

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 9​:34 AM, Father Chrysostomos
<perlbug-followup@​perl.org> wrote​:

# New Ticket Created by Father Chrysostomos
# Please include the string​: [perl #120441]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120441 >

perlsub says​:

The fully qualified name
of the original subroutine magically appears in the global $AUTOLOAD
variable of the same package as the C\<AUTOLOAD> routine\.

That means the $AUTOLOAD used is the one in the same package where
the sub is defined, not necessarily the same package as the *AUTOLOAD
glob where it was found.

Since lexical subs are not defined in any package, what should lexical
AUTOLOAD subs do?

This is what they actually do​:

$ ./perl -Ilib -XMfeature=​:all -e 'my sub foo {} *AUTOLOAD = \&foo; bar()'
Segmentation fault​: 11

My initial thought was to use the package of the *AUTOLOAD glob. But
what if the same originally-lexical sub is used for multiple packages?
Where is it supposed to look?

Lexical subs just happen to record which package they were declared
in, more as an implementation detail than intentionally. In fact, I
was hoping at one point to get rid of that. However, it seems that
that might be what we need to use here. That would allow​:

my sub AUTOLOAD_to_be {
my $subname = our $AUTOLOAD;
...
}
*{$some_package."​::AUTOLOAD"} = \&AUTOLOAD_to_be;

--

Father Chrysostomos

Compare to regular subs​:

perl -e '
use Data​::Dumper;
package A;
sub foo { print Data​::Dumper​::Dumper($AUTOLOAD);}
package B;
*AUTOLOAD = \&A​::foo;
package C;
*AUTOLOAD = \&A​::foo;
package main;
bless my $a = {}, "A";
bless my $b = {}, "B";
bless my $c = {}, "C";
$b->Test; $c->Test;'

#### Output​:

$VAR1 = 'B​::Test';
$VAR1 = 'C​::Test';

So, I think it makes the most sense to use the package of the AUTOLOAD
glob that was actually called, regardless of the underlying sub

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 9​:49 AM, Peter Martini <petercmartini@​gmail.com> wrote​:

On Sat, Nov 2, 2013 at 9​:34 AM, Father Chrysostomos
<perlbug-followup@​perl.org> wrote​:

# New Ticket Created by Father Chrysostomos
# Please include the string​: [perl #120441]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120441 >

perlsub says​:

The fully qualified name
of the original subroutine magically appears in the global $AUTOLOAD
variable of the same package as the C\<AUTOLOAD> routine\.

That means the $AUTOLOAD used is the one in the same package where
the sub is defined, not necessarily the same package as the *AUTOLOAD
glob where it was found.

Since lexical subs are not defined in any package, what should lexical
AUTOLOAD subs do?

This is what they actually do​:

$ ./perl -Ilib -XMfeature=​:all -e 'my sub foo {} *AUTOLOAD = \&foo; bar()'
Segmentation fault​: 11

My initial thought was to use the package of the *AUTOLOAD glob. But
what if the same originally-lexical sub is used for multiple packages?
Where is it supposed to look?

Lexical subs just happen to record which package they were declared
in, more as an implementation detail than intentionally. In fact, I
was hoping at one point to get rid of that. However, it seems that
that might be what we need to use here. That would allow​:

my sub AUTOLOAD_to_be {
my $subname = our $AUTOLOAD;
...
}
*{$some_package."​::AUTOLOAD"} = \&AUTOLOAD_to_be;

--

Father Chrysostomos

Compare to regular subs​:

perl -e '
use Data​::Dumper;
package A;
sub foo { print Data​::Dumper​::Dumper($AUTOLOAD);}
package B;
*AUTOLOAD = \&A​::foo;
package C;
*AUTOLOAD = \&A​::foo;
package main;
bless my $a = {}, "A";
bless my $b = {}, "B";
bless my $c = {}, "C";
$b->Test; $c->Test;'

#### Output​:

$VAR1 = 'B​::Test';
$VAR1 = 'C​::Test';

So, I think it makes the most sense to use the package of the AUTOLOAD
glob that was actually called, regardless of the underlying sub

Can we just dummy up the lexical subs into the __ANON__ package like
anonymous subs?

[p5pRT]

From @dmcbride

On Saturday November 2 2013 9​:49​:29 AM Peter Martini wrote​:

On Sat, Nov 2, 2013 at 9​:34 AM, Father Chrysostomos

<perlbug-followup@​perl.org> wrote​:

# New Ticket Created by Father Chrysostomos
# Please include the string​: [perl #120441]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120441 >

perlsub says​:
The fully qualified name
of the original subroutine magically appears in the global $AUTOLOAD
variable of the same package as the C<AUTOLOAD> routine.

That means the $AUTOLOAD used is the one in the same package where
the sub is defined, not necessarily the same package as the *AUTOLOAD
glob where it was found.

Since lexical subs are not defined in any package, what should lexical
AUTOLOAD subs do?

This is what they actually do​:

$ ./perl -Ilib -XMfeature=​:all -e 'my sub foo {} *AUTOLOAD = \&foo; bar()'
Segmentation fault​: 11

My initial thought was to use the package of the *AUTOLOAD glob. But
what if the same originally-lexical sub is used for multiple packages?
Where is it supposed to look?

Lexical subs just happen to record which package they were declared
in, more as an implementation detail than intentionally. In fact, I
was hoping at one point to get rid of that. However, it seems that
that might be what we need to use here. That would allow​:

my sub AUTOLOAD_to_be {

my $subname = our $AUTOLOAD;
\.\.\.

}
*{$some_package."​::AUTOLOAD"} = \&AUTOLOAD_to_be;

--

Father Chrysostomos

Compare to regular subs​:

perl -e '
use Data​::Dumper;
package A;
sub foo { print Data​::Dumper​::Dumper($AUTOLOAD);}
package B;
*AUTOLOAD = \&A​::foo;
package C;
*AUTOLOAD = \&A​::foo;
package main;
bless my $a = {}, "A";
bless my $b = {}, "B";
bless my $c = {}, "C";
$b->Test; $c->Test;'

#### Output​:

$VAR1 = 'B​::Test';
$VAR1 = 'C​::Test';

So, I think it makes the most sense to use the package of the AUTOLOAD
glob that was actually called, regardless of the underlying sub

Isn't that showing the opposite? The $AUTOLOAD variable being used is
$A​::AUTOLOAD. I think FC is on to something - it would need to be the package
that the actual sub is declared in, which seems to be the same as what we have
now for "regular" AUTOLOAD subs.

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 9​:56 AM, Darin McBride <dmcbride@​cpan.org> wrote​:

On Saturday November 2 2013 9​:49​:29 AM Peter Martini wrote​:

On Sat, Nov 2, 2013 at 9​:34 AM, Father Chrysostomos

<perlbug-followup@​perl.org> wrote​:

# New Ticket Created by Father Chrysostomos
# Please include the string​: [perl #120441]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120441 >

perlsub says​:
The fully qualified name
of the original subroutine magically appears in the global $AUTOLOAD
variable of the same package as the C<AUTOLOAD> routine.

That means the $AUTOLOAD used is the one in the same package where
the sub is defined, not necessarily the same package as the *AUTOLOAD
glob where it was found.

Since lexical subs are not defined in any package, what should lexical
AUTOLOAD subs do?

This is what they actually do​:

$ ./perl -Ilib -XMfeature=​:all -e 'my sub foo {} *AUTOLOAD = \&foo; bar()'
Segmentation fault​: 11

My initial thought was to use the package of the *AUTOLOAD glob. But
what if the same originally-lexical sub is used for multiple packages?
Where is it supposed to look?

Lexical subs just happen to record which package they were declared
in, more as an implementation detail than intentionally. In fact, I
was hoping at one point to get rid of that. However, it seems that
that might be what we need to use here. That would allow​:

my sub AUTOLOAD_to_be {

my $subname = our $AUTOLOAD;
\.\.\.

}
*{$some_package."​::AUTOLOAD"} = \&AUTOLOAD_to_be;

--

Father Chrysostomos

Compare to regular subs​:

perl -e '
use Data​::Dumper;
package A;
sub foo { print Data​::Dumper​::Dumper($AUTOLOAD);}
package B;
*AUTOLOAD = \&A​::foo;
package C;
*AUTOLOAD = \&A​::foo;
package main;
bless my $a = {}, "A";
bless my $b = {}, "B";
bless my $c = {}, "C";
$b->Test; $c->Test;'

#### Output​:

$VAR1 = 'B​::Test';
$VAR1 = 'C​::Test';

So, I think it makes the most sense to use the package of the AUTOLOAD
glob that was actually called, regardless of the underlying sub

Isn't that showing the opposite? The $AUTOLOAD variable being used is
$A​::AUTOLOAD. I think FC is on to something - it would need to be the package
that the actual sub is declared in, which seems to be the same as what we have
now for "regular" AUTOLOAD subs.

Doh, too early for me, you're right

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 9​:58 AM, Peter Martini <petercmartini@​gmail.com> wrote​:

On Sat, Nov 2, 2013 at 9​:56 AM, Darin McBride <dmcbride@​cpan.org> wrote​:

Isn't that showing the opposite? The $AUTOLOAD variable being used is
$A​::AUTOLOAD. I think FC is on to something - it would need to be the package
that the actual sub is declared in, which seems to be the same as what we have
now for "regular" AUTOLOAD subs.

Doh, too early for me, you're right

The more I think about it, the more I like the idea of just using the
__ANON__ stash. A variety of other places use the __ANON__ namespace
when there's no GV available, so there's precedent for it. A
PL_anonstash with HvNAME of __ANON__, especially if its not found by a
lookup for __ANON__, could make a variety of things easier, right?
That's another way to fix the call checker situation, too, since it
just means anyone using the call checker API can check if CvGV ==
PL_anonstash and omit the name.

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 10​:36 AM, Peter Martini <petercmartini@​gmail.com> wrote​:

On Sat, Nov 2, 2013 at 9​:58 AM, Peter Martini <petercmartini@​gmail.com> wrote​:

On Sat, Nov 2, 2013 at 9​:56 AM, Darin McBride <dmcbride@​cpan.org> wrote​:

Isn't that showing the opposite? The $AUTOLOAD variable being used is
$A​::AUTOLOAD. I think FC is on to something - it would need to be the package
that the actual sub is declared in, which seems to be the same as what we have
now for "regular" AUTOLOAD subs.

Doh, too early for me, you're right

The more I think about it, the more I like the idea of just using the
__ANON__ stash. A variety of other places use the __ANON__ namespace
when there's no GV available, so there's precedent for it. A
PL_anonstash with HvNAME of __ANON__, especially if its not found by a
lookup for __ANON__, could make a variety of things easier, right?
That's another way to fix the call checker situation, too, since it
just means anyone using the call checker API can check if CvGV ==
PL_anonstash and omit the name.

Bah, I'm sure I mangled the details, I hope I got the gist across.

[p5pRT]

From perl5-porters@perl.org

Peter Martini wrote​:

The more I think about it, the more I like the idea of just using the
__ANON__ stash. A variety of other places use the __ANON__ namespace
when there's no GV available, so there's precedent for it. A
PL_anonstash with HvNAME of __ANON__, especially if its not found by a
lookup for __ANON__, could make a variety of things easier, right?
That's another way to fix the call checker situation, too, since it
just means anyone using the call checker API can check if CvGV ==
PL_anonstash and omit the name.

That doesn’t sound too bad. But note that the package for anonymous
subs is not normally __ANON__, but only the GV​:

SV = IV(0x7ffa898060d8) at 0x7ffa898060e8
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x7ffa8982ef08
  SV = PVCV(0x7ffa8982d900) at 0x7ffa8982ef08
  REFCNT = 2
  FLAGS = (PADMY,ANON,WEAKOUTSIDE,CVGV_RC,DYNFILE)
  COMP_STASH = 0x7ffa89805f68"main"
  START = 0x7ffa8941fbf0 ===> 1
  ROOT = 0x7ffa8941fbb0
  GVGV​::GV = 0x7ffa8982ef50"main" :​: "__ANON__"
  FILE = "-e"
  DEPTH = 0
  FLAGS = 0x1490
  OUTSIDE_SEQ = 45
  PADLIST = 0x7ffa89406378
  PADNAME = 0x7ffa8982ef98(0x7ffa894063e8) PAD = 0x7ffa8982eef0(0x7ffa8940eac8)
  OUTSIDE = 0x7ffa898062f8 (MAIN)

Having my sub foo be __ANON__​::foo would make things easier, though,
and if people use lex subs for AUTOLOAD we could just tell them not
to do that.

(Note that anonymous subs are in a package, so $AUTOLOAD is set on the
package in which they are declared, if they are used for AUTOLOAD.)

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 10​:43 AM, Father Chrysostomos <sprout@​cpan.org> wrote​:

Peter Martini wrote​:

The more I think about it, the more I like the idea of just using the
__ANON__ stash. A variety of other places use the __ANON__ namespace
when there's no GV available, so there's precedent for it. A
PL_anonstash with HvNAME of __ANON__, especially if its not found by a
lookup for __ANON__, could make a variety of things easier, right?
That's another way to fix the call checker situation, too, since it
just means anyone using the call checker API can check if CvGV ==
PL_anonstash and omit the name.

That doesn’t sound too bad. But note that the package for anonymous
subs is not normally __ANON__, but only the GV​:

SV = IV(0x7ffa898060d8) at 0x7ffa898060e8
REFCNT = 1
FLAGS = (TEMP,ROK)
RV = 0x7ffa8982ef08
SV = PVCV(0x7ffa8982d900) at 0x7ffa8982ef08
REFCNT = 2
FLAGS = (PADMY,ANON,WEAKOUTSIDE,CVGV_RC,DYNFILE)
COMP_STASH = 0x7ffa89805f68"main"
START = 0x7ffa8941fbf0 ===> 1
ROOT = 0x7ffa8941fbb0
GVGV​::GV = 0x7ffa8982ef50"main" :​: "__ANON__"
FILE = "-e"
DEPTH = 0
FLAGS = 0x1490
OUTSIDE_SEQ = 45
PADLIST = 0x7ffa89406378
PADNAME = 0x7ffa8982ef98(0x7ffa894063e8) PAD = 0x7ffa8982eef0(0x7ffa8940eac8)
OUTSIDE = 0x7ffa898062f8 (MAIN)

Having my sub foo be __ANON__​::foo would make things easier, though,
and if people use lex subs for AUTOLOAD we could just tell them not
to do that.

(Note that anonymous subs are in a package, so $AUTOLOAD is set on the
package in which they are declared, if they are used for AUTOLOAD.)

Yeah, I know, it just always felt like an implementation detail to me.
I'm struggling to see, from a big picture point of views, whether
there's a reason why an anonymous sub must belong to a package, and if
there's any reason why a lexical sub should be treated differently.
The package provides global scope, but it does that the same way for
both. And neither case registers a name from the global level, so
shouldn't they treat their stashes the same way?

[p5pRT]

From @rurban

Anonymous subs must only belong to a class, but not necessarily to a package.
The __ANON__ stash seems best to me, if we cannot get rid of the CvGV at all.
As analogy​: our sub needs a package, my sub not. my sub just needs a targ slot.

The AUTOLOAD magic needs to ignore lexical subs as lexical subs should
not support name lookup. That they do is an unfortunate implementation quirks.

On Sat, Nov 2, 2013 at 10​:12 AM, Peter Martini <petercmartini@​gmail.com> wrote​:

On Sat, Nov 2, 2013 at 10​:43 AM, Father Chrysostomos <sprout@​cpan.org> wrote​:

Peter Martini wrote​:

The more I think about it, the more I like the idea of just using the
__ANON__ stash. A variety of other places use the __ANON__ namespace
when there's no GV available, so there's precedent for it. A
PL_anonstash with HvNAME of __ANON__, especially if its not found by a
lookup for __ANON__, could make a variety of things easier, right?
That's another way to fix the call checker situation, too, since it
just means anyone using the call checker API can check if CvGV ==
PL_anonstash and omit the name.

That doesn’t sound too bad. But note that the package for anonymous
subs is not normally __ANON__, but only the GV​:

SV = IV(0x7ffa898060d8) at 0x7ffa898060e8
REFCNT = 1
FLAGS = (TEMP,ROK)
RV = 0x7ffa8982ef08
SV = PVCV(0x7ffa8982d900) at 0x7ffa8982ef08
REFCNT = 2
FLAGS = (PADMY,ANON,WEAKOUTSIDE,CVGV_RC,DYNFILE)
COMP_STASH = 0x7ffa89805f68"main"
START = 0x7ffa8941fbf0 ===> 1
ROOT = 0x7ffa8941fbb0
GVGV​::GV = 0x7ffa8982ef50"main" :​: "__ANON__"
FILE = "-e"
DEPTH = 0
FLAGS = 0x1490
OUTSIDE_SEQ = 45
PADLIST = 0x7ffa89406378
PADNAME = 0x7ffa8982ef98(0x7ffa894063e8) PAD = 0x7ffa8982eef0(0x7ffa8940eac8)
OUTSIDE = 0x7ffa898062f8 (MAIN)

Having my sub foo be __ANON__​::foo would make things easier, though,
and if people use lex subs for AUTOLOAD we could just tell them not
to do that.

(Note that anonymous subs are in a package, so $AUTOLOAD is set on the
package in which they are declared, if they are used for AUTOLOAD.)

Yeah, I know, it just always felt like an implementation detail to me.
I'm struggling to see, from a big picture point of views, whether
there's a reason why an anonymous sub must belong to a package, and if
there's any reason why a lexical sub should be treated differently.
The package provides global scope, but it does that the same way for
both. And neither case registers a name from the global level, so
shouldn't they treat their stashes the same way?

--
Reini Urban
http​://cpanel.net/ http​://www.perl-compiler.org/

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 11​:25 AM, Reini Urban <rurban@​x-ray.at> wrote​:

Anonymous subs must only belong to a class, but not necessarily to a package.
The __ANON__ stash seems best to me, if we cannot get rid of the CvGV at all.
As analogy​: our sub needs a package, my sub not. my sub just needs a targ slot.

That's all implementation details though, and specifically
implementation details of where we store the name of the sub. The
name of the sub is a reference to the body, but it's by no means the
only reference. So my point is, is the code 'my sub foo ...' supposed
to declare a lexically scoped name pointing to a standard body, or
does declaring 'my sub foo' alter the body of the sub in some way as
well - and if so, how?

The AUTOLOAD magic needs to ignore lexical subs as lexical subs should
not support name lookup. That they do is an unfortunate implementation quirks.

The question becomes, why should lexical subs not support name lookup,
but *anonymous* subs do?

--
Reini Urban
http​://cpanel.net/ http​://www.perl-compiler.org/

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 10​:43 AM, Father Chrysostomos <sprout@​cpan.org> wrote​:

Peter Martini wrote​:

The more I think about it, the more I like the idea of just using the
__ANON__ stash. A variety of other places use the __ANON__ namespace
when there's no GV available, so there's precedent for it. A
PL_anonstash with HvNAME of __ANON__, especially if its not found by a
lookup for __ANON__, could make a variety of things easier, right?
That's another way to fix the call checker situation, too, since it
just means anyone using the call checker API can check if CvGV ==
PL_anonstash and omit the name.

That doesn’t sound too bad. But note that the package for anonymous
subs is not normally __ANON__, but only the GV​:

SV = IV(0x7ffa898060d8) at 0x7ffa898060e8
REFCNT = 1
FLAGS = (TEMP,ROK)
RV = 0x7ffa8982ef08
SV = PVCV(0x7ffa8982d900) at 0x7ffa8982ef08
REFCNT = 2
FLAGS = (PADMY,ANON,WEAKOUTSIDE,CVGV_RC,DYNFILE)
COMP_STASH = 0x7ffa89805f68"main"
START = 0x7ffa8941fbf0 ===> 1
ROOT = 0x7ffa8941fbb0
GVGV​::GV = 0x7ffa8982ef50"main" :​: "__ANON__"
FILE = "-e"
DEPTH = 0
FLAGS = 0x1490
OUTSIDE_SEQ = 45
PADLIST = 0x7ffa89406378
PADNAME = 0x7ffa8982ef98(0x7ffa894063e8) PAD = 0x7ffa8982eef0(0x7ffa8940eac8)
OUTSIDE = 0x7ffa898062f8 (MAIN)

Having my sub foo be __ANON__​::foo would make things easier, though,
and if people use lex subs for AUTOLOAD we could just tell them not
to do that.

(Note that anonymous subs are in a package, so $AUTOLOAD is set on the
package in which they are declared, if they are used for AUTOLOAD.)

Actually - would it be possible to create a GV referencing the stash,
but not have the stash point back to the GV? Then we could set a
CvFLAGS bit to indicate that the CV was declared lexically, and have
the various name dumpers use that to mean don't print the package in
the various error routines. If there's no reference from the stash
back to the GV, it doesn't change any of the name resolution rules
that we have for lexical subs now.

We have spare cv_flags bits :-)

[p5pRT]

From perl5-porters@perl.org

Peter Martini wrote​:

I'm struggling to see, from a big picture point of views, whether
there's a reason why an anonymous sub must belong to a package

*AUTOLOAD = sub { ... $AUTOLOAD ... };

[p5pRT]

From PeterCMartini@GMail.com

Peter Martini wrote​:

I'm struggling to see, from a big picture point of views, whether
there's a reason why an anonymous sub must belong to a package

*AUTOLOAD = sub { ... $AUTOLOAD ... };

I know :-)

Whats the difference between that and lexical subs, philosophically? They're both accessing variables in an outer, package scope at the point of definition, is there a reason a lexical sub shouldnt just keep a reference to the package where it was declared as well?

[p5pRT]

From perl5-porters@perl.org

Peter Martini wrote​:

Actually - would it be possible to create a GV referencing the stash,
but not have the stash point back to the GV? Then we could set a
CvFLAGS bit to indicate that the CV was declared lexically, and have
the various name dumpers use that to mean don't print the package in
the various error routines. If there's no reference from the stash
back to the GV, it doesn't change any of the name resolution rules
that we have for lexical subs now.

That would bloat the memory usage of lexical subs, making them match
package subs. In fact, I have been trying to see a way to eliminate
typeglobs for most subs, to save memory there ($​::{foo} = sub{...}
effectively). I would rather not go in the opposite direction. :-)

Lexical subs already have a CvSTASH pointer BTW. In fact, maybe we
could use that for package subs, too....

[p5pRT]

From perl5-porters@perl.org

Peter Martini asked​:

Whats the difference between that and lexical subs, philosophically?
They're both accessing variables in an outer, package scope at the point
of definition, is there a reason a lexical sub shouldnt just keep a
reference to the package where it was declared as well?

They do. :-) Via CvSTASH.

Is your question in reponse to my statement, ‘In fact, I was hoping at
one point to get rid of that’?

In that case, I see your point.

The only difference between lexical subs and anonymous subs (apart
from the fact that scope entry will magically upgrade a stub into a
fully-fledged sub) is that lexical subs need names for the sake of
error reporting.

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 12​:58 PM, Father Chrysostomos <sprout@​cpan.org> wrote​:

Peter Martini wrote​:

I'm struggling to see, from a big picture point of views, whether
there's a reason why an anonymous sub must belong to a package

*AUTOLOAD = sub { ... $AUTOLOAD ... };

perl -XMfeature=​:all -E '
  package B;
  my sub foo { say $Q;} *bar = \&foo;
  package C;
  *baz = \&B​::bar;
  package main;
  $B​::Q = 4;
  $C​::Q = 5;
  C​::baz();'
4

Am I still missing something obvious? It looks like lexical subs are
already keeping a reference to the package where they were defined,
and if $AUTOLOAD doesn't work its just implemenation details leaking
through?

[p5pRT]

From PeterCMartini@GMail.com

On Sat, Nov 2, 2013 at 1​:31 PM, Father Chrysostomos <sprout@​cpan.org> wrote​:

Peter Martini asked​:

Whats the difference between that and lexical subs, philosophically?
They're both accessing variables in an outer, package scope at the point
of definition, is there a reason a lexical sub shouldnt just keep a
reference to the package where it was declared as well?

They do. :-) Via CvSTASH.

Is your question in reponse to my statement, ‘In fact, I was hoping at
one point to get rid of that’?

In that case, I see your point.

The only difference between lexical subs and anonymous subs (apart
from the fact that scope entry will magically upgrade a stub into a
fully-fledged sub) is that lexical subs need names for the sake of
error reporting.

Ah, okay :-) So ignore my last email, this one answers it - I think.
So the lack of a GV is an implementation detail for memory and
performance, and in practical terms, that means that for AUTOLOAD,
like the call checker, it just needs a workaround to get to the GV we
could have stored but chose not to?

[p5pRT]

From perl5-porters@perl.org

Peter Martini wrote​:

On Sat, Nov 2, 2013 at 12​:58 PM, Father Chrysostomos <sprout@​cpan.org> wrote​:

Peter Martini wrote​:

I'm struggling to see, from a big picture point of views, whether
there's a reason why an anonymous sub must belong to a package

*AUTOLOAD = sub { ... $AUTOLOAD ... };

perl -XMfeature=​:all -E '
package B;
my sub foo { say $Q;} *bar = \&foo;
package C;
*baz = \&B​::bar;
package main;
$B​::Q = 4;
$C​::Q = 5;
C​::baz();'
4

Am I still missing something obvious? It looks like lexical subs are
already keeping a reference to the package where they were defined,

That is because every statement is compiled into some package.
The body of the sub, in this case, is in the B package, so $Q
resolves to $B​::Q.

That is unrelated to the $AUTOLOAD problem, as ‘package foo; sub bar​::AUTOLOAD { $AUTOLOAD }’ will not work, since the body of the sub is not in the same package the sub is defined in.

and if $AUTOLOAD doesn't work its just implemenation details leaking
through?

But the whole problem (the reason for this thread) is that perl needs
to know *which* $AUTOLOAD to set. It’s doing it based on the GV the
CV points to. I am still wondering why we can’t just use CvSTASH and
why we have to go through GvSTASH(CvGV(cv)) to begin with (apart from
the XS autoload aberration, which is easy enough to work around).

What is CvSTASH for, apart from?

As you may have noticed, I haven’t really thought anything through in
this thread. Most of my responses are brain dumps with loose ends.

[p5pRT]

From perl5-porters@perl.org

Peter Martini wrote​:

Ah, okay :-) So ignore my last email, this one answers it - I think.
So the lack of a GV is an implementation detail for memory and
performance, and in practical terms, that means that for AUTOLOAD,
like the call checker, it just needs a workaround to get to the GV we
could have stored but chose not to?

I suppose so. As I said in my previous message, I have not really
thought this all through.

Do we want CvGV to vivify and cache a GV? Then we can use the lexical
flag that you suggested to suppress the name.

To avoid vivifying GVs unnecessarily, we can add the cv_name function
previously discussed, possibly with a variation that returns a string,
rather than a CV. Then we need to use that function zealously and
strip out most uses of CvGV from the perl core.

(We have no convention for indicating the return type in a function
name. cv_name_pvn implies that it takes a string/length pair. What
would we call it?)

BTW, here is another victim of this problem​:

$ ./perl -Ilib -XMfeature=​:all -e 'state sub foo {} use overload "%{}"=>\&foo; %{bless+[]}'
Segmentation fault​: 11

Again, it’s a null CvGV.

[p5pRT]

From PeterCMartini@GMail.com

Sent from my iPhone

On Nov 2, 2013, at 14​:18, Father Chrysostomos <sprout@​cpan.org> wrote​:

Peter Martini wrote​:

On Sat, Nov 2, 2013 at 12​:58 PM, Father Chrysostomos <sprout@​cpan.org> wrote​:

Peter Martini wrote​:

I'm struggling to see, from a big picture point of views, whether
there's a reason why an anonymous sub must belong to a package

*AUTOLOAD = sub { ... $AUTOLOAD ... };

perl -XMfeature=​:all -E '
package B;
my sub foo { say $Q;} *bar = \&foo;
package C;
*baz = \&B​::bar;
package main;
$B​::Q = 4;
$C​::Q = 5;
C​::baz();'
4

Am I still missing something obvious? It looks like lexical subs are
already keeping a reference to the package where they were defined,

That is because every statement is compiled into some package.
The body of the sub, in this case, is in the B package, so $Q
resolves to $B​::Q.

That is unrelated to the $AUTOLOAD problem, as ‘package foo; sub bar​::AUTOLOAD { $AUTOLOAD }’ will not work, since the body of the sub is not in the same package the sub is defined in.

and if $AUTOLOAD doesn't work its just implemenation details leaking
through?

But the whole problem (the reason for this thread) is that perl needs
to know *which* $AUTOLOAD to set. It’s doing it based on the GV the
CV points to. I am still wondering why we can’t just use CvSTASH and
why we have to go through GvSTASH(CvGV(cv)) to begin with (apart from
the XS autoload aberration, which is easy enough to work around).

What is CvSTASH for, apart from?

ISA?

As you may have noticed, I haven’t really thought anything through in
this thread. Most of my responses are brain dumps with loose ends.

Ditto :-)

[p5pRT]

From perl5-porters@perl.org

Peter Martini wrote​:

What is CvSTASH for, apart from?

ISA?

I don’t understand your response. It makes about as much sense as my
question. :-)

What I meant to write was​: What is CvSTASH for, apart from providing
a way for XS modules to find out what package a sub was declared in
(i.e., its documented use).

I was about to say it’s not used in care, but I now see it is used to
see whether a sub belongs to the DB package (CvSTASH == PL_debstash)
and also in toke.c to see whether a sub belongs to the _charnames
package (the latter of more recent provenance).

Then there is also the XS AUTOLOAD hack, which we may well simply have
to work around.

Those are the only uses. Most of the time we have GvSTASH(GvCV(cv)),
which in all legitimate cases returns the same thing as CvSTASH, I
believe (where XS AUTOLOAD routines are not involved).

[p5pRT]

From zefram@fysh.org

Father Chrysostomos wrote​:

I am still wondering why we can't just use CvSTASH and
why we have to go through GvSTASH(CvGV(cv)) to begin with

They can be different, and we have backcompat to think of.

What is CvSTASH for, apart from?

Almost nothing. As far as I can see, the main effect is that subroutines
defined in the DB package are not subject to debugging.

-zefram

[p5pRT]

From zefram@fysh.org

Father Chrysostomos wrote​:

Those are the only uses. Most of the time we have GvSTASH(GvCV(cv)),
which in all legitimate cases returns the same thing as CvSTASH, I
believe (where XS AUTOLOAD routines are not involved).

How legitimate is "sub OtherPackage​::foo {}"?

-zefram

[p5pRT]

From perl5-porters@perl.org

Zefram wrote​:

Father Chrysostomos wrote​:

Those are the only uses. Most of the time we have GvSTASH(GvCV(cv)),
which in all legitimate cases returns the same thing as CvSTASH, I
believe (where XS AUTOLOAD routines are not involved).

How legitimate is "sub OtherPackage​::foo {}"?

That shows I misunderstood the code that sets CvSTASH, as does
Devel​::Peek​:

$ ./perl -Ilib -e 'use Devel​::Peek; sub foo​::bar {} Dump \&foo​::Bar'
SV = IV(0x7f7f940062a0) at 0x7f7f940062b0
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x7f7f940060e8
  SV = PVCV(0x7f7f9404ba30) at 0x7f7f940060e8
  REFCNT = 2
  FLAGS = (DYNFILE)
  COMP_STASH = 0x7f7f94005f68"main"
  ROOT = 0x0
  GVGV​::GV = 0x7f7f9402eed8"foo" :​: "Bar"
  FILE = "-e"
  DEPTH = 0
  FLAGS = 0x1000
  OUTSIDE_SEQ = 0
  PADLIST = 0x0
  OUTSIDE = 0x0 (null)

[p5pRT]

From @mauke

On 02.11.2013 22​:29, Father Chrysostomos wrote​:

Zefram wrote​:

Father Chrysostomos wrote​:

Those are the only uses. Most of the time we have GvSTASH(GvCV(cv)),
which in all legitimate cases returns the same thing as CvSTASH, I
believe (where XS AUTOLOAD routines are not involved).

How legitimate is "sub OtherPackage​::foo {}"?

That shows I misunderstood the code that sets CvSTASH, as does
Devel​::Peek​:

$ ./perl -Ilib -e 'use Devel​::Peek; sub foo​::bar {} Dump \&foo​::Bar'

I don't know if it makes any difference but you defined foo​::bar but you
dumped foo​::Bar (b vs. B).

--
Lukas Mai <plokinom@​gmail.com>

[p5pRT]

From perl5-porters@perl.org

Lukas Mai wrote​:

I don't know if it makes any difference but you defined foo​::bar but you
dumped foo​::Bar (b vs. B).

Oops! In this case, the stub was vivified in the main stash, so it
still demonstrated what I was trying to show. :-)

[p5pRT]

From perl5-porters@perl.org

Peter Martini wrote​:

On Nov 2, 2013, at 14​:18, Father Chrysostomos <sprout@​cpan.org> wrote​:

As you may have noticed, I haven=92t really thought anything through in
this thread. Most of my responses are brain dumps with loose ends.

Ditto :-)

Nevertheless, this conversion has given me ideas I would not have come
up with otherwise.

Proposed solution​:

• Lexical subs get an extra flag indicating that the stash name must
  be omitted from the name (CVf_LEXICAL).
• All subs, at least conceptually, have GVs. Lexical subs own
  theirs, and hold a reference count on it, as it does not reside in
  the stash.
• Any sub can have CVf_NAMED set, indicating that xcv_gv_u holds a
  HEK, in which case CvSTASH + CvNAME_HEK combined imply the GV.
• CvGV will vivify a GV from CvSTASH and CvNAME_HEK when accessed.
• Whenever GvSTASH(CvGV(cv)) would differ from CvSTASH, we skip this
  optimisation.
• To avoid vivifying GVs, always check !CvNAMED before using CvGV.
• Or use cv_name, which could have this signature​:
  void Perl_cv_name(pTHX_ CV *cv, SV **svp, char **pvp, STRLEN *lenp)
  where the last three parameters are for returning values.

The first stage would be to switch lexical subs over to the new system.

Then we can optimise other subs by making

  sub foo {}

put a code ref in $​::{foo}.

[p5pRT]

From @cpansprout

On Sat Nov 02 17​:32​:46 2013, perl5-porters@​perl.org wrote​:

Peter Martini wrote​:

On Nov 2, 2013, at 14​:18, Father Chrysostomos <sprout@​cpan.org> wrote​:

As you may have noticed, I haven=92t really thought anything through in
this thread. Most of my responses are brain dumps with loose ends.

Ditto :-)

Nevertheless, this conversion has given me ideas I would not have come
up with otherwise.

Proposed solution​:

• Lexical subs get an extra flag indicating that the stash name must
be omitted from the name (CVf_LEXICAL).
• All subs, at least conceptually, have GVs. Lexical subs own
theirs, and hold a reference count on it, as it does not reside in
the stash.
• Any sub can have CVf_NAMED set, indicating that xcv_gv_u holds a
HEK, in which case CvSTASH + CvNAME_HEK combined imply the GV.
• CvGV will vivify a GV from CvSTASH and CvNAME_HEK when accessed.
• Whenever GvSTASH(CvGV(cv)) would differ from CvSTASH, we skip this
optimisation.
• To avoid vivifying GVs, always check !CvNAMED before using CvGV.
• Or use cv_name, which could have this signature​:
void Perl_cv_name(pTHX_ CV *cv, SV **svp, char **pvp, STRLEN *lenp)
where the last three parameters are for returning values.

The first stage would be to switch lexical subs over to the new system.

Then we can optimise other subs by making

sub foo \{\}

put a code ref in $​::{foo}.

Oh boy! I had completely forgotten about this thread. I have just fixed the original issue with 1869162 (before finding this thread again), but the other things in the quoted message (which would make for cleaner code) have not been addressed.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

On Thu Aug 28 12​:31​:56 2014, sprout wrote​:

Oh boy! I had completely forgotten about this thread. I have just
fixed the original issue with 1869162 (before finding this thread
again), but the other things in the quoted message (which would make
for cleaner code) have not been addressed.

Now they have. See commit f9d9e96.

--

Father Chrysostomos

[p5pRT]

@cpansprout - Status changed from 'open' to 'resolved'