Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blead Breaks CPAN: YANICK/List-Lazy-0.3.0.tar.gz #16377

Closed
p5pRT opened this issue Jan 24, 2018 · 57 comments
Closed

Blead Breaks CPAN: YANICK/List-Lazy-0.3.0.tar.gz #16377

p5pRT opened this issue Jan 24, 2018 · 57 comments
Labels
BBC distro-All

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Jan 24, 2018

Migrated from rt.perl.org#132760 (status was 'resolved')

Searchable as RT132760$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 24, 2018

From @eserte

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.8.


List-Lazy-0.3.0 does not work anymore since v5.27.7-212-g894f226e51.
This is the change which swapped signatures and attributes.
This change is already documented in perldelta, but I have just
two notes​:

- The error output does not look nice. In this case it begins with

Array found where operator expected at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43, near "$$@​)"
  (Missing operator before @​)?)
"my" variable $step masks earlier declaration in same statement at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 44.
syntax error at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 36, near ") :"
Global symbol "$generator" requires explicit package name (did you forget to declare "my $generator"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 38.
Global symbol "$state" requires explicit package name (did you forget to declare "my $state"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 39.
Global symbol "$min" requires explicit package name (did you forget to declare "my $min"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$max" requires explicit package name (did you forget to declare "my $max"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$step" requires explicit package name (did you forget to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43.
Invalid separator character '{' in attribute list at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 44, near "$step : sub "
Global symbol "$step" requires explicit package name (did you forget to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 44.
...

  which does not really say anything about the problem. Is it possible
  to detect this situation and improve diagnostics?

- What does this change means in terms of usability of signatures? Users
  mixing signatures and prototypes must increase their perl prerequisite
  from 5.22 to 5.28, which may mean it could be less likely that
  signatures are used in the next time. Is this worth for this change?

BTW, the issue for the CPAN module is
yanick/List-Lazy#3

-- Slaven



Flags​:
  category=core
  severity=low


Site configuration information for perl 5.27.8​:

Configured by eserte at Sat Jan 20 09​:22​:10 CET 2018.

Summary of my perl5 (revision 5 version 27 subversion 8) configuration​:
 
  Platform​:
  osname=linux
  osvers=3.16.0-4-amd64
  archname=x86_64-linux
  uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-12-13) x86_64 gnulinux '
  config_args='-ds -e -Dprefix=/opt/perl-5.27.8 -Dusedevel -Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
  optimize='-O2'
  cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.9.2'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.19.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


@​INC for perl 5.27.8​:
  /opt/perl-5.27.8/lib/site_perl/5.27.8/x86_64-linux
  /opt/perl-5.27.8/lib/site_perl/5.27.8
  /opt/perl-5.27.8/lib/5.27.8/x86_64-linux
  /opt/perl-5.27.8/lib/5.27.8


Environment for perl 5.27.8​:
  HOME=/home/eserte
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
  PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
  PERL_BADLANG (unset)
  SHELL=/bin/zsh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 24, 2018

From @jkeenan

On Wed, 24 Jan 2018 20​:04​:24 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.8.

-----------------------------------------------------------------
List-Lazy-0.3.0 does not work anymore since v5.27.7-212-g894f226e51.
This is the change which swapped signatures and attributes.
This change is already documented in perldelta, but I have just
two notes​:

- The error output does not look nice. In this case it begins with

Array found where operator expected at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 43, near "$$@​)"
(Missing operator before @​)?)
"my" variable $step masks earlier declaration in same statement at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 44.
syntax error at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 36, near ") :"
Global symbol "$generator" requires explicit package name (did you
forget to declare "my $generator"?) at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 38.
Global symbol "$state" requires explicit package name (did you forget
to declare "my $state"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 39.
Global symbol "$min" requires explicit package name (did you forget to
declare "my $min"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-
0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$max" requires explicit package name (did you forget to
declare "my $max"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-
0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$step" requires explicit package name (did you forget
to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43.
Invalid separator character '{' in attribute list at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 44, near "$step : sub "
Global symbol "$step" requires explicit package name (did you forget
to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 44.
...

which does not really say anything about the problem. Is it possible
to detect this situation and improve diagnostics?

- What does this change means in terms of usability of signatures?
Users
mixing signatures and prototypes must increase their perl
prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?

BTW, the issue for the CPAN module is
yanick/List-Lazy#3

-- Slaven

Test failures confirmed.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 24, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 24, 2018

From @Grinnz

On Wed, Jan 24, 2018 at 3​:04 PM, slaven@​rezic.de <perlbug-followup@​perl.org>
wrote​:

- What does this change means in terms of usability of signatures? Users
mixing signatures and prototypes must increase their perl prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?

The alternative is signatures remaining experimental forever because
attributes can't work correctly with them, so the sooner this change is
made the better in my opinion.

-Dan

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 27, 2018

From @xsawyerx

On 01/24/2018 09​:04 PM, slaven@​rezic.de wrote​:

[...]

- What does this change means in terms of usability of signatures? Users
mixing signatures and prototypes must increase their perl prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?

We don't like it, but the alternative is that you have partially broken
signatures. If we demand that signatures stay where they are, we are
demanding they stay broken when it comes to cooperating with other,
stable syntax - namely, subroutine attributes. This is literally broken
by design, in that sense.

Honestly, I feel this is getting ridiculous. We cannot even change
experimental features that were *always* experimental because people
already started using it?

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 27, 2018

From @Abigail

On Sat, Jan 27, 2018 at 12​:21​:12PM +0100, Sawyer X wrote​:

On 01/24/2018 09​:04 PM, slaven@​rezic.de wrote​:

[...]

- What does this change means in terms of usability of signatures? Users
mixing signatures and prototypes must increase their perl prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?

We don't like it, but the alternative is that you have partially broken
signatures. If we demand that signatures stay where they are, we are
demanding they stay broken when it comes to cooperating with other,
stable syntax - namely, subroutine attributes. This is literally broken
by design, in that sense.

Honestly, I feel this is getting ridiculous. We cannot even change
experimental features that were *always* experimental because people
already started using it?

That's the danger of keeping useful features experimental for a long
time, specially, if they have been stable for a while.

I'd very much like to see experimental features to be time boxed​:
after say, 2 releases, either the feature is considered stable,
or it will be scratched (at least, as is).

Abigail

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 27, 2018

From @eserte

Dana Sat, 27 Jan 2018 03​:21​:32 -0800, xsawyerx@​gmail.com reče​:

On 01/24/2018 09​:04 PM, slaven@​rezic.de wrote​:

[...]

- What does this change means in terms of usability of signatures? Users
mixing signatures and prototypes must increase their perl prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?

We don't like it, but the alternative is that you have partially broken
signatures. If we demand that signatures stay where they are, we are
demanding they stay broken when it comes to cooperating with other,
stable syntax - namely, subroutine attributes. This is literally broken
by design, in that sense.

Honestly, I feel this is getting ridiculous. We cannot even change
experimental features that were *always* experimental because people
already started using it?

I was just asking if it could be done better (which is probably not a good sign that I have to ask...). So that's the dilemma of experimental features​: people should not use them because they can change, or vanish; but without using them possible problems get noticed too late.

But I found the 1st question in the ticket more important. The now not implemented order "first signature, then attributes" seems to be the natural order for me, and it's possible that this kind of error happens often in future. So can we improve diagnostics, or even support both types of order with some safety net where the order matters (like in the case of lvalue)?

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 27, 2018

From @xsawyerx

On 01/27/2018 01​:17 PM, slaven@​rezic.de via RT wrote​:

Dana Sat, 27 Jan 2018 03​:21​:32 -0800, xsawyerx@​gmail.com reče​:

On 01/24/2018 09​:04 PM, slaven@​rezic.de wrote​:

[...]

- What does this change means in terms of usability of signatures? Users
mixing signatures and prototypes must increase their perl prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?
We don't like it, but the alternative is that you have partially broken
signatures. If we demand that signatures stay where they are, we are
demanding they stay broken when it comes to cooperating with other,
stable syntax - namely, subroutine attributes. This is literally broken
by design, in that sense.

Honestly, I feel this is getting ridiculous. We cannot even change
experimental features that were *always* experimental because people
already started using it?
I was just asking if it could be done better (which is probably not a good sign that I have to ask...). So that's the dilemma of experimental features​: people should not use them because they can change, or vanish; but without using them possible problems get noticed too late.

Theoretically, yes. But it is assuming on foresight, which we are rarely
privileged to. Abigail's idea of a two-year limit to experimental could
help, but we already use two-year to determine that it's stable.

Let's say we have two years to either determine it's stable (as now) or
remove the feature (per suggestion). One year in, we realize there's a
certain limitation we can fix. Should we fix it? We can't. If we fix it,
we go above the two year limitation for removal. This can keep going, as
in the case of subroutine signatures, where we keep finding something.
The problem is that every time we assume it really is the last. For
example, in this case, we don't want to change subroutine signatures,
just their location. As a structure, they're actually stable. Their
location, however, is problematic.

What we could do is provide a limit that is longer than two years, but
we then limit ourselves to how much we can fix. We have to find all the
problems within a certain time or the feature is gone.

This sits in contrast to the benefit of releasing it as experimental.
Unless we allow it to be used, many errors will not be discovered. The
more it is out there, the more likely we are to be able to spot the
problems.

But I found the 1st question in the ticket more important. The now not implemented order "first signature, then attributes" seems to be the natural order for me, and it's possible that this kind of error happens often in future. So can we improve diagnostics, or even support both types of order with some safety net where the order matters (like in the case of lvalue)?

I agree it is more visually pleasing to see signatures prior to
attributes. However, that breaks. We had several threads in which I
provided two options for allowing it to work but both provided a hairy,
fragile solution. I did not provide an answer to your first question
because I don't think I have a good answer other than what we had
already discussed. I have also expressed the urgency of these and by now
we have passed the deadline on this change. If we were to find a
solution and change it once again, we would hit your second question
again - something that worked a certain way as experimental now has to
change.

Basically, we're damned if we do and we're damned if we don't.​:/

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 28, 2018

From @eserte

Dana Sat, 27 Jan 2018 06​:35​:49 -0800, xsawyerx@​gmail.com reče​:

On 01/27/2018 01​:17 PM, slaven@​rezic.de via RT wrote​:
[...]

But I found the 1st question in the ticket more important. The now
not implemented order "first signature, then attributes" seems to be
the natural order for me, and it's possible that this kind of error
happens often in future. So can we improve diagnostics, or even
support both types of order with some safety net where the order
matters (like in the case of lvalue)?

I agree it is more visually pleasing to see signatures prior to
attributes. However, that breaks. We had several threads in which I
provided two options for allowing it to work but both provided a
hairy,
fragile solution.

I would like to look into this discussion, but it's difficult to find these threads... well, we really need something like a TIP or PEP process...

The only thing which I see as a rationale for the change is the examples in perl5.27.8's perldelta. And the lvalue example there looks quite contrived to me.

I did not provide an answer to your first question
because I don't think I have a good answer other than what we had
already discussed. I have also expressed the urgency of these and by
now
we have passed the deadline on this change.

What deadline? If there needs to be the choice between meeting a deadline and having a good language definition, I would know what to choose.

Regards,
  Slaven

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 28, 2018

From zefram@fysh.org

slaven@​rezic.de via RT wrote​:

What deadline?

Contentious changes freeze for 5.28. The deadline was 5.27.8. Any change
to this aspect of signatures is bound to be controversial, so now it
can't be changed again until 5.29.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 28, 2018

From zefram@fysh.org

slaven@​rezic.de wrote​:

Array found where operator expected at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43, near "$$@​)"
(Missing operator before @​)?)

Mm, that's not nice. The real error is the third one emitted​:

syntax error at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 36, near ") :"

I'd be satisfied if the real error at least were the first one emitted.
It is not surprising that other errors arise following the syntax error,
but it is annoying that they would be emitted earlier. It is also
potentially misleading that the first one emitted is actually concerned
with a different sub from the one that incurs the real first error.
Clearly there's an interaction here between parser error recovery and
the statefulness of the tokeniser.

Simpler cases also emit the error messages in the wrong order, even if
not as wildly off track as in the module​:

$ perl5.27.8 -Mexperimental=signatures -lwe 'sub foo ($a) :prototype($$@​) {}'
Array found where operator expected at -e line 1, near "$$@​)"
  (Missing operator before @​)?)
  syntax error at -e line 1, near ") :"
  Execution of -e aborted due to compilation errors.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jan 28, 2018

From @cpansprout

On Sun, 28 Jan 2018 03​:23​:23 -0800, zefram@​fysh.org wrote​:

slaven@​rezic.de via RT wrote​:

What deadline?

Contentious changes freeze for 5.28. The deadline was 5.27.8. Any change
to this aspect of signatures is bound to be controversial, so now it
can't be changed again until 5.29.

I don’t think a revert of a contentious change has ever been considered contentious. (I’m not saying it should be reverted; I’m just trying to clarify how this policy has thus far been applied.)

--

Father Chrysostomos

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 4, 2018

From @xsawyerx

On 01/28/2018 11​:51 AM, slaven@​rezic.de via RT wrote​:

Dana Sat, 27 Jan 2018 06​:35​:49 -0800, xsawyerx@​gmail.com reče​:

On 01/27/2018 01​:17 PM, slaven@​rezic.de via RT wrote​:
[...]

But I found the 1st question in the ticket more important. The now
not implemented order "first signature, then attributes" seems to be
the natural order for me, and it's possible that this kind of error
happens often in future. So can we improve diagnostics, or even
support both types of order with some safety net where the order
matters (like in the case of lvalue)?
I agree it is more visually pleasing to see signatures prior to
attributes. However, that breaks. We had several threads in which I
provided two options for allowing it to work but both provided a
hairy,
fragile solution.
I would like to look into this discussion, but it's difficult to find these threads...

Agreed.

One was titled "Urgent subroutine signatures problems" from December
13th. Zefram noted this in RT #132141 on September 21st.

well, we really need something like a TIP or PEP process...

Agreed.

I wrote about such an option in a response to "We need a language design
process." on January 1st of this year.

The only thing which I see as a rationale for the change is the examples in perl5.27.8's perldelta. And the lvalue example there looks quite contrived to me.

I've had long conversations with Stevan Little about this, who pushed
that there is basically only one use that breaks​: lvalue (which is a
core subroutine attribute) with a signature that calls "return" in the
signature. On the other hand, the syntax with signatures following
attributes is quite unfriendly and can also confuse easily with
Catalyst. It might actually be better to bite the bullet on this and say
"Okay, this one case really doesn't work. If you're using lvalue
subroutine attribute, you can't return in the signature." and leave it
as the much nicer syntax.

Considering that code was in until recently, I deem it okay to move it
back one development version after. However, I will not accept this if
there is just one person (Stevan) suggesting it. I would like further
comments from others - like yourself, providing feedback on it.

I'm CC'ing Stevan here and will let him make his claim.

I did not provide an answer to your first question
because I don't think I have a good answer other than what we had
already discussed. I have also expressed the urgency of these and by
now
we have passed the deadline on this change.
What deadline? If there needs to be the choice between meeting a deadline and having a good language definition, I would know what to choose.

Theoretically freeze. I tried to signal that in the email I wrote on it.
However, in this case we have a bit of leeway moving back, as explained
above.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 6, 2018

From zefram@fysh.org

Sawyer X wrote​:

     It might actually be better to bite the bullet on this and say

"Okay, this one case really doesn't work. If you're using lvalue
subroutine attribute, you can't return in the signature." and leave it
as the much nicer syntax.

It would be asinine to compromise the functionality for what is at best
a cosmetic improvement. (Personally I find it cosmetically neutral,
so don't agree on it being an improvement in that manner, but even
the people arguing for it don't ascribe it any more than cosmetic
value.) People have decided that the cosmetics trump grammatical and
structural coherence, but surely the line must be drawn before tangible
functionality.

Not only would current functionality be lost, but also a class of
potential functionality that we're likely to want to explore. This is
not an insignificant cost, and, as with the lvalue issue, was not taken
into account in the decision to move signatures to precede attributes,
because it wasn't specifically noticed at the time. (I eventually
identified it in the @​_-suppression thread.)

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 6, 2018

From @xsawyerx

On 02/06/2018 12​:38 PM, Zefram wrote​:

Sawyer X wrote​:

     It might actually be better to bite the bullet on this and say

"Okay, this one case really doesn't work. If you're using lvalue
subroutine attribute, you can't return in the signature." and leave it
as the much nicer syntax.
It would be asinine to compromise the functionality for what is at best
a cosmetic improvement. (Personally I find it cosmetically neutral,
so don't agree on it being an improvement in that manner, but even
the people arguing for it don't ascribe it any more than cosmetic
value.) People have decided that the cosmetics trump grammatical and
structural coherence, but surely the line must be drawn before tangible
functionality.

What I am reading here is implying that cosmetic changes are
meaningless. Take into account how Catalyst breaks differently if
someone were to introduce a space, from "​:CaptureArgs(...)" to
"​:CaptureArgs (...)". This is cosmetic but will now result in a
completely different error (if at all), which could be quite confusing
to the user. We care about that.

Furthermore, one could also argue that calling return from a signature
serves little purpose and is also a cosmetic difference and is possibly
equally meaningless. Is there any value in returning from a signature
(especially when using :lvalue) which cannot be just as easily
ascertained without?

It's just a wrong place to start the discussion from, IMHO. The more I
spoke with Stevan about this, the more I realized this needs more
thought. I wish more people joined the conversation.

Not only would current functionality be lost, but also a class of
potential functionality that we're likely to want to explore.

Could you expand on this?

Have we ever explored this class of functionality? Where is the
likeliness coming from?

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 6, 2018

From zefram@fysh.org

Sawyer X wrote​:

What I am reading here is implying that cosmetic changes are
meaningless.

That's not my position. Cosmetic issues are of lower importance than
functionality and coherence, but are not entirely devoid of value.

Furthermore, one could also argue that calling return from a signature
serves little purpose and is also a cosmetic difference

No, that's not cosmetic, that's a difference in functionality. One might
still argue that it serves little purpose, of course. But that argument
is about behaviour, not aesthetics.

                Is there any value in returning from a signature

Some, yes. Returning from a default value expression can be a convenient
way to make a subroutine do different jobs depending on the arguments
supplied to it. In non-signature-using Perl it's common enough to
see constructs along the lines of "defined $foo or return $blah".
Compared to putting this logic in the body block after the signature,
having it in the signature will at times be beneficial in a number of
ways. Firstly, it is likely to group related concerns. If there are
other non-trivial default expressions then it may also be important to
return before executing the next default expression, and putting all of
that logic into the body block would involve quite a lot more verbiage.
If default value expressions have to be moved into the body block then
we also run into the issue of parameter predicates​: currently they're
not available as part of the signature system, and some people are
arguing against them ever being so, so this might force ugly use of @​_
(or incorrect use of defined()) alongside the signature.

There's also more value than this in permitting returning from a
signature. Perl benefits from the popular pardigm of interchangeability
of expressions, that any expression can go anywhere that an expression
is permitted. A return expression is a type of expression, and it
doesn't take any verbiage to explain, or neurons to remember, that it is
permitted inside a while() condition, or in a sub parameter expression,
or in any other particular place that takes an expression. But if it's
not permitted in a default value expression in a signature, that's an
exception to the general rule, which does need to be specially explained
in the documentation, and does need to be specially remembered by users,
to explain why their program doesn't work on the rare occasion that the
nature of a subroutine leads them to put this type of expression in that
type of expression slot.

It's just a wrong place to start the discussion from, IMHO.

What is?

Have we ever explored this class of functionality? Where is the
likeliness coming from?

The potential that would be lost is to have attributes affect the
compilation of a subroutine's body code. The :lvalue attribute is an
instance of that kind of functionality. Another likely instance of it
came up in one of the threads about @​_ suppression.

The general concept being discussed was that we're going to make direct
use of @​_ obsolete, through signatures and maybe other features, so it
would be nice for performance if we could suppress the construction of
the dynamic @​_ on subs that don't need it. The starting point for the
discussion was the idea of tying this suppression to signature syntax
per se, either to its enablement or to actual signatures, but we wisely
decided against that. Among control mechanisms discussed was the fairly
obvious possibility of a subroutine attribute offering explicit control
of @​_ setup for a specific subroutine. This in itself doesn't pose an
ordering problem.

Another aspect of @​_ suppression being discussed was what should happen
to direct references to @​_ that appear in the body of an @​_-suppressed
subroutine, and one of the main options was that this should be an error.
For it to be an error requires that the @​_ status of a subroutine be
known before any of its body code is compiled, and in the absence of
signatures this would be entirely compatible with attribute control.
But with signatures coming before attributes, some body code that might
refer to @​_ would be compiled before the @​_ status of the subroutine
is determined. Any option for @​_ suppression that affects how direct
references to @​_ compile is incompatible with control by an attribute
if the attribute would appear after a signature.

There's quite a problem here in that it looks somewhat inevitable
for there to be some kind of per-sub control over @​_ suppression.
Lexical flags can set defaults, but there's always going to be
the occasional exception. What we find here is that if subroutine
attributes preceding the body didn't exist then it would be necessary
to invent them. Currently we have them, but if they are abolished again
by moving signatures in front of attributes again then this would create
some design pressure to invent something new that does a similar job to
attributes but appears before a signature.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 8, 2018

From @xsawyerx

On 02/06/2018 03​:43 PM, Zefram wrote​:

Sawyer X wrote​:

What I am reading here is implying that cosmetic changes are
meaningless.
That's not my position. Cosmetic issues are of lower importance than
functionality and coherence, but are not entirely devoid of value.

Agreed.

Furthermore, one could also argue that calling return from a signature
serves little purpose and is also a cosmetic difference
No, that's not cosmetic, that's a difference in functionality. One might
still argue that it serves little purpose, of course. But that argument
is about behaviour, not aesthetics.

This functionality is equivalent to writing "return" inside the sub.

(Though this does suggest that signatures themselves are cosmetic, which
in a way, they are.)

                Is there any value in returning from a signature

Some, yes. Returning from a default value expression can be a convenient
way to make a subroutine do different jobs depending on the arguments
supplied to it. In non-signature-using Perl it's common enough to
see constructs along the lines of "defined $foo or return $blah".
Compared to putting this logic in the body block after the signature,
having it in the signature will at times be beneficial in a number of
ways. Firstly, it is likely to group related concerns. If there are
other non-trivial default expressions then it may also be important to
return before executing the next default expression, and putting all of
that logic into the body block would involve quite a lot more verbiage.

I fail to imagine this being less verbose in a signature, rather than
the subroutine body.

If default value expressions have to be moved into the body block then
we also run into the issue of parameter predicates​: currently they're
not available as part of the signature system, and some people are
arguing against them ever being so, so this might force ugly use of @​_
(or incorrect use of defined()) alongside the signature.

This isn't the cost of not moving them. The cost is a narrow case of​:

* Using :lvalue
* Using return in the subroutine

That's it, as far as I understand.

There's also more value than this in permitting returning from a
signature. Perl benefits from the popular pardigm of interchangeability
of expressions, that any expression can go anywhere that an expression
is permitted. A return expression is a type of expression, and it
doesn't take any verbiage to explain, or neurons to remember, that it is
permitted inside a while() condition, or in a sub parameter expression,
or in any other particular place that takes an expression. But if it's
not permitted in a default value expression in a signature, that's an
exception to the general rule, which does need to be specially explained
in the documentation, and does need to be specially remembered by users,
to explain why their program doesn't work on the rare occasion that the
nature of a subroutine leads them to put this type of expression in that
type of expression slot.

True, but it is an edge case. Edge cases exist, especially in Perl. The
question is, how much of an edge it is, how valuable it is to have it
vs. having signatures that are more visually consistent and can provide
better output on a mistake for attributes that contain spaces.

It's just a wrong place to start the discussion from, IMHO.
What is?

That referred to cosmetics. You can ignore it because we addressed it above.

Have we ever explored this class of functionality? Where is the
likeliness coming from?
The potential that would be lost is to have attributes affect the
compilation of a subroutine's body code. The :lvalue attribute is an
instance of that kind of functionality.

Are there any others?

Another likely instance of it
came up in one of the threads about @​_ suppression.

The general concept being discussed was that we're going to make direct
use of @​_ obsolete, through signatures and maybe other features, so it
would be nice for performance if we could suppress the construction of
the dynamic @​_ on subs that don't need it. The starting point for the
discussion was the idea of tying this suppression to signature syntax
per se, either to its enablement or to actual signatures, but we wisely
decided against that. Among control mechanisms discussed was the fairly
obvious possibility of a subroutine attribute offering explicit control
of @​_ setup for a specific subroutine. This in itself doesn't pose an
ordering problem.

I'm sorry. I couldn't follow this.

Another aspect of @​_ suppression being discussed was what should happen
to direct references to @​_ that appear in the body of an @​_-suppressed
subroutine, and one of the main options was that this should be an error.
For it to be an error requires that the @​_ status of a subroutine be
known before any of its body code is compiled, and in the absence of
signatures this would be entirely compatible with attribute control.
But with signatures coming before attributes, some body code that might
refer to @​_ would be compiled before the @​_ status of the subroutine
is determined. Any option for @​_ suppression that affects how direct
references to @​_ compile is incompatible with control by an attribute
if the attribute would appear after a signature.

This is also hard for me to follow. Is the summary that we would not be
able to make direct references @​_ that appear in the body of an
@​_-suppressed body an error?

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 8, 2018

From zefram@fysh.org

Sawyer X wrote​:

I fail to imagine this being less verbose in a signature, rather than
the subroutine body.

Consider

  sub foo :lvalue ($a = return $p, $b = assign_default_b($a)) {
  return substr($a // $q, $b // 3, 1);
  }

versus

  sub foo ($a = undef, $b = undef) :lvalue {
  return $p if @​_ == 0;
  $b = assign_default_b($a) if @​_ <= 1;
  return substr($a // $q, $b // 3, 1);
  }

Observe that the conditions involving the presence of arguments, which
were implicit in the signature with the use of default value expressions,
have to be written explicitly in the body block. There's also a need
for fake default values in the signature. Those aspects add verbosity.
Note also that the conditions have to be written using @​_ and argument
indices, adding a maintenance burden over having them in the signature.
The real defaulting code is also separated from the parameter declaration
in the signature, thus making one jump up and down the code when
reading it.

This isn't the cost of not moving them. The cost is a narrow case of​:

I'm not clear what you're trying to say here. You seem to be trying to
say that the situation I described doesn't apply to the lvalue issue?
It would be incorrect to say that. I described costs of having to
move parameter defaulting logic from a signature into the body block.
Putting signatures before attributes of course does not mean that *all*
defaulting logic would have to be moved. It only affects some subset
of default value expressions, likely just those containing return
expressions. Maybe even only on lvalue subs, though implementing that
would be an extra level of trickiness because of the :lvalue attribute
coming late. In the affected situations, the issues that I described
would be free to occur.

True, but it is an edge case.

Getting the edge cases right matters. Getting them wrong means the
abstractions are broken and users need to worry about whether the edge
cases could occur.

vs. having signatures that are more visually consistent

Er, what? Consistent with what? Signatures *after* attributes is what
is consistent with the meanings of signatures and attributes.

I'm sorry. I couldn't follow this.

Perhaps you'd do better reading the actual thread in which it arose.
Though there's quite a lot of it.

This is also hard for me to follow. Is the summary that we would not be
able to make direct references @​_ that appear in the body of an
@​_-suppressed body an error?

Pretty much.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 9, 2018

From @eserte

Dana Wed, 24 Jan 2018 12​:04​:24 -0800, slaven@​rezic.de reče​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.8.

-----------------------------------------------------------------
List-Lazy-0.3.0 does not work anymore since v5.27.7-212-g894f226e51.
This is the change which swapped signatures and attributes.
This change is already documented in perldelta, but I have just
two notes​:

- The error output does not look nice. In this case it begins with

Array found where operator expected at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 43, near "$$@​)"
(Missing operator before @​)?)
"my" variable $step masks earlier declaration in same statement at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 44.
syntax error at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 36, near ") :"
Global symbol "$generator" requires explicit package name (did you
forget to declare "my $generator"?) at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 38.
Global symbol "$state" requires explicit package name (did you forget
to declare "my $state"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 39.
Global symbol "$min" requires explicit package name (did you forget to
declare "my $min"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-
0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$max" requires explicit package name (did you forget to
declare "my $max"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-
0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$step" requires explicit package name (did you forget
to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43.
Invalid separator character '{' in attribute list at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 44, near "$step : sub "
Global symbol "$step" requires explicit package name (did you forget
to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 44.
...

which does not really say anything about the problem. Is it possible
to detect this situation and improve diagnostics?

- What does this change means in terms of usability of signatures?
Users
mixing signatures and prototypes must increase their perl
prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?

BTW, the issue for the CPAN module is
yanick/List-Lazy#3

-- Slaven

Also affected​: ALEXBYK/Evo-0.0405.tar.gz
(Bisect result found out by Andreas)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 15, 2018

From zefram@fysh.org

I wrote​:

I'd be satisfied if the real error at least were the first one emitted.

Looking more closely, the reason why the real error isn't emitted first
is that it's queued up through the qerror() system. This is a deliberate
move to collect up multiple compilation errors, so that multiple errors
can be reported even if errors are being reported by means other than
emitting a message to stderr. For example, an eval() that encounters
multiple compilation errors will yield a $@​ that contains multiple
concatenated error messages. (And indeed, multiple errors do occur
with List​::Lazy.)

The reason why some other messages appear before the errors is that the
earlier messages aren't errors, they're warnings. These go straight to
stderr as they arise, rather than being collected with the errors.

All of these bits of behaviour are individually intentional, but they
have the undesirable effect of losing synchronisation between the warning
stream and the error stream. This can't be fixed without some fairly big
change to the way compile-time diagnostics are managed. We could perhaps
abandon the queuing of errors, aborting compilation immediately on the
first error; in this case that would lead to emitting the "syntax error"
message and nothing else. Doing that would inevitably garner complaints
about situations in which multiple error messages are actually helpful.
Any approach around shuffling the order in which messages are emitted
would be hideously complicated. Maybe we could do something else to
distinguish between warnings and errors, to draw greater attention to
the first error even if it's not the first message emitted.

We can't fix this for 5.28.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 15, 2018

From Eirik-Berg.Hanssen@allverden.no

On Thu, Feb 15, 2018 at 8​:06 PM, Zefram <zefram@​fysh.org> wrote​:

All of these bits of behaviour are individually intentional, but they
have the undesirable effect of losing synchronisation between the warning
stream and the error stream. This can't be fixed without some fairly big
change to the way compile-time diagnostics are managed. We could perhaps
abandon the queuing of errors, aborting compilation immediately on the
first error; in this case that would lead to emitting the "syntax error"
message and nothing else. Doing that would inevitably garner complaints
about situations in which multiple error messages are actually helpful.
Any approach around shuffling the order in which messages are emitted
would be hideously complicated. Maybe we could do something else to
distinguish between warnings and errors, to draw greater attention to
the first error even if it's not the first message emitted.

  Naïve suggestion​: On the first error, make all warnings fatal for the
duration of the queue, so the warnings are queued with the errors?

Eirik

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 15, 2018

From zefram@fysh.org

Eirik Berg Hanssen wrote​:

Naive suggestion​: On the first error, make all warnings fatal for the
duration of the queue, so the warnings are queued with the errors?

That would break nested things that would incur warnings in a different
context.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 15, 2018

From Eirik-Berg.Hanssen@allverden.no

On Thu, Feb 15, 2018 at 8​:35 PM, Zefram <zefram@​fysh.org> wrote​:

Eirik Berg Hanssen wrote​:

Naive suggestion​: On the first error, make all warnings fatal for the
duration of the queue, so the warnings are queued with the errors?

That would break nested things that would incur warnings in a different
context.

  Still naïve, I fear​: Does it matter?

  This is "on the first error" and "for the duration of the queue", so what
it breaks wouldn't work anyway, would it?

Eirik

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 15, 2018

From @cpansprout

On Thu, 15 Feb 2018 11​:35​:56 -0800, zefram@​fysh.org wrote​:

Eirik Berg Hanssen wrote​:

Naive suggestion​: On the first error, make all warnings fatal for the
duration of the queue, so the warnings are queued with the errors?

That would break nested things that would incur warnings in a different
context.

If, when the first error is queued, we just make warnings fatal for the current compilation scope, I think it would work. You don’t have to worry about nested scopes, because BEGIN blocks will not run after an error.

--

Father Chrysostomos

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 15, 2018

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

You don't have to worry about nested scopes, because BEGIN blocks will
not run after an error.

There are plenty of ways for code to run other than BEGIN blocks.
Also ways for the warning flags to become visible other than through
warnings being generated.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 17, 2018

From @nwc10

On Tue, Feb 06, 2018 at 10​:38​:40AM +0000, Zefram wrote​:

Sawyer X wrote​:

     It might actually be better to bite the bullet on this and say

"Okay, this one case really doesn't work. If you're using lvalue
subroutine attribute, you can't return in the signature." and leave it
as the much nicer syntax.

"can't return" - which would need to be enforced? At compile time? As a hard
error?

Before the recent change (with attributes after signatures) what happened
if one use a return in a signature for a function with an lvalue attribute?

A) error at compile time (presume not, but referenced below)
B) error at run time
C) silently does the wrong thing

?

and if we wanted to go back to the other syntax order

1) what of those three we do in the case of discovering the lvalue attribute
  after we've compiled the code in signature?
2) would we ban all returns in signatures (to be consistent)?

Something about all this is bothering me a lot.

(And it's bothering me that I can't get it clear enough in my head to write
it without forward references.)

I think that it's a combination of two things. The second comes below, and
relates to this​:

Not only would current functionality be lost, but also a class of
potential functionality that we're likely to want to explore. This is
not an insignificant cost, and, as with the lvalue issue, was not taken
into account in the decision to move signatures to precede attributes,
because it wasn't specifically noticed at the time. (I eventually
identified it in the @​_-suppression thread.)

-zefram

But the first is that we're seriously proposing to (re-)add a special case.

Yes, the implementation is full of them. But we should be striving to remove
them, not take the viewpoint that "another one isn't going to make much of a
difference". As Zefram writes in a later reply​:

There's also more value than this in permitting returning from a
signature. Perl benefits from the popular pardigm of interchangeability
of expressions, that any expression can go anywhere that an expression
is permitted. A return expression is a type of expression, and it
doesn't take any verbiage to explain, or neurons to remember, that it is
permitted inside a while() condition, or in a sub parameter expression,
or in any other particular place that takes an expression. But if it's
not permitted in a default value expression in a signature, that's an
exception to the general rule, which does need to be specially explained
in the documentation, and does need to be specially remembered by users,
to explain why their program doesn't work on the rare occasion that the
nature of a subroutine leads them to put this type of expression in that
type of expression slot.

And I think it very unlikely that folks are going to read the documentation,
or at least *remember* it at the time that they're writing the code.

Putting attributes first (at the cost of another special case to remember)
seems to be one of those places where we're (roughly) making it easier for
the "beginner" at the cost of making it harder for everyone else.

There's a Larry Wall quote/observation about this (which I can't find) to
the effect of being hesitant to make this type of design choice because it
actually penalises everyone (forever), *including* the beginners, because
they don't stay as beginners. (But I can't find it.)

The second thing that bothers me a lot is this​:

The potential that would be lost is to have attributes affect the
compilation of a subroutine's body code. The :lvalue attribute is an
instance of that kind of functionality. Another likely instance of it
came up in one of the threads about @​_ suppression.

[more detail]

There's quite a problem here in that it looks somewhat inevitable
for there to be some kind of per-sub control over @​_ suppression.
Lexical flags can set defaults, but there's always going to be
the occasional exception. What we find here is that if subroutine
attributes preceding the body didn't exist then it would be necessary
to invent them. Currently we have them, but if they are abolished again
by moving signatures in front of attributes again then this would create
some design pressure to invent something new that does a similar job to
attributes but appears before a signature.

Effectively here with lvalues we've already got a non-theoretical use case
of why attributes need to be known before the signature is parsed.

(Due to how perl 5 compiles code to optrees as soon as it's parsed. And
*that* isn't going to change without a 7-figure development budget.
The tokeniser is about 12K lines of code, op.c a further 17K. It's all
intertwined code with no real documentation, and enough corner cases that
almost any change will break *something* on CPAN, which requires a human
to work out which side needs to fix what.)

Zefram states another - any pragmatic implementation of @​_ suppression is
going to want to change how the optree is generated, and the signature is
code and hence makes an optree.

Any other functionality that we might want to add that involves

1) argument parsing
2) code that wraps functions
3) optree generation

needs to be known about before the signature is parsed, which means that
whatever enables it has to come before it. Whilst most features make sense
to enable lexically, the ones that make sense *per-subroutine* (rather than
*per-scope*) belong as attributes.

I tried to think of stuff that we might want to do where they are most
logically enabled via (core) subroutine attributes and apply to the signature
too​:

sub foo :inlineable {
  ...
}

to signal that a subroutine should be inlined (where possible, and hence
assume not to be redefinable or wrapped).

Likely implementing such a feature would need to compile to a different set
of template OPs, which can then be used to splice into caller subroutines,
but that requirement for template OPs would apply to a signature too.

and then likely rather hard to implement​:

sub foo :multimethod(Bar​: $bar, Baz​: $baz) {
  ...
}

but the declaration of which arguments are part of dispatch likely can't fit
within the syntax of a signature, as Perl 5 doesn't have the colon available.

(Remember, Perl 6 changes the ternary op to C<?? !!> which frees up the colon
and hence lot of syntax possibilities which just can't work in Perl 5)

and given that the attribute has declared 2 (or more) arguments, they wouldn't
be repeated in the signature, but would affect argument processing, and hence
how @​_ is handled and optimised.

For completeness, but probably not useful​:

sub foo :cached {
  ...
}

a built-in Memoize equivalent. Which logically *should* wrap code in the
signature too, as signatures can make expensive computations, or hide this
as calls to functions which are expensive.

and also possibly not useful​:

sub foo :method {
  # implicit $self
  ...
}

sub foo :method(Bar​: $this) {
  ...
}

as it's more logically covered by expanding signatures to take types (if
there is syntax that is workable) or a keyword.

But such an attribute could prefix the subroutine's code with a check that
the first argument is an object (or an object of the correct type), and
throw a meaningful exception if not, instead of generating a "cryptic"
"Can't call method "..." without a package or object reference" from the
code inside the signature or sub body. And again like the multimethod
attribute, one wouldn't repeat the argument declaration in the signature.

  [Sawyer]​:
[Zefram]​:

This isn't the cost of not moving them. The cost is a narrow case of​:

I'm not clear what you're trying to say here. You seem to be trying to
say that the situation I described doesn't apply to the lvalue issue?
It would be incorrect to say that. I described costs of having to
move parameter defaulting logic from a signature into the body block.
Putting signatures before attributes of course does not mean that *all*
defaulting logic would have to be moved. It only affects some subset
of default value expressions, likely just those containing return
expressions. Maybe even only on lvalue subs, though implementing that
would be an extra level of trickiness because of the :lvalue attribute
coming late. In the affected situations, the issues that I described
would be free to occur.

I'm not sure either.

But there is a clear cost here of preventing a whole class of future
improvements. It's not just the narrow case of lvalue subroutines.

True, but it is an edge case.

Getting the edge cases right matters. Getting them wrong means the
abstractions are broken and users need to worry about whether the edge
cases could occur.

Particularly if we can't detect the edge cases at compile time.
And even if we *can*, that might in itself be more implementation cost.

This one is not easy to balance either. It's already really damn hard to
maintain the parser code - there seem to be about 3 people total in the
world who are doing anything more than keyhole surgery bugfixes.
If we make trade-offs that increase this cost we may reduce this number
towards zero, which becomes a big problem as it's very hard to unwind and
hence get back out of the dead end.

Nicholas Clark

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 18, 2018

From @xsawyerx

Nicholas, thank you for replying in such detail. Please forgive my
brevity in return.

On 02/17/2018 12​:00 PM, Nicholas Clark wrote​:

[...]
But the first is that we're seriously proposing to (re-)add a special case.

Yes, the implementation is full of them. But we should be striving to remove
them, not take the viewpoint that "another one isn't going to make much of a
difference".

You are right. I thought I had made the decision on changing the syntax
back in haste and decided to pause and take more time to reflect on it.
In retrospective, it might have been done in haste, but it's likely to
be the best of the two options nonetheless.

[...]
The second thing that bothers me a lot is this​:

The potential that would be lost is to have attributes affect the
compilation of a subroutine's body code. The :lvalue attribute is an
instance of that kind of functionality. Another likely instance of it
came up in one of the threads about @​_ suppression.
[more detail]

There's quite a problem here in that it looks somewhat inevitable
for there to be some kind of per-sub control over @​_ suppression.
Lexical flags can set defaults, but there's always going to be
the occasional exception. What we find here is that if subroutine
attributes preceding the body didn't exist then it would be necessary
to invent them. Currently we have them, but if they are abolished again
by moving signatures in front of attributes again then this would create
some design pressure to invent something new that does a similar job to
attributes but appears before a signature.
Effectively here with lvalues we've already got a non-theoretical use case
of why attributes need to be known before the signature is parsed.

(Due to how perl 5 compiles code to optrees as soon as it's parsed. And
*that* isn't going to change without a 7-figure development budget.
The tokeniser is about 12K lines of code, op.c a further 17K. It's all
intertwined code with no real documentation, and enough corner cases that
almost any change will break *something* on CPAN, which requires a human
to work out which side needs to fix what.)

Zefram states another - any pragmatic implementation of @​_ suppression is
going to want to change how the optree is generated, and the signature is
code and hence makes an optree.

Any other functionality that we might want to add that involves

1) argument parsing
2) code that wraps functions
3) optree generation

needs to be known about before the signature is parsed, which means that
whatever enables it has to come before it. Whilst most features make sense
to enable lexically, the ones that make sense *per-subroutine* (rather than
*per-scope*) belong as attributes.

I tried to think of stuff that we might want to do where they are most
logically enabled via (core) subroutine attributes and apply to the signature
too​:

sub foo :inlineable {
...
}

to signal that a subroutine should be inlined (where possible, and hence
assume not to be redefinable or wrapped).

Likely implementing such a feature would need to compile to a different set
of template OPs, which can then be used to splice into caller subroutines,
but that requirement for template OPs would apply to a signature too.

and then likely rather hard to implement​:

sub foo :multimethod(Bar​: $bar, Baz​: $baz) {
...
}

but the declaration of which arguments are part of dispatch likely can't fit
within the syntax of a signature, as Perl 5 doesn't have the colon available.

(Remember, Perl 6 changes the ternary op to C<?? !!> which frees up the colon
and hence lot of syntax possibilities which just can't work in Perl 5)

and given that the attribute has declared 2 (or more) arguments, they wouldn't
be repeated in the signature, but would affect argument processing, and hence
how @​_ is handled and optimised.

For completeness, but probably not useful​:

sub foo :cached {
...
}

a built-in Memoize equivalent. Which logically *should* wrap code in the
signature too, as signatures can make expensive computations, or hide this
as calls to functions which are expensive.

and also possibly not useful​:

sub foo :method {
# implicit $self
...
}

sub foo :method(Bar​: $this) {
...
}

as it's more logically covered by expanding signatures to take types (if
there is syntax that is workable) or a keyword.

But such an attribute could prefix the subroutine's code with a check that
the first argument is an object (or an object of the correct type), and
throw a meaningful exception if not, instead of generating a "cryptic"
"Can't call method "..." without a package or object reference" from the
code inside the signature or sub body. And again like the multimethod
attribute, one wouldn't repeat the argument declaration in the signature.

[Sawyer]​:
[Zefram]​:

This isn't the cost of not moving them. The cost is a narrow case of​:
I'm not clear what you're trying to say here. You seem to be trying to
say that the situation I described doesn't apply to the lvalue issue?
It would be incorrect to say that. I described costs of having to
move parameter defaulting logic from a signature into the body block.
Putting signatures before attributes of course does not mean that *all*
defaulting logic would have to be moved. It only affects some subset
of default value expressions, likely just those containing return
expressions. Maybe even only on lvalue subs, though implementing that
would be an extra level of trickiness because of the :lvalue attribute
coming late. In the affected situations, the issues that I described
would be free to occur.
I'm not sure either.

But there is a clear cost here of preventing a whole class of future
improvements. It's not just the narrow case of lvalue subroutines.

[...]

I appreciate the time and effort you took in writing this. The points
both Zefram and yourself raise provide a solid case and we should indeed
keep it the way it is right now.

Thank you, Nick, Zefram, and Yves.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

From @nwc10

On Sun, Feb 18, 2018 at 11​:12​:01PM +0200, Sawyer X wrote​:

Nicholas, thank you for replying in such detail. Please forgive my
brevity in return.

That is really not a problem.

On 02/17/2018 12​:00 PM, Nicholas Clark wrote​:

[...]
But the first is that we're seriously proposing to (re-)add a special case.

Yes, the implementation is full of them. But we should be striving to remove
them, not take the viewpoint that "another one isn't going to make much of a
difference".

You are right. I thought I had made the decision on changing the syntax
back in haste and decided to pause and take more time to reflect on it.
In retrospective, it might have been done in haste, but it's likely to
be the best of the two options nonetheless.

For some reason your choice of words made me laugh (in a good way).
Which is a good thing on a cold Monday morning.

I think you're right to reflect on it. I'm not meaning to sound pessemistic,
but it does feel somewhat like "least worst choice". Both "suck", but one
"sucks" more than the other. (I think more nuanced, "both have
disadvantages, but the disadvantages of one run far deeper than than the
other".)

I suspect that a lot of this awkwardness comes from how signatures feel like
they are part of the subroutine's name/declaration/"shape", but as soon as
you have defaults they are (implementation wise) part of its body, because
they define the first section of code in the body.

Maybe I'm over-generalising, or making a false connection, but it seems like
it's the same problem/frustration as the (quite reasonable) desire of
various people to provide a way to introspect signatures - "how many
arguments does this function take?", "what types?", with the expectation
that this can be mapped into a data structure. Which it can. Until it can't.
"90%" of cases will, but as soon as defaults are more complex than
constants it unravells.

And over-over-generalising, that makes me think of

  If the builtin's arguments cannot be adequately expressed by a prototype
  (such as L<C<system>|/system LIST>), L<C<prototype>|/prototype FUNCTION>
  returns L<C<undef>|/undef EXPR>, because the builtin does not really
  behave like a Perl function. Otherwise, the string describing the
  equivalent prototype is returned.

C<prototype> returns the prototype. Unless it can't.

Thank you, Nick, Zefram, and Yves.

Thank you for all the time you put into this, and trying to ensure that
we're taking decisions that are the least worst trade-off. (Even though we
don't always get it right. Although hopefully we learn from previous
mistakes and oversights, so that we can make brave new mistakes instead of
just riffs and variations of the ones before.)

And it was more fun than what I was doing on Sunday. (I don't know exactly
how they made the ceilings of our house so hard, but it's the most horrible
thing I've ever had to drill into.)

Nicholas Clark

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 20, 2018

From @iabyn

On Mon, Feb 19, 2018 at 07​:44​:49AM +0000, Nicholas Clark wrote​:
[snip lots of stuff about attr and sig order]

The more I think about this, the more I think that attributes should
be allowed in *either* position, with a warning or croak for specific
attributes found in the "wrong" location​: currently just :lvalue,
but potentially other attributes added later that could affect the
signature etc.

This has the following advantages​:

1) It allows :lvalue (and any future attributes) to be put in the
correct place.

2) It allows current and future code using signatures to run on perls back
as far as 5.22.0 (assuming that basic signatures don't change radically
between now and whenever they stop being experimental).

In particular it will allow the relatively common idiom of
  sub foo (....) :prototype(...) { ... }
to work in future and as far back as 5.22.0.

3) It doesn't need any fragile hacks like disallowing 'return' within
default signature code.

I'm prepared to implement this if a consensus is reached (although I
suspect it may be slightly tricky).

--
Standards (n). Battle insignia or tribal totems.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 20, 2018

From @demerphq

On 21 Feb 2018 00​:14, "Dave Mitchell" <davem@​iabyn.com> wrote​:

On Mon, Feb 19, 2018 at 07​:44​:49AM +0000, Nicholas Clark wrote​:
[snip lots of stuff about attr and sig order]

The more I think about this, the more I think that attributes should
be allowed in *either* position, with a warning or croak for specific
attributes found in the "wrong" location​: currently just :lvalue,
but potentially other attributes added later that could affect the
signature etc.

This has the following advantages​:

1) It allows :lvalue (and any future attributes) to be put in the
correct place.

2) It allows current and future code using signatures to run on perls back
as far as 5.22.0 (assuming that basic signatures don't change radically
between now and whenever they stop being experimental).

In particular it will allow the relatively common idiom of
  sub foo (....) :prototype(...) { ... }
to work in future and as far back as 5.22.0.

3) It doesn't need any fragile hacks like disallowing 'return' within
default signature code.

I'm prepared to implement this if a consensus is reached (although I
suspect it may be slightly tricky).

Before we say yes can you outline how one would mark an attribute as
requiring LHS usage?

Yves

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 21, 2018

From zefram@fysh.org

Dave Mitchell wrote​:

The more I think about this, the more I think that attributes should
be allowed in *either* position, with a warning or croak for specific
attributes found in the "wrong" location​:

That would be a bad idea. Although it would make it possible to put
:lvalue in the right place, possibility isn't enough. The idea has most
of the other downsides of signatures preceding attributes. It still
breaks the attribute abstraction, by forcing a visible distinction
between attributes that are permitted to follow signatures and those that
are not. This distinction would have to be documented, and remembered
by programmers, who would no longer be able to just compose features in
an orthogonal manner.

Any attribute that has been classed as permitted to follow a signature
would be barred from ever being reimplemented in a way that alters
compilation of ordinary code. That kind of reimplementation is an
option that is otherwise open to attributes. In the likely event that
we later want to apply non-core attributes early enough to affect body
compilation, of course non-core attributes would labour under these
same burdens. But further, we'd have to introduce some extra API for
non-core attributes to declare which category they fall into.

Under these conditions, we couldn't really say that signatures were
playing nicely with the existing stable feature of the :lvalue attribute.
They would certainly be impeding likely future developments. This could
bar the blessing of signatures, in that form, as a stable feature.

Relative to the present situation of having signatures after all
attributes, the proposal for double helpings of attributes has only one
advantage, namely compatibility with the 5.22-5.26 version of signatures.
There certainly would be some benefit there, but it's quite limited,
because the compatibility is only with an experimental feature (marked as
such) in its early releases. The same kind of compatibility benefit with
respect to 5.20 didn't dissuade people from moving signatures into the
wrong place for 5.22. In this case, we must weigh against this temporary
gain the permanent damage that dual location would do to attributes,
as described above.

The balancing exercise is easy​: compatibility that was disclaimed can't
outweigh long-term language coherence. It is fortunate that we had
the formalised concept of experimental features in time for signatures,
in order to make it this easy.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 21, 2018

From @eserte

Dana Fri, 09 Feb 2018 14​:21​:58 -0800, slaven@​rezic.de reče​:

Dana Wed, 24 Jan 2018 12​:04​:24 -0800, slaven@​rezic.de reče​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.8.

-----------------------------------------------------------------
List-Lazy-0.3.0 does not work anymore since v5.27.7-212-g894f226e51.
This is the change which swapped signatures and attributes.
This change is already documented in perldelta, but I have just
two notes​:

- The error output does not look nice. In this case it begins with

Array found where operator expected at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 43, near "$$@​)"
(Missing operator before @​)?)
"my" variable $step masks earlier declaration in same statement at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 44.
syntax error at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 36, near ") :"
Global symbol "$generator" requires explicit package name (did you
forget to declare "my $generator"?) at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 38.
Global symbol "$state" requires explicit package name (did you forget
to declare "my $state"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 39.
Global symbol "$min" requires explicit package name (did you forget to
declare "my $min"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-
0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$max" requires explicit package name (did you forget to
declare "my $max"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-
0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$step" requires explicit package name (did you forget
to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43.
Invalid separator character '{' in attribute list at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 44, near "$step : sub "
Global symbol "$step" requires explicit package name (did you forget
to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 44.
...

which does not really say anything about the problem. Is it possible
to detect this situation and improve diagnostics?

- What does this change means in terms of usability of signatures?
Users
mixing signatures and prototypes must increase their perl
prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?

BTW, the issue for the CPAN module is
yanick/List-Lazy#3

-- Slaven

Also affected​: ALEXBYK/Evo-0.0405.tar.gz
(Bisect result found out by Andreas)

Also effected (very probably)​: STEVAN/Moxie-0.07.tar.gz
(This was yet unnoticed because the distribution did not build at all since 5.27.6, probably due to broken dependencies)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 22, 2018

From @iabyn

On Wed, Feb 21, 2018 at 06​:56​:26PM +0000, Zefram wrote​:

Dave Mitchell wrote​:

The more I think about this, the more I think that attributes should
be allowed in *either* position, with a warning or croak for specific
attributes found in the "wrong" location​:

That would be a bad idea.
[snip]

Ok, I'm convinced (mostly).

I plan to do the following two things shortly (assuming they prove
feasible)​:

1) change the parser so that subs under 'use feature signatures' use
a different grammar rule than subs not under it - possibly by making the
toker return two different sub tokens, e.g. 'SUB' and a new 'SIGSUB'
token say, depending on whether the feature is in scope.

This will then help reduce confusing errors, e.g. this code with a syntax
error (attrs before prototype)​:

  no feature 'signatures';
  sub f :lvalue ($$@​) { $x = 1 }

currently gives​:

  Illegal character following sigil in a subroutine signature at
  ..., near "($"

It's parsing the ($$@​) as a sub signature even though signatures aren't in
scope, so the error message is confusing.

2) For the signature sub grammar rule, allow it to spot attributes
following a signature and croak with a meaningful error. Which is what the
OP requested.

--
But Pity stayed his hand. "It's a pity I've run out of bullets",
he thought. -- "Bored of the Rings"

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 22, 2018

From @ribasushi

On 02/21/2018 07​:56 PM, Zefram wrote​:

Relative to the present situation of having signatures after all
attributes, the proposal for double helpings of attributes has only one
advantage, namely compatibility with the 5.22-5.26 version of signatures.

The above statement is false.

Zefram's argument over the past several months is predicated on the idea
that signatures never existed in a stable form within the Perl5
ecosystem, and didn't exist at all before 5.22.

That idea is false (at best)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 22, 2018

From @demerphq

On 22 Feb 2018 20​:11, "Peter Rabbitson" <rabbit-p5p@​rabbit.us> wrote​:

On 02/21/2018 07​:56 PM, Zefram wrote​:

Relative to the present situation of having signatures after all
attributes, the proposal for double helpings of attributes has only one
advantage, namely compatibility with the 5.22-5.26 version of signatures.

The above statement is false.

Zefram's argument over the past several months is predicated on the idea
that signatures never existed in a stable form within the Perl5 ecosystem,
and didn't exist at all before 5.22.

The above statement is at best a half truth. Not only that it is an
unhelpful contribution to the discussion.

The key problem is that putting signatures before attributes means that the
potential uses of attributes are severely limited, especially given the
complexity we have allowed in the signature definition.

This point is utterly devastating to the counter-arguments that have been
made. They all fall flat when this is taken into account.

The additional arguments about experimental status are merely icing on the
cake, or perhaps better put as the final nails in the coffin.

If you have something more useful to contribute that accusing a key
contributor of telling falsehoods then please do so, but the type of
feedback provided here is unhelpful and imo unwelcome.

Thanks,
Yved

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 22, 2018

From zefram@fysh.org

Peter Rabbitson wrote​:

The above statement is false.

It still seems true to me. Perhaps you could explicate what other
advantage the dual-location proposal has, relative to having signatures
after all attributes.

Zefram's argument over the past several months is predicated on the idea that
signatures never existed in a stable form within the Perl5 ecosystem, and
didn't exist at all before 5.22.

That idea is false (at best)

My argument is in no way predicated on signatures not having existed
before 5.22. The idea that they didn't exist certainly is false,
and I'm acutely aware of it, due to my role in getting signatures into
5.20 and in arguing against them being damaged into their 5.22 form.
I am mystified as to how you come to think that I would entertain that
patently false idea.

The idea that signatures have never been stable is true, unless you're
playing games with the wording. The stable releases of Perl that have
included signatures have all included explicit warnings, in documentation
and at compile time, that signatures are experimental. They have thus
never qualified as a stable feature, in our usual parlance. Perhaps you
mean something else by "stable", but in that case you'll have to explain
what you mean in order to further your argument.

My argument doesn't entirely depend on the experimental status of
signatures, though because they do have that status I have couched my
argument in that context. Preceding attributes is the wrong place for
signatures regardless of stability. Signatures are also much newer
than attributes (and the :lvalue attribute in particular) and tacitly
more experimental, regardless of formal status. The relevance of the
formal stability status is only to provide a big explicit bias in the
coherence vs stability tradeoff with which we are faced. It would still
be reasonable to judge in favour of coherence without it.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 26, 2018

From @iabyn

On Thu, Feb 22, 2018 at 10​:27​:18AM +0000, Dave Mitchell wrote​:

I plan to do the following two things shortly (assuming they prove
feasible)​:

1) change the parser so that subs under 'use feature signatures' use
a different grammar rule than subs not under it - possibly by making the
toker return two different sub tokens, e.g. 'SUB' and a new 'SIGSUB'
token say, depending on whether the feature is in scope.

This will then help reduce confusing errors, e.g. this code with a syntax
error (attrs before prototype)​:

no feature 'signatures';
sub f :lvalue \($$@&#8203;\) \{ $x = 1 \}

currently gives​:

Illegal character following sigil in a subroutine signature at
    \.\.\.\, near "\($"

It's parsing the ($$@​) as a sub signature even though signatures aren't in
scope, so the error message is confusing.

2) For the signature sub grammar rule, allow it to spot attributes
following a signature and croak with a meaningful error. Which is what the
OP requested.

I've now done this and pushed as branch smoke-me/davem/sig_attr_croak.
If there's a rough consensus, I'll merge it soon.

The two important commits in that branch are​:

commit 07431a2
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Mon Feb 26 18​:52​:23 2018 +0000
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Mon Feb 26 22​:36​:25 2018 +0000

  detect sub attributes following a signature
 
  RT #132760
 
  A recent commit (v5.27.7-212-g894f226) moved subroutine attributes back
  before the subroutine's signature​: e.g.
 
  sub foo :prototype($$) ($a, $b) { ... } # 5.18 and 5.28 +
  sub foo ($a, $b) :prototype($$) { ... } # 5.20 .. 5.26
 
  This change means that any code still using an attribute following the
  signature is going to trigger a syntax error. However, the error, followed
  by error recovery and further warnings and errors, is very unfriendly and
  gives no indication of the root cause. This commit introduces a new error,
  "Subroutine attributes must come before the signature".
 
  For example, List​::Lazy, the subject of the ticket, failed to compile
  tests, with output like​:
 
  Array found where operator expected at blib/lib/List/Lazy.pm line 43,
  near "$$@​)" (Missing operator before @​)?)
  "my" variable $step masks earlier declaration in same statement at
  blib/lib/List/Lazy.pm line 44.
  syntax error at blib/lib/List/Lazy.pm line 36, near ") :"
  Global symbol "$generator" requires explicit package name (did you
  forget to declare "my $generator"?) at blib/lib/List/Lazy.pm line 38.
  Global symbol "$state" requires explicit package name (did you forget
  to declare "my $state"?) at blib/lib/List/Lazy.pm line 39.
  Global symbol "$min" requires explicit package name (did you forget to
  declare "my $min"?) at blib/lib/List/Lazy.pm line 43.
  Global symbol "$max" requires explicit package name (did you forget to
  declare "my $max"?) at blib/lib/List/Lazy.pm line 43.
  Global symbol "$step" requires explicit package name (did you forget
  to declare "my $step"?) at blib/lib/List/Lazy.pm line 43.
  Invalid separator character '{' in attribute list at
  blib/lib/List/Lazy.pm line 44, near "$step : sub "
  Global symbol "$step" requires explicit package name (did you forget
  to declare "my $step"?) at blib/lib/List/Lazy.pm line 44.
 
  But following this commit, it now just outputs​:
 
  Subroutine attributes must come before the signature at
  blib/lib/List/Lazy.pm line 36.
  Compilation failed in require at t/append.t line 5.
  BEGIN failed--compilation aborted at t/append.t line 5.
 
  It works by​:
 
  1) adding a boolean flag (sig_seen) to the parser state to indicate that a
  signature has been parsed;
  2) at the end of parsing a signature, PL_expect is set to XATTRBLOCK
  rather than XBLOCK.
 
  Then if something looking like one or more attributes is encountered
  by the lexer immediately afterwards, it scans it as if it were an
  attribute, but then if sig_seen is true, it croaks.

commit a47e280
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Thu Feb 22 14​:44​:51 2018 +0000
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Mon Feb 26 22​:36​:05 2018 +0000

  rationalise subroutine parsing rules
 
  Now that the parser rules have been split into separate rules for subs
  under 'use feature "signatures"' and not, refine the rules to reflect the
  different regimes. In particular​:
 
  1) no longer include 'proto' in the signature variants​: as it happens the
  toker would never return a proto THING under signatures anyway, but
  removing it from the grammar makes it clearer what's expected and not
  expected.
 
  2) Remove 'subsignature' from non-sig rules​: what used to happen before
  was that outside of 'use feature "signatures"', it might still try to
  parse a signature, e.g.
 
  $ perl5279 -we 'sub f :lvalue ($$@​) { $x = 1 }'
  Illegal character following sigil in a subroutine signature at -e line
  1, near "($"
  syntax error at -e line 1, near "$$@​"
 
  Now it's just a plain syntax error.

--
"Foul and greedy Dwarf - you have eaten the last candle."
  -- "Hordes of the Things", BBC Radio.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 27, 2018

From @jkeenan

On Mon, 26 Feb 2018 23​:48​:20 GMT, davem wrote​:

On Thu, Feb 22, 2018 at 10​:27​:18AM +0000, Dave Mitchell wrote​:

I plan to do the following two things shortly (assuming they prove
feasible)​:

1) change the parser so that subs under 'use feature signatures' use
a different grammar rule than subs not under it - possibly by making
the
toker return two different sub tokens, e.g. 'SUB' and a new 'SIGSUB'
token say, depending on whether the feature is in scope.

This will then help reduce confusing errors, e.g. this code with a
syntax
error (attrs before prototype)​:

no feature 'signatures';
sub f :lvalue ($$@​) { $x = 1 }

currently gives​:

Illegal character following sigil in a subroutine signature at
..., near "($"

It's parsing the ($$@​) as a sub signature even though signatures
aren't in
scope, so the error message is confusing.

2) For the signature sub grammar rule, allow it to spot attributes
following a signature and croak with a meaningful error. Which is
what the
OP requested.

I've now done this and pushed as branch smoke-me/davem/sig_attr_croak.
If there's a rough consensus, I'll merge it soon.

I built this branch on FreeBSD-11.0, then installed 'cpanm' against it and used cpanm to try to install the three modules cited in this ticket​: List​::Lazy, Evo, and Moxie. Tests failed in all three with this kind of message​:

#####
Subroutine attributes must come before the signature at /usr/home/jkeenan/.cpanm/work/1519692082.14147/Moxie-0.07/blib/lib/Moxie/Traits/Provider/Accessor.pm line 19.
#####

Output of perl -V and gzipped build.log attached. Should I have expected better results?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 27, 2018

From @jkeenan

Summary of my perl5 (revision 5 version 27 subversion 9) configuration​:
  Commit id​: 07431a2
  Platform​:
  osname=freebsd
  osvers=11.0-release-p1
  archname=amd64-freebsd
  uname='freebsd kid51-freebsd-11-install-3 11.0-release-p1 freebsd 11.0-release-p1 #0 r306420​: thu sep 29 01​:43​:23 utc 2016 root@​releng2.nyi.freebsd.org​:usrobjusrsrcsysgeneric amd64 '
  config_args='-des -Dusedevel -Uversiononly -Dprefix=/home/jkeenan/testing/smoke-me/davem/sig_attr_croak -Dman1dir=none -Dman3dir=none'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
  optimize='-O2'
  cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.2.1 Compatible FreeBSD Clang 3.8.0 (tags/RELEASE_380/final 262564)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags ='-Wl,-E -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib /usr/bin/../lib/clang/3.8.0/lib /usr/lib
  libs=-lpthread -lgdbm -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lm -lcrypt -lutil -lc
  libc=
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags=' '
  cccdlflags='-DPIC -fPIC'
  lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  Built under freebsd
  Compiled at Feb 26 2018 19​:37​:55
  %ENV​:
  PERL2DIR="/home/jkeenan/gitwork/perl2"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /home/jkeenan/testing/smoke-me/davem/sig_attr_croak/lib/perl5/site_perl/5.27.9/amd64-freebsd
  /home/jkeenan/testing/smoke-me/davem/sig_attr_croak/lib/perl5/site_perl/5.27.9
  /home/jkeenan/testing/smoke-me/davem/sig_attr_croak/lib/perl5/5.27.9/amd64-freebsd
  /home/jkeenan/testing/smoke-me/davem/sig_attr_croak/lib/perl5/5.27.9

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 27, 2018

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 27, 2018

From @karenetheridge

On Thu, Feb 22, 2018 at 9​:18 AM, Zefram <zefram@​fysh.org> wrote​:

The idea that signatures have never been stable is true, unless you're
playing games with the wording. The stable releases of Perl that have
included signatures have all included explicit warnings, in documentation
and at compile time, that signatures are experimental. They have thus
never qualified as a stable feature, in our usual parlance. Perhaps you
mean something else by "stable", but in that case you'll have to explain
what you mean in order to further your argument.

​Well, there's https://metacpan.org/release/signatures, which has had a
stable API for a decade, but I don't know if this is what Peter is thinking
of.​

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 27, 2018

From @iabyn

On Mon, Feb 26, 2018 at 05​:47​:04PM -0800, James E Keenan via RT wrote​:

I built this branch on FreeBSD-11.0, then installed 'cpanm' against it and used cpanm to try to install the three modules cited in this ticket​: List​::Lazy, Evo, and Moxie. Tests failed in all three with this kind of message​:

#####
Subroutine attributes must come before the signature at /usr/home/jkeenan/.cpanm/work/1519692082.14147/Moxie-0.07/blib/lib/Moxie/Traits/Provider/Accessor.pm line 19.
#####

Output of perl -V and gzipped build.log attached. Should I have expected better results?

My branch addresses the main complaint of the OP, that the error message
gave no indication of what the issue is. With my branch, all those modules
are now getting a clear and unambiguous error message, which if looked up
in perldiag, explains the shift in attribute position for 5.28.

So, no, you shouldn't have expected better results.

--
Hofstadter's Law​: It always takes longer than you expect, even when you
take into account Hofstadter's Law.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 27, 2018

From @ilmari

Dave Mitchell <davem@​iabyn.com> writes​:

On Thu, Feb 22, 2018 at 10​:27​:18AM +0000, Dave Mitchell wrote​:

I plan to do the following two things shortly (assuming they prove
feasible)​:

1) change the parser so that subs under 'use feature signatures' use
a different grammar rule than subs not under it - possibly by making the
toker return two different sub tokens, e.g. 'SUB' and a new 'SIGSUB'
token say, depending on whether the feature is in scope.

This will then help reduce confusing errors, e.g. this code with a syntax
error (attrs before prototype)​:

no feature 'signatures';
sub f :lvalue \($$@&#8203;\) \{ $x = 1 \}

currently gives​:

Illegal character following sigil in a subroutine signature at
    \.\.\.\, near "\($"

It's parsing the ($$@​) as a sub signature even though signatures aren't in
scope, so the error message is confusing.

2) For the signature sub grammar rule, allow it to spot attributes
following a signature and croak with a meaningful error. Which is what the
OP requested.

I've now done this and pushed as branch smoke-me/davem/sig_attr_croak.
If there's a rough consensus, I'll merge it soon.

LGTM

- ilmari
--
- Twitter seems more influential [than blogs] in the 'gets reported in
  the mainstream press' sense at least. - Matt McLeod
- That'd be because the content of a tweet is easier to condense down
  to a mainstream media article. - Calle Dybedahl

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 27, 2018

From @cpansprout

On Tue, 27 Feb 2018 03​:49​:59 -0800, ilmari wrote​:

Dave Mitchell <davem@​iabyn.com> writes​:

I've now done this and pushed as branch smoke-
me/davem/sig_attr_croak.
If there's a rough consensus, I'll merge it soon.

LGTM

Same here.

(LGTM = Laughing goats trot merrily.)

--

Father Chrysostomos

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 2, 2018

From @iabyn

Dave Mitchell <davem@​iabyn.com> writes​:

I've now done this and pushed as branch smoke-
me/davem/sig_attr_croak.
If there's a rough consensus, I'll merge it soon.

Now merged with v5.27.8-432-gbe68a30

--
The Enterprise's efficient long-range scanners detect a temporal vortex
distortion in good time, allowing it to be safely avoided via a minor
course correction.
  -- Things That Never Happen in "Star Trek" #21

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 5, 2018

@iabyn - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed Mar 5, 2018
@p5pRT
Copy link
Author

@p5pRT p5pRT commented Mar 30, 2018

From zefram@fysh.org

The perl version numbers in the perldiag entry were slightly wrong.
Fixed in commit 3b98040.

-zefram

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 5, 2018

From @jkeenan

On Tue, 27 Feb 2018 06​:54​:17 GMT, davem wrote​:

On Mon, Feb 26, 2018 at 05​:47​:04PM -0800, James E Keenan via RT wrote​:

I built this branch on FreeBSD-11.0, then installed 'cpanm' against
it and used cpanm to try to install the three modules cited in this
ticket​: List​::Lazy, Evo, and Moxie. Tests failed in all three with
this kind of message​:

#####
Subroutine attributes must come before the signature at
/usr/home/jkeenan/.cpanm/work/1519692082.14147/Moxie-
0.07/blib/lib/Moxie/Traits/Provider/Accessor.pm line 19.
#####

Output of perl -V and gzipped build.log attached. Should I have
expected better results?

My branch addresses the main complaint of the OP, that the error
message
gave no indication of what the issue is. With my branch, all those
modules
are now getting a clear and unambiguous error message, which if looked
up
in perldiag, explains the shift in attribute position for 5.28.

So, no, you shouldn't have expected better results.

YANICK has uploaded List-Lazy-0.3.1, which now PASSes on FreeBSD-11.0.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 21, 2018

From darren@DarrenDuncan.net

For the record, I strongly support the idea that things which don't need to be
in a particular sequence to be easily understood should not be required to be in
a particular sequence.

Signatures and attributes should be declarable in either sequence, either one
before or after the other. Its really not that difficult for users to
understand that either order works.

Even if something is officially experimental, that doesn't mean there isn't
still a high bar for breaking compatibility between Perl versions, and I don't
believe that switching the mandatory order here qualifies.

-- Darren Duncan

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 22, 2018

From @xsawyerx

On 04/21/2018 09​:45 PM, Darren Duncan wrote​:

For the record, I strongly support the idea that things which don't
need to be in a particular sequence to be easily understood should not
be required to be in a particular sequence.

Signatures and attributes should be declarable in either sequence,
either one before or after the other.  Its really not that difficult
for users to understand that either order works.

What are you determining this on?

Even if something is officially experimental, that doesn't mean there
isn't still a high bar for breaking compatibility between Perl
versions, and I don't believe that switching the mandatory order here
qualifies.

The change was to revert breaking compatibility. I see people keep
getting this confused.

Keeping subroutine signatures where they were would break :lvalue
subroutine attribute, which arrived far before subroutine signatures.
This *is* about backwards compatibility.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented May 2, 2018

From @Leont

On Sun, Apr 22, 2018 at 11​:25 AM, Sawyer X <xsawyerx@​gmail.com> wrote​:

Even if something is officially experimental, that doesn't mean there
isn't still a high bar for breaking compatibility between Perl
versions, and I don't believe that switching the mandatory order here
qualifies.

The change was to revert breaking compatibility. I see people keep
getting this confused.

Keeping subroutine signatures where they were would break :lvalue
subroutine attribute, which arrived far before subroutine signatures.
This *is* about backwards compatibility.

Either I'm not understanding you, or you are not understanding him. I
don't think anyone is arguing against the necessity of change. He's
arguing for the other possible solution because it's more compatible
with both recent perl versions and with signature implementations on
CPAN.

Leon

@p5pRT
Copy link
Author

@p5pRT p5pRT commented May 2, 2018

From @iabyn

On Wed, May 02, 2018 at 08​:50​:45AM +0200, Leon Timmermans wrote​:

Either I'm not understanding you, or you are not understanding him. I
don't think anyone is arguing against the necessity of change. He's
arguing for the other possible solution because it's more compatible
with both recent perl versions and with signature implementations on
CPAN.

I'm not aware of "the other possible solution". What is that?

--
"Emacs isn't a bad OS once you get used to it.
It just lacks a decent editor."

@p5pRT
Copy link
Author

@p5pRT p5pRT commented May 2, 2018

From @xsawyerx

On 05/02/2018 09​:50 AM, Leon Timmermans wrote​:

On Sun, Apr 22, 2018 at 11​:25 AM, Sawyer X <xsawyerx@​gmail.com> wrote​:

Even if something is officially experimental, that doesn't mean there
isn't still a high bar for breaking compatibility between Perl
versions, and I don't believe that switching the mandatory order here
qualifies.
The change was to revert breaking compatibility. I see people keep
getting this confused.

Keeping subroutine signatures where they were would break :lvalue
subroutine attribute, which arrived far before subroutine signatures.
This *is* about backwards compatibility.
Either I'm not understanding you, or you are not understanding him. I
don't think anyone is arguing against the necessity of change. He's
arguing for the other possible solution because it's more compatible
with both recent perl versions and with signature implementations on
CPAN.

I did understand this is an argument for the option of changing the
signatures to be on both sides. Perhaps I was not clear on my response.
My comment was in response to this sentence fragment​:

  Even if something is officially experimental, that doesn't mean
  there isn't still a high bar for breaking compatibility between Perl
  versions [...]

which I understood as "Don't break signatures even though they're
experimental," meaning we should avoid breaking compatibility. My
comment was to say we *do* care about compatibility, there is a high
bar, and that this was *the* reason we decided to revert the order,
*because* of compatibility. (Not of signatures but of :lvalue which was
available much earlier than signatures, so it takes precedence.) Darren
expressed the need for compatibility and I agreed, but with something
else, not signatures.

I should have added that we did raise the possibility of putting
signatures on both sides. (In fact, *I* raised it when looking for
comments on it.) However, that did not receive backing and there was
expressed disinterest in doing this.

I apologize for the confusion caused by my response.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented May 24, 2018

From @eserte

Dana Fri, 09 Feb 2018 14​:21​:58 -0800, slaven@​rezic.de reče​:

Dana Wed, 24 Jan 2018 12​:04​:24 -0800, slaven@​rezic.de reče​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.27.8.

-----------------------------------------------------------------
List-Lazy-0.3.0 does not work anymore since v5.27.7-212-g894f226e51.
This is the change which swapped signatures and attributes.
This change is already documented in perldelta, but I have just
two notes​:

- The error output does not look nice. In this case it begins with

Array found where operator expected at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 43, near "$$@​)"
(Missing operator before @​)?)
"my" variable $step masks earlier declaration in same statement at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 44.
syntax error at /home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 36, near ") :"
Global symbol "$generator" requires explicit package name (did you
forget to declare "my $generator"?) at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 38.
Global symbol "$state" requires explicit package name (did you forget
to declare "my $state"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 39.
Global symbol "$min" requires explicit package name (did you forget to
declare "my $min"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-
0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$max" requires explicit package name (did you forget to
declare "my $max"?) at /home/eserte/.cpan/build/2018012418/List-Lazy-
0.3.0-2/blib/lib/List/Lazy.pm line 43.
Global symbol "$step" requires explicit package name (did you forget
to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 43.
Invalid separator character '{' in attribute list at
/home/eserte/.cpan/build/2018012418/List-Lazy-0.3.0-
2/blib/lib/List/Lazy.pm line 44, near "$step : sub "
Global symbol "$step" requires explicit package name (did you forget
to declare "my $step"?) at /home/eserte/.cpan/build/2018012418/List-
Lazy-0.3.0-2/blib/lib/List/Lazy.pm line 44.
...

which does not really say anything about the problem. Is it possible
to detect this situation and improve diagnostics?

- What does this change means in terms of usability of signatures?
Users
mixing signatures and prototypes must increase their perl
prerequisite
from 5.22 to 5.28, which may mean it could be less likely that
signatures are used in the next time. Is this worth for this change?

BTW, the issue for the CPAN module is
yanick/List-Lazy#3

-- Slaven

Also affected​: ALEXBYK/Evo-0.0405.tar.gz
(Bisect result found out by Andreas)

This ticket was closed, but at least this module (and possibly also others mentioned here) was not handled.
I created an issue in the module's issue tracker​: alexbyk/perl-evo#6

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 4, 2018

From @iabyn

On Wed, May 23, 2018 at 11​:30​:15PM -0700, slaven@​rezic.de via RT wrote​:

This ticket was closed, but at least this module (and possibly also others mentioned here) was not handled.
I created an issue in the module's issue tracker​: alexbyk/perl-evo#6

All affected distributions mentioned in this ticket now either pass on
blead or have the following tickets​:

  Evo-0.0405
  alexbyk/perl-evo#6
  Moxie-0.07
  stevan/p5-Moxie#9

--
I thought I was wrong once, but I was mistaken.

@p5pRT p5pRT added BBC Severity High distro-All labels Oct 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BBC distro-All
Projects
None yet
Development

No branches or pull requests

1 participant