Feature request: builtin array holding ($1,$2,$3,...)

[p5pRT]

Migrated from rt.perl.org#120521 (status was 'resolved')

Searchable as RT120521$

[p5pRT]

From @epa

Created by @epa

It would be useful to have a builtin array variable which holds the
values captured by the last successful regexp match. While you can
use m// in list context to return a list of matches, this doesn't help
so much when you want to get an array of all matches on the RHS of a
s///.

Suppose the builtin array is called @​CAPTURE. Then

  $_ = 'abc';
  /(.)(.)(.)/ or die;
  say scalar @​CAPTURE; # prints 3
  say $CAPTURE[0]; # prints a

There is a fair bit of boilerplate code with ($1,$2,$3,$4,$5) up to
some arbitrary limit, which would be made simpler and correct if there
were a way to get all captures as an array.

Perl Info

Flags:
    category=core
    severity=wishlist

Site configuration information for perl 5.16.3:

Configured by Red Hat, Inc. at Fri May  3 12:10:03 UTC 2013.

Summary of my perl5 (revision 5 version 16 subversion 3) configuration:
   
  Platform:
    osname=linux, osvers=2.6.32-358.2.1.el6.x86_64, archname=x86_64-linux-thread-multi
    uname='linux buildvm-24.phx2.fedoraproject.org 2.6.32-358.2.1.el6.x86_64 #1 smp wed feb 20 12:17:37 est 2013 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4  -m64 -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4  -m64 -mtune=generic -Wl,-z,relro  -DDEBUGGING=-g -Dversion=5.16.3 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.7.2 20121109 (Red Hat 4.7.2-8)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector'
    libpth=/usr/local/lib64 /lib64 /usr/lib64
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.16'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags -Wl,-rpath,/usr/lib64/perl5/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro '

Locally applied patches:
    


@INC for perl 5.16.3:
    /home/eda/lib/perl5/
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .


Environment for perl 5.16.3:
    HOME=/home/eda
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=C
    LC_CTYPE=en_GB.UTF-8
    LC_MESSAGES=en_GB.UTF-8
    LC_MONETARY=en_GB.UTF-8
    LC_NUMERIC=en_GB.UTF-8
    LC_TIME=en_GB.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/eda/bin:/home/eda/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/sbin:/usr/sbin
    PERL5LIB=/home/eda/lib/perl5/
    PERL_BADLANG (unset)
    SHELL=/bin/bash

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

[p5pRT]

From @Hugmeir

On Tue, Nov 12, 2013 at 2​:26 PM, Ed Avis <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by "Ed Avis"
# Please include the string​: [perl #120521]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120521 >

This is a bug report for perl from eda@​waniasset.com,
generated with the help of perlbug 1.39 running under perl 5.16.3.

-----------------------------------------------------------------
[Please describe your issue here]

It would be useful to have a builtin array variable which holds the
values captured by the last successful regexp match. While you can
use m// in list context to return a list of matches, this doesn't help
so much when you want to get an array of all matches on the RHS of a
s///.

Suppose the builtin array is called @​CAPTURE. Then

$\_ = 'abc';
/\(\.\)\(\.\)\(\.\)/ or die;
say scalar @​CAPTURE; \# prints 3
say $CAPTURE\[0\];     \# prints a

There is a fair bit of boilerplate code with ($1,$2,$3,$4,$5) up to
some arbitrary limit, which would be made simpler and correct if there
were a way to get all captures as an array.

Eh, this looks like a feature better suited for CPAN. You can already
implement this in just a few lines​:

package Capture {
  use Tie​::Array;
  our @​ISA = qw(Tie​::StdArray);
  sub FETCH {
  ${*{$_[1]+1}{SCALAR}}
  }
  sub FETCHSIZE {
  scalar(@​-) - 1
  }
}

tie @​CAPTURE, "Capture";

Although a proper implementation could avoid ties, I imagine. Also, rather
than @​CAPTURE, you could take @​{^CAPTURE}, and make it available everywhere.

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @demerphq

On 12 November 2013 20​:11, Brian Fraser <fraserbn@​gmail.com> wrote​:

On Tue, Nov 12, 2013 at 2​:26 PM, Ed Avis <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by "Ed Avis"
# Please include the string​: [perl #120521]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120521 >

This is a bug report for perl from eda@​waniasset.com,
generated with the help of perlbug 1.39 running under perl 5.16.3.

-----------------------------------------------------------------
[Please describe your issue here]

It would be useful to have a builtin array variable which holds the
values captured by the last successful regexp match. While you can
use m// in list context to return a list of matches, this doesn't help
so much when you want to get an array of all matches on the RHS of a
s///.

Suppose the builtin array is called @​CAPTURE. Then

$\_ = 'abc';
/\(\.\)\(\.\)\(\.\)/ or die;
say scalar @​CAPTURE; \# prints 3
say $CAPTURE\[0\];     \# prints a

There is a fair bit of boilerplate code with ($1,$2,$3,$4,$5) up to
some arbitrary limit, which would be made simpler and correct if there
were a way to get all captures as an array.

Eh, this looks like a feature better suited for CPAN. You can already
implement this in just a few lines​:

package Capture {
use Tie​::Array;
our @​ISA = qw(Tie​::StdArray);
sub FETCH {
${*{$_[1]+1}{SCALAR}}

Ugh!

\}
sub FETCHSIZE \{
    scalar\(@​\-\) \- 1
\}

}

tie @​CAPTURE, "Capture";

Although a proper implementation could avoid ties, I imagine. Also, rather
than @​CAPTURE, you could take @​{^CAPTURE}, and make it available everywhere.

$1 is a tie itself. So @​{^CAPTURE} would be a tie as well. The
difference between a core @​CAPTURE and a non-core one is that the
non-core one would be a tie which accesses a second tie.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

[p5pRT]

From @demerphq

On 12 November 2013 18​:26, Ed Avis <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by "Ed Avis"
# Please include the string​: [perl #120521]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=120521 >

This is a bug report for perl from eda@​waniasset.com,
generated with the help of perlbug 1.39 running under perl 5.16.3.

-----------------------------------------------------------------
[Please describe your issue here]

It would be useful to have a builtin array variable which holds the
values captured by the last successful regexp match. While you can
use m// in list context to return a list of matches, this doesn't help
so much when you want to get an array of all matches on the RHS of a
s///.

Suppose the builtin array is called @​CAPTURE. Then

$\_ = 'abc';
/\(\.\)\(\.\)\(\.\)/ or die;
say scalar @​CAPTURE; \# prints 3
say $CAPTURE\[0\];     \# prints a

There is a fair bit of boilerplate code with ($1,$2,$3,$4,$5) up to
some arbitrary limit, which would be made simpler and correct if there
were a way to get all captures as an array.

But there is​:

  @​CAPTURE= /(.)(.)(.)/ or die;

IOW, in scalar context a regex match returns a list of items captured,
which can be fed into an array.

Why would having a magic var for this be better?

cheers,
Yves

[p5pRT]

From @epa

Yves Orton wrote​:

@​CAPTURE= /(.)(.)(.)/ or die;

Why would having a magic var for this be better?

I mentioned one reason in the original report​: in s/// you don't have access to this.
You have to do s/$re/foo($1,$2,$3,$4,$5...)/

Another reason is that m// in list context makes it hard to distinguish between a
match which failed, and one which succeeded but did not have capturing groups.

  $_ = 'abc';
  my @​CAPTURE = /abc/;
  say scalar @​CAPTURE;
  @​CAPTURE = /(abc)/;
  say scalar @​CAPTURE;

Both print '1'. So, if you have some code like

  if (/$re/) { do_something($1, $2, $3, $4, $5, ...) }

there is not an obvious way to turn it into the form you suggest​:

  @​CAPTURE = /$re/;
  # um, did the regexp succeed or not?

I am thinking of regular expressions which are specified in a configuration table,
or generated at run time. So the number of capturing groups in the regexp is
not known in advance.

But if @​CAPTURE were available as a builtin, you could simply say

  if (/$re/) { do_something(@​CAPTURE) }

--
Ed Avis <eda@​waniasset.com>

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http​://www.symanteccloud.com
______________________________________________________________________

[p5pRT]

From @epa

I would add that in general the reason to prefer an array @​CAPTURE over individual $1, $2, $3
is the same reason why we usually work with arrays rather than a long list of variables.
Beginners to programming will sometimes write code with $one, $two, $three, $four and so on.

Similarly, think of @​ARGV. We could do without that and instead have $ARG1, $ARG2, $ARG3 etc.
But it is neater and cleaner to provide an array interface.

In the case of @​CAPTURE, the array would have to be read-only (so you can't shift it, etc) but
still you can work with it using the normal idioms for manipulating arrays and lists in Perl.

Has nobody else here ever written code like foo($1, $2, $3, $4, $5)? And never wished there were
a more simple and obvious way to express it, that won't have an arbitrary limit on the number
of captures?

--
Ed Avis <eda@​waniasset.com>

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http​://www.symanteccloud.com
______________________________________________________________________

[p5pRT]

From @emazep

On 2013-11-15 11​:55, Ed Avis wrote​:

Another reason is that m// in list context makes it hard to distinguish between a
match which failed, and one which succeeded but did not have capturing groups.

$\_ = 'abc';
my @​CAPTURE = /abc/;
say scalar @​CAPTURE;
@​CAPTURE = /\(abc\)/;
say scalar @​CAPTURE;

Both print '1'. So, if you have some code like

if \(/$re/\) \{ do\_something\($1\, $2\, $3\, $4\, $5\, \.\.\.\) \}

there is not an obvious way to turn it into the form you suggest​:

@​CAPTURE = /$re/;
\# um\, did the regexp succeed or not?

I am thinking of regular expressions which are specified in a configuration table,
or generated at run time. So the number of capturing groups in the regexp is
not known in advance.

But if @​CAPTURE were available as a builtin, you could simply say

if \(/$re/\) \{ do\_something\(@​CAPTURE\) \}

You can currently do​:

if ( my @​CAPTURE =()= /$re/ ) { say @​CAPTURE }

P.S.

Shouldn't be @​CAPTURE spelled @​CAPTURES?

-Emanuele

[p5pRT]

From @demerphq

On 15 November 2013 13​:02, Emanuele Zeppieri <emazep@​gmail.com> wrote​:

On 2013-11-15 11​:55, Ed Avis wrote​:

Another reason is that m// in list context makes it hard to distinguish between a
match which failed, and one which succeeded but did not have capturing groups.

$\_ = 'abc';
my @​CAPTURE = /abc/;
say scalar @​CAPTURE;
@​CAPTURE = /\(abc\)/;
say scalar @​CAPTURE;

Both print '1'. So, if you have some code like

if \(/$re/\) \{ do\_something\($1\, $2\, $3\, $4\, $5\, \.\.\.\) \}

there is not an obvious way to turn it into the form you suggest​:

@​CAPTURE = /$re/;
\# um\, did the regexp succeed or not?

I am thinking of regular expressions which are specified in a configuration table,
or generated at run time. So the number of capturing groups in the regexp is
not known in advance.

But if @​CAPTURE were available as a builtin, you could simply say

if \(/$re/\) \{ do\_something\(@​CAPTURE\) \}

You can currently do​:

if ( my @​CAPTURE =()= /$re/ ) { say @​CAPTURE }

P.S.

Shouldn't be @​CAPTURE spelled @​CAPTURES?

Style preference. I tend to prefer to use singular for arrays where I
tend to access sub elements more than the whole list​:

if ($CAPTURE[1]) { }

reads better to me than

if ($CAPTURES[1]) { }

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

[p5pRT]

From @demerphq

On 15 November 2013 11​:55, Ed Avis <eda@​waniasset.com> wrote​:

Yves Orton wrote​:

@​CAPTURE= /(.)(.)(.)/ or die;

Why would having a magic var for this be better?

I mentioned one reason in the original report​: in s/// you don't have access to this.
You have to do s/$re/foo($1,$2,$3,$4,$5...)/

Another reason is that m// in list context makes it hard to distinguish between a
match which failed, and one which succeeded but did not have capturing groups.

$\_ = 'abc';
my @​CAPTURE = /abc/;
say scalar @​CAPTURE;
@​CAPTURE = /\(abc\)/;
say scalar @​CAPTURE;

Both print '1'. So, if you have some code like

if \(/$re/\) \{ do\_something\($1\, $2\, $3\, $4\, $5\, \.\.\.\) \}

there is not an obvious way to turn it into the form you suggest​:

@​CAPTURE = /$re/;
\# um\, did the regexp succeed or not?

I am thinking of regular expressions which are specified in a configuration table,
or generated at run time. So the number of capturing groups in the regexp is
not known in advance.

But if @​CAPTURE were available as a builtin, you could simply say

if \(/$re/\) \{ do\_something\(@​CAPTURE\) \}

Color me convinced. :-)

If FC doesnt beat me to it (as he so often does) I will add support for this.

Let the debate begin about the name. I wont participate except to note
that there is a bit of a lack of consistency if we have

@​+
@​-
%+
%-

and then @​CAPTURE(S) as well. I leave that for other to agonize over.
Ever since mauve I refuse to play the bike-shed game.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

[p5pRT]

From @epa

Emanuele Zeppieri wrote​:

if ( my @​CAPTURE =()= /$re/ ) { say @​CAPTURE }

I don't think that works​:

  $_ = 'abc';
  my $re = qr/abc/;
  if ( my @​CAPTURE =()= /$re/ ) { say @​CAPTURE } else { say 'no' }
  if (/$re/) { say 'simple match' }

For me this prints

no
simple match

--
Ed Avis <eda@​waniasset.com>

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http​://www.symanteccloud.com
______________________________________________________________________

[p5pRT]

From @epa

Thank you Yves for offering to implement this. You can call it @​BIKESHED if you want, and I'll still be delighted...

--
Ed Avis <eda@​waniasset.com>

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http​://www.symanteccloud.com
______________________________________________________________________

[p5pRT]

From @Leont

On Fri, Nov 15, 2013 at 11​:19 AM, demerphq <demerphq@​gmail.com> wrote​:

Eh, this looks like a feature better suited for CPAN. You can already
implement this in just a few lines​:

package Capture {
use Tie​::Array;
our @​ISA = qw(Tie​::StdArray);
sub FETCH {
${*{$_[1]+1}{SCALAR}}

Ugh!

\}
sub FETCHSIZE \{
    scalar\(@​\-\) \- 1
\}

}

tie @​CAPTURE, "Capture";

Although a proper implementation could avoid ties, I imagine. Also,
rather
than @​CAPTURE, you could take @​{^CAPTURE}, and make it available
everywhere.

$1 is a tie itself. So @​{^CAPTURE} would be a tie as well. The
difference between a core @​CAPTURE and a non-core one is that the
non-core one would be a tie which accesses a second tie.

Now if only we had array vtables, we could have implemented this in C
without having to use ties :-/.

Leon

[p5pRT]

From @demerphq

The = () = is superfluous.

Yves

On 15 November 2013 13​:08, Ed Avis <eda@​waniasset.com> wrote​:

Emanuele Zeppieri wrote​:

if ( my @​CAPTURE =()= /$re/ ) { say @​CAPTURE }

I don't think that works​:

$\_ = 'abc';
my $re = qr/abc/;
if \( my @​CAPTURE =\(\)= /$re/ \) \{ say @​CAPTURE \} else \{ say 'no' \}
if \(/$re/\) \{ say 'simple match' \}

For me this prints

no
simple match

--
Ed Avis <eda@​waniasset.com>

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http​://www.symanteccloud.com
______________________________________________________________________

--
perl -Mre=debug -e "/just|another|perl|hacker/"

[p5pRT]

From @hvds

Ed Avis <eda@​waniasset.com> wrote​:
:I would add that in general the reason to prefer an array @​CAPTURE over individual $1, $2, $3
:is the same reason why we usually work with arrays rather than a long list of variables.
:Beginners to programming will sometimes write code with $one, $two, $three, $four and so on.
:
:Similarly, think of @​ARGV. We could do without that and instead have $ARG1, $ARG2, $ARG3 etc.
:But it is neater and cleaner to provide an array interface.
:
:In the case of @​CAPTURE, the array would have to be read-only (so you can't shift it, etc) but
:still you can work with it using the normal idioms for manipulating arrays and lists in Perl.
:
:Has nobody else here ever written code like foo($1, $2, $3, $4, $5)? And never wished there were
:a more simple and obvious way to express it, that won't have an arbitrary limit on the number
:of captures?

I haven't, but I have frequently written code that makes the relevant array
using @​+ and @​-; I've always found that perfectly adequate so haven't felt
a need for yet another magic regexp variable.

I'd rather see this prototyped on CPAN first, if only because IMO we have
a definite habit of failing to get the semantics tied down in ideal form
first time round whenever we add something new.

But I also feel that while the magic variable approach maybe made sense
when there were hardly any, we've moved incrementally far beyond sanity​:
I would rather see us moving in the direction of a capture object that
you can ask all this sort of thing of via method calls, like​:

  # assuming a new //mauve option that means "give me an object"
  if ($match = /(r)(e)/mauve) {
  print "Saw <$_>" for $match->captures;
  }

(Sorry I haven't really been following this discussion, I'm in the middle
of moving house, and will shortly be losing internet access for a few days.)

Hugo