Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: builtin to apply double-quote interpolation to a scalar #16592

Open
p5pRT opened this issue Jun 21, 2018 · 7 comments
Open

Feature: builtin to apply double-quote interpolation to a scalar #16592

p5pRT opened this issue Jun 21, 2018 · 7 comments

Comments

@p5pRT
Copy link
Collaborator

@p5pRT p5pRT commented Jun 21, 2018

Migrated from rt.perl.org#133292 (status was 'open')

Searchable as RT133292$

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 21, 2018

From @epa

Created by @epa

Sometimes you might want to make a template string with variables, and
interpolate them later.

my $template = 'hello $name';
foreach my $name (qw(Fred Jim)) {
  my $s = eval '"' . $template . '"';
  die $@​ if $@​;
  say $s;
}

That trick with eval is awkward and unhygienic (what if $template
contains a " character?). You could instead do some s///e magic to
find variable names and interpolate them, but that too is error-prone
and hard to generalize to all the things that doublequote
interpolation supports.

Presumably the perl interpreter has an internal routine that performs
double-quote interpolation on a string. That should be exposed as a
builtin function, so you can say

  my $s = interpolate($template);

and get broadly the same semantics as the eval trick above, but
without having to rely on a particular delimiter character that
doesn't appear in the string (or the string not ending in backslash),
and throwing errors rather than leaving them in $@​.

I know there are many excellent templating systems out there. This is
for much more basic uses, particularly those where you started off
with a doublequoted string and then wanted to separate the definition
of that bit of code from where it is used.

Perl Info

Flags:
    category=core
    severity=wishlist

Site configuration information for perl 5.22.2:

Configured by Red Hat, Inc. at Fri Nov  4 14:35:02 UTC 2016.

Summary of my perl5 (revision 5 version 22 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=4.7.9-200.fc24.x86_64, archname=x86_64-linux-thread-multi
    uname='linux buildvm-12.phx2.fedoraproject.org 4.7.9-200.fc24.x86_64 #1 smp thu oct 20 14:26:16 utc 2016 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=none -Dccflags=-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mtune=generic -Dldflags=-Wl,-z,relro  -Dccdlflags=-Wl,--enable-new-dtags -Wl,-z,relro  -Dlddlflags=-shared -Wl,-z,relro  -Dshrpdir=/usr/lib64 -DDEBUGGING=-g -Dversion=5.22.2 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fwrapv -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='  -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fwrapv -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='5.3.1 20160406 (Red Hat 5.3.1-6)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags ='-Wl,-z,relro  -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib64 /lib64 /usr/lib64 /usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib
    libs=-lpthread -lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lresolv -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.22.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.22'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags -Wl,-z,relro '
    cccdlflags='-fPIC', lddlflags='-shared -Wl,-z,relro  -L/usr/local/lib -fstack-protector-strong'

Locally applied patches:
    Fedora Patch1: Removes date check, Fedora/RHEL specific
    Fedora Patch3: support for libdir64
    Fedora Patch4: use libresolv instead of libbind
    Fedora Patch5: USE_MM_LD_RUN_PATH
    Fedora Patch6: Skip hostname tests, due to builders not being network capable
    Fedora Patch7: Dont run one io test due to random builder failures
    Fedora Patch15: Define SONAME for libperl.so
    Fedora Patch16: Install libperl.so to -Dshrpdir value
    Fedora Patch22: Document Math::BigInt::CalcEmu requires Math::BigInt (CPAN RT#85015)
    Fedora Patch26: Make *DBM_File desctructors thread-safe (RT#61912)
    Fedora Patch27: Make PadlistNAMES() lvalue again (CPAN RT#101063)
    Fedora Patch28: Make magic vtable writable as a work-around for Coro (CPAN RT#101063)
    Fedora Patch29: Fix duplicating PerlIO::encoding when spawning threads (RT#31923)
    Fedora Patch30: Do not let XSLoader load relative paths (CVE-2016-6185)
    Fedora Patch31: Avoid loading optional modules from default . (CVE-2016-1238)
    Fedora Patch32: Fix a crash in lexical scope warnings (RT#128597)
    Fedora Patch33: Do not mangle errno from failed socket calls (RT#128316)
    Fedora Patch34: Fix crash in "evalbytes S" (RT#129196)
    Fedora Patch35: Fix crash in "evalbytes S" (RT#129196)
    Fedora Patch36: Fix crash in "evalbytes S" (RT#129196)
    Fedora Patch37: Fix crash in splice (RT#129164, RT#129166, RT#129167)
    Fedora Patch38: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch39: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch40: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch41: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch42: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch43: Fix crash when matching UTF-8 string with non-UTF-8 substrings (RT#129350)
    Fedora Patch44: Fix parsing perl options in shell bang line (RT#129336)
    Fedora Patch45: Fix firstchar bitmap under UTF-8 with prefix optimization (RT#129950)
    Fedora Patch46: Avoid infinite loop in h2xs tool if enum and type have the same name (RT130001)
    Fedora Patch47: Fix stack handling when calling chdir without an argument (RT#129130)
    Fedora Patch200: Link XS modules to libperl.so with EU::CBuilder on Linux
    Fedora Patch201: Link XS modules to libperl.so with EU::MM on Linux


@INC for perl 5.22.2:
    /home/eda/lib64/perl5/
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5


Environment for perl 5.22.2:
    HOME=/home/eda
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=C
    LC_CTYPE=en_GB.UTF-8
    LC_MESSAGES=en_GB.UTF-8
    LC_MONETARY=en_GB.UTF-8
    LC_NUMERIC=en_GB.UTF-8
    LC_TIME=en_GB.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/eda/bin:/home/eda/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/sbin:/usr/sbin
    PERL5LIB=/home/eda/lib64/perl5/
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 25, 2018

From @iabyn

On Thu, Jun 21, 2018 at 02​:40​:44AM -0700, Ed Avis (via RT) wrote​:

Sometimes you might want to make a template string with variables, and
interpolate them later.

my $template = 'hello $name';
foreach my $name (qw(Fred Jim)) {
my $s = eval '"' . $template . '"';
die $@​ if $@​;
say $s;
}

That trick with eval is awkward and unhygienic (what if $template
contains a " character?). You could instead do some s///e magic to
find variable names and interpolate them, but that too is error-prone
and hard to generalize to all the things that doublequote
interpolation supports.

Presumably the perl interpreter has an internal routine that performs
double-quote interpolation on a string. That should be exposed as a
builtin function, so you can say

my $s = interpolate\($template\);

Well, not really. The lexer has code to understand a double-quoted string,
and alter what stream of tokens it passes to the parser. So

  "result=$foo[$x + 1]\n"

gets lexed into the stream of tokens

  STRINGIFY
  '('
  THING('result=')
  ADDOP(concat)
  '$'
  PRIVATEREF(padany)
  '['
  '$'
  BAREWORD('x')
  ADDOP('+')
  THING(1)
  ']'
  ADDOP(concat)
  THING("\n")

so the parser thinks that the source code was actually

  'result=' . $foo[$x + 1] . "\n".

There is no simple function that can be exposed.

--
A walk of a thousand miles begins with a single step...
then continues for another 1,999,999 or so.

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 25, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 25, 2018

From @epa

Thanks. I forgot that string interpolation is converted to concatenation at compile time. So the way to implement an interpolate() operator would be to make a new parser at run time and parse the argument's value as a doublequoted string, then evaluate that. It would still be preferable to the trick with eval() because you don't need to worry about the delimiter character appearing in the string.

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 25, 2018

From @iabyn

On Mon, Jun 25, 2018 at 01​:43​:44AM -0700, Ed Avis via RT wrote​:

Thanks. I forgot that string interpolation is converted to
concatenation at compile time. So the way to implement an interpolate()
operator would be to make a new parser at run time and parse the
argument's value as a doublequoted string, then evaluate that. It would
still be preferable to the trick with eval() because you don't need to
worry about the delimiter character appearing in the string.

It would have to be implemented using the underlying eval mechanism, which
would still be very slow.

--
Counsellor Troi states something other than the blindingly obvious.
  -- Things That Never Happen in "Star Trek" #16

1 similar comment
@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Jun 25, 2018

From @iabyn

On Mon, Jun 25, 2018 at 01​:43​:44AM -0700, Ed Avis via RT wrote​:

Thanks. I forgot that string interpolation is converted to
concatenation at compile time. So the way to implement an interpolate()
operator would be to make a new parser at run time and parse the
argument's value as a doublequoted string, then evaluate that. It would
still be preferable to the trick with eval() because you don't need to
worry about the delimiter character appearing in the string.

It would have to be implemented using the underlying eval mechanism, which
would still be very slow.

--
Counsellor Troi states something other than the blindingly obvious.
  -- Things That Never Happen in "Star Trek" #16

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Aug 13, 2018

From @tonycoz

On Mon, 25 Jun 2018 01​:43​:44 -0700, ed wrote​:

Thanks. I forgot that string interpolation is converted to
concatenation at compile time. So the way to implement an
interpolate() operator would be to make a new parser at run time and
parse the argument's value as a doublequoted string, then evaluate
that. It would still be preferable to the trick with eval() because
you don't need to worry about the delimiter character appearing in the
string.

You could just forbid some unusual character from the source. The
obvious choice is NUL​:

  $source =~ /\0/ and die;
  my $result = eval "qq\0$source\0";

Tony

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.