Closures returned from threads don't wrap around the right lexicals

[p5pRT]

Migrated from rt.perl.org#96538 (status was 'resolved')

Searchable as RT96538$

[p5pRT]

From perl@profvince.com

Created by perl@profvince.com

On join, threads.pm calls sv_dup on the values returned to the caller
thread, but with an empty pointer table. This is wrong, because when a
code reference is returned, sv_dup is then unable to relink the variables
captured by the coderef to their proper incarnation from the parent
thread. Here's an example :

perl -Mthreads -wE'my $x; my $set_x = sub { $x = shift }; $set_x->(123);
say $x; threads->create(sub { return $set_x })->join->(456); say $x'

This should print 123 and then 456, but it prints 123. The duplicated copy
of $set_x returned by the thread assigns to a scalar that is not the local
instance of $x.

Perl Info

Flags:
     category=core
     severity=medium

Site configuration information for perl 5.14.1:

Configured by vince at Fri Jul  8 19:46:39 CEST 2011.

Summary of my perl5 (revision 5 version 14 subversion 1) configuration:

   Platform:
     osname=linux, osvers=2.6.39.1-fuuka.profvince.com, 
archname=x86_64-linux-thread-multi
     uname='linux fuuka 2.6.39.1-fuuka.profvince.com #1 smp fri jun 10 
20:18:17 cest 2011 x86_64 intel(r) core(tm)2 duo cpu e6750 @ 2.66ghz 
genuineintel gnulinux '
     config_args='-Doptimize=-march=core2 -O3 -fomit-frame-pointer 
-ftree-vectorize -ftree-vectorizer-verbose=1 -pipe -DNDEBUG -des 
-Dusedevel -Dman1dir=none -Dman3dir=none -Dcf_email=vpit@cpan.org 
-Dmydomain=fuuka.profvince.com -DEBUGGING=none -Dusethreads 
-Dusemmorebits -Dprefix=/home/vince/perl/builds/thr/64/5.14.1'
     hint=recommended, useposix=true, d_sigaction=define
     useithreads=define, usemultiplicity=define
     useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
     use64bitint=define, use64bitall=define, uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing 
-pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
     optimize='-march=core2 -O3 -fomit-frame-pointer -ftree-vectorize 
-ftree-vectorizer-verbose=1 -pipe -DNDEBUG',
     cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe 
-fstack-protector'
     ccversion='', gccversion='4.5.2', gccosandvers=''
     intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
     ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=8, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
     libpth=/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /usr/lib 
/lib64 /usr/lib64 /usr/local/lib64
     libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc 
-lgdbm_compat
     perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
     libc=/lib/libc-2.13.so, so=so, useshrplib=false, libperl=libperl.a
     gnulibc_version='2.13'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
     cccdlflags='-fPIC', lddlflags='-shared -march=core2 -O3 
-fomit-frame-pointer -ftree-vectorize -ftree-vectorizer-verbose=1 -pipe 
-DNDEBUG -L/usr/local/lib -fstack-protector'

Locally applied patches:



@INC for perl 5.14.1:
     
/home/vince/perl/builds/thr/64/5.14.1/lib/site_perl/5.14.1/x86_64-linux-thread-multi
     /home/vince/perl/builds/thr/64/5.14.1/lib/site_perl/5.14.1
     
/home/vince/perl/builds/thr/64/5.14.1/lib/5.14.1/x86_64-linux-thread-multi
     /home/vince/perl/builds/thr/64/5.14.1/lib/5.14.1
     .


Environment for perl 5.14.1:
     HOME=/home/vince
     LANG=fr_FR.UTF-8
     LANGUAGE (unset)
     LC_ALL=fr_FR.UTF-8
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
     
PATH=/home/vince/bin:/home/vince/perl/builds/bin:/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.5.2:/opt/intel/cce/10.1.018/bin:/usr/games/bin
     PERL_BADLANG (unset)
     SHELL=/bin/bash

[p5pRT]

From @jdhedden

From the 'threads' POD​:

  Returning closures from threads
  Returning closures from threads should not be relied upon. Depending
  of the Perl version and the application code, results may range from
  success, to (apparently harmless) warnings of leaked scalar, or all
  the way up to crashing of the Perl interpreter.

On Mon, Aug 8, 2011 at 08​:29, perl@​profvince.com
<perlbug-followup@​perl.org> wrote​:

# New Ticket Created by  perl@​profvince.com
# Please include the string​:  [perl #96538]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=96538 >

This is a bug report for perl from perl@​profvince.com,
generated with the help of perlbug 1.39 running under perl 5.14.1.

-----------------------------------------------------------------
[Please describe your issue here]

On join, threads.pm calls sv_dup on the values returned to the caller
thread, but with an empty pointer table. This is wrong, because when a
code reference is returned, sv_dup is then unable to relink the variables
captured by the coderef to their proper incarnation from the parent
thread. Here's an example :

perl -Mthreads -wE'my $x; my $set_x = sub { $x = shift }; $set_x->(123);
say $x; threads->create(sub { return $set_x })->join->(456); say $x'

This should print 123 and then 456, but it prints 123. The duplicated copy
of $set_x returned by the thread assigns to a scalar that is not the local
instance of $x.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
    category=core
    severity=medium
---
Site configuration information for perl 5.14.1​:

Configured by vince at Fri Jul  8 19​:46​:39 CEST 2011.

Summary of my perl5 (revision 5 version 14 subversion 1) configuration​:

  Platform​:
    osname=linux, osvers=2.6.39.1-fuuka.profvince.com,
archname=x86_64-linux-thread-multi
    uname='linux fuuka 2.6.39.1-fuuka.profvince.com #1 smp fri jun 10
20​:18​:17 cest 2011 x86_64 intel(r) core(tm)2 duo cpu e6750 @​ 2.66ghz
genuineintel gnulinux '
    config_args='-Doptimize=-march=core2 -O3 -fomit-frame-pointer
-ftree-vectorize -ftree-vectorizer-verbose=1 -pipe -DNDEBUG -des
-Dusedevel -Dman1dir=none -Dman3dir=none -Dcf_email=vpit@​cpan.org
-Dmydomain=fuuka.profvince.com -DEBUGGING=none -Dusethreads
-Dusemmorebits -Dprefix=/home/vince/perl/builds/thr/64/5.14.1'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler​:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
-pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-march=core2 -O3 -fomit-frame-pointer -ftree-vectorize
-ftree-vectorizer-verbose=1 -pipe -DNDEBUG',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector'
    ccversion='', gccversion='4.5.2', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries​:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /usr/lib
/lib64 /usr/lib64 /usr/local/lib64
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
-lgdbm_compat
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.13.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.13'
  Dynamic Linking​:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -march=core2 -O3
-fomit-frame-pointer -ftree-vectorize -ftree-vectorizer-verbose=1 -pipe
-DNDEBUG -L/usr/local/lib -fstack-protector'

Locally applied patches​:

---
@​INC for perl 5.14.1​:

/home/vince/perl/builds/thr/64/5.14.1/lib/site_perl/5.14.1/x86_64-linux-thread-multi
    /home/vince/perl/builds/thr/64/5.14.1/lib/site_perl/5.14.1

/home/vince/perl/builds/thr/64/5.14.1/lib/5.14.1/x86_64-linux-thread-multi
    /home/vince/perl/builds/thr/64/5.14.1/lib/5.14.1
    .

---
Environment for perl 5.14.1​:
    HOME=/home/vince
    LANG=fr_FR.UTF-8
    LANGUAGE (unset)
    LC_ALL=fr_FR.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/home/vince/bin​:/home/vince/perl/builds/bin​:/usr/local/bin​:/usr/bin​:/bin​:/opt/bin​:/usr/x86_64-pc-linux-gnu/gcc-bin/4.5.2​:/opt/intel/cce/10.1.018/bin​:/usr/games/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @dcollinsn

It sounds like the response 5 years ago was that this isn't a bug. Is there work to be done? Is this a wishlist ticket?

--
Respectfully,
Dan Collins

[p5pRT]

From perl@profvince.com

Le 18/08/2016 à 14​:04, Dan Collins via RT a écrit :

It sounds like the response 5 years ago was that this isn't a bug. Is there work to be done? Is this a wishlist ticket?

If it can crash the interpreter, then there's definitely work to be done.

Vincent

[p5pRT]

From @iabyn

On Thu, Aug 18, 2016 at 02​:17​:09PM -0300, Vincent Pit (VPIT) wrote​:

Le 18/08/2016 à 14​:04, Dan Collins via RT a écrit :

It sounds like the response 5 years ago was that this isn't a bug. Is there work to be done? Is this a wishlist ticket?

If it can crash the interpreter, then there's definitely work to be done.

I don't think there's any evidence that it can crash the interpreter on
current perls.

--
I don't want to achieve immortality through my work...
I want to achieve it through not dying.
  -- Woody Allen

[p5pRT]

From perl@profvince.com

Le 18/08/2016 à 17​:51, Dave Mitchell via RT a écrit :

On Thu, Aug 18, 2016 at 02​:17​:09PM -0300, Vincent Pit (VPIT) wrote​:

Le 18/08/2016 à 14​:04, Dan Collins via RT a écrit :

It sounds like the response 5 years ago was that this isn't a bug. Is there work to be done? Is this a wishlist ticket?

If it can crash the interpreter, then there's definitely work to be done.

I don't think there's any evidence that it can crash the interpreter on
current perls.

Well that's what threads.pm claims in its "bugs and limitations"
section, so it should either be fixed or the documentation should be
updated.

[p5pRT]

From zefram@fysh.org

We can't possibly make this work as originally requested. To preserve
object identity across thread creation and joining would require
maintaining the pointer table linking the threads, with the pointer table
owning refs to everything it knows about, which would prevent anything
from being freed in the parent thread.

The documentation about this was poor, though. I've revised it in commit
27d3197, removing the obsolete caveats
about crashing and about subs generally not working, and adding some
discussion of the object identity issue raised in this ticket.

-zefram

[p5pRT]

@iabyn - Status changed from 'open' to 'resolved'