Custom cop_warnings take up 17 bytes per COP.

[p5pRT]

Migrated from rt.perl.org#132451 (status was 'open')

Searchable as RT132451$

[p5pRT]

From @toddr

Created by @toddr

Every COP (line of perl code) stores a 17 byte (mostly) binary string in the COP
struct's cop_warnings field. This string is also known as ${^WARNING_BITS}.
Originally this meant that every line of code had 17 bytes allocated
to the every
line of Perl code to store this information. Way back in 2006, Nick
Clark optimized
this by having 3 of the most common warnings not actually be strings,
but instead
be special pointers (pWARN_ALL, pWARN_NONE, pWARN_STD) which literally
point to memory address 0,1,2. This saved a ton of memory but it means that
anything not in this special class will still take up more space at run time.

An example of this problem can be seen here​:

$> perl -e 'use warnings "once";
foreach (1 .. 10000)
{ eval "sub foo$_ { print qq{hello world\n};}";}
open $proc, "/proc/self/status";
print readline($proc);' | grep RSS

VmRSS​: 30896 kB

$> perl -e 'use warnings;
foreach (1 .. 10000)
{ eval "sub foo$_ { print qq{hello world\n};}";}
open $proc, "/proc/self/status";
print readline($proc);' | grep RSS

VmRSS​: 30632 kB

This example is a little contrived but shows the problem. My suggestion is
that we just store the shared strings in PL_strtab. I would prefer not to
refcount their use and just skip the free when the COP is let go but I'm
not committed to that position.

I'm vaguely concerned with XS poking at this field an manipulating it at
run time. I've not found any examples of that on CPAN but I don't feel
I've made a thorough search.

I'm working on a proposed patch for this. If anyone has concerns,
please let me know.

Perl Info

Flags:
    category=core
    severity=low
    module=

Site configuration information for perl 5.27.6:

Configured by root at Fri Nov 10 21:43:02 CST 2017.

Summary of my perl5 (revision 5 version 27 subversion 6) configuration:
  Commit id: 8d6363863eb2ae5eb05a83c01c276a321727c828
  Platform:
    osname=linux
    osvers=3.10.0-693.2.2.el7.x86_64
    archname=x86_64-linux-thread-multi
    uname='linux toddr.dev.cpanel.net
 3.10.0-693.2.2.el7.x86_64 #1 smp
tue sep 12 22:26:13 utc 2017 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Dusedevel -Uversiononly -Dcc=gcc -Dusethreads
-Dprefix= -DDEBUGGING'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='gcc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
-D_FORTIFY_SOURCE=2'
    optimize='-O2 -g'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include'
    ccversion=''
    gccversion='4.8.5 20150623 (Red Hat 4.8.5-16)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='gcc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64
/lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.17.so


    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.17'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.27.6:
    /root/projects/perl/lib
    /usr/local/lib/perl5/site_perl/5.27.6/x86_64-linux-thread-multi
    /usr/local/lib/perl5/site_perl/5.27.6
    /usr/local/lib/perl5/5.27.6/x86_64-linux-thread-multi
    /usr/local/lib/perl5/5.27.6


Environment for perl 5.27.6:
    HOME=/root
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/root/projects/perl
    LOGDIR (unset)
    PATH=/bin
    PERL_BADLANG (unset)
    SHELL=/bin/zsh

[p5pRT]

From @jkeenan

Have you been able to make progress on that patch?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @toddr

On Mon, 03 Sep 2018 17​:47​:59 -0700, jkeenan wrote​:

Have you been able to make progress on that patch?

Thank you very much.

I have not but we're looking at B​::C for 5.28.0 right now, so let me get back to you in a week.

[atoomic]

one liner to see the issue

# pWARN_STD
╰─> perl -e 'use warnings; eval "sub foo$_ { print qq{hello world\n};}" for (1..100_000); print qx{grep RSS /proc/$$/status}'
VmRSS:	  293304 kB
# warnings once
╰─> perl -e 'use warnings "once"; eval "sub foo$_ { print qq{hello world\n};}" for (1..100_000); print qx{grep RSS /proc/$$/status}'
VmRSS:	  297888 kB