gcc12: new build-time warnings associated with 1604cfb027

[jkeenan]

In #19098, user @Jamaika1 reports C-level warnings emanating from utf8.c and utf8.h using a devel version of gcc12. Context is sparse; I've requested more detail.

That prompted me to install a devel version of gcc12 on FreeBSD12 (one slightly earlier than used by reporter). I encountered build-time warnings that I did not encounter when building on the same platform with gcc10.

$ gcc12 --version | head -n 2
gcc12 (FreeBSD Ports Collection) 12.0.0 20210822 (experimental)
Copyright (C) 2021 Free Software Foundation, Inc.

I then built Perl 5 blead with sh ./Configure -des -Dusedevel -Duseithreads -Dcc=gcc12 and parsed the build log for C-level warnings:

$ report-build-warnings 756f0910f9.freebsd.gcc12.threaded.maketp.output.txt.gz 
File:  756f0910f9.freebsd.gcc12.threaded.maketp.output.txt.gz

  Warray-bounds                              6
  Wc++-compat                              240
  Wempty-body                                6
  Wmaybe-uninitialized                       2
  Wsign-compare                              4
  Wtype-limits                               2

The last previous version of gcc I've build perl with on this machine was gcc10. At that time I got these build-time warnings:

$ report-build-warnings f72873660d.freebsd.gcc10.maketp.output.txt.gz
File:  f72873660d.freebsd.gcc10.maketp.output.txt.gz

  Wc++-compat                              240
  Wempty-body                                6
  Wsign-compare                              4
  Wtype-limits                               2

So the Warraybounds warnings (6) and the Wmaybe-uninitialized warnings (2) are new in either gcc11 or gcc12; the others appear to have existed at least from gcc10.

The Warraybounds warnings are these:

  {
    char   => 9,
    group  => "Warray-bounds",
    line   => 461,
    source => "hv.h",
    text   => "array subscript 0 is outside array bounds of 'HEK[768614336404564650]' {aka 'struct hek[768614336404564650]'}",
  },
  {
    char   => 9,
    group  => "Warray-bounds",
    line   => 461,
    source => "hv.h",
    text   => "array subscript 0 is outside array bounds of 'HEK[768614336404564650]' {aka 'struct hek[768614336404564650]'}",
  },
...
  {
    char   => 9,
    group  => "Warray-bounds",
    line   => 461,
    source => "hv.h",
    text   => "array subscript 0 is outside array bounds of 'HEK[768614336404564650]' {aka 'struct hek[768614336404564650]'}",
  },
  {
    char   => 9,
    group  => "Warray-bounds",
    line   => 461,
    source => "hv.h",
    text   => "array subscript 0 is outside array bounds of 'HEK[768614336404564650]' {aka 'struct hek[768614336404564650]'}",
  },
  {
    char   => 9,
    group  => "Warray-bounds",
    line   => 461,
    source => "hv.h",
    text   => "array subscript 0 is outside array bounds of 'HEK[768614336404564650]' {aka 'struct hek[768614336404564650]'}",
  },
  {
    char   => 9,
    group  => "Warray-bounds",
    line   => 461,
    source => "hv.h",
    text   => "array subscript 0 is outside array bounds of 'HEK[768614336404564650]' {aka 'struct hek[768614336404564650]'}",
  },

Line 461 in hv.h was last touched in this commit:

commit 1604cfb0273418ed479719f39def5ee559bffda2
Author:     Michael G. Schwern <schwern@pobox.com>
AuthorDate: Mon Dec 28 18:04:52 2020 -0800
Commit:     Karl Williamson <khw@cpan.org>
CommitDate: Sun Jan 17 09:18:15 2021 -0700

    style: Detabify indentation of the C code maintained by the core.
    
    This just detabifies to get rid of the mixed tab/space indentation.
    
    Applying consistent indentation and dealing with other tabs are another issue.
    
    Done with `expand -i`.
    
    * vutil.* left alone, it's part of version.
    * Left regen managed files alone for now.

Relevant lines:

468 #define hv_exists_ent(hv, keysv, hash)                  \
469     cBOOL(hv_common((hv), (keysv), NULL, 0, 0, HV_FETCH_ISEXISTS, 0, (hash)))
470 #define hv_fetch_ent(hv, keysv, lval, hash)             \
471     ((HE *) hv_common((hv), (keysv), NULL, 0, 0,            \
472                       ((lval) ? HV_FETCH_LVALUE : 0), NULL, (hash)))

The Wmaybe-uninitialized warnings are these:

  {
    char   => 21,
    group  => "Wmaybe-uninitialized",
    line   => 390,
    source => "thread.h",
    text   => "'my_perl' may be used uninitialized",
  },
  {
    char   => 21,
    group  => "Wmaybe-uninitialized",
    line   => 390,
    source => "thread.h",
    text   => "'my_perl' may be used uninitialized",
  },

Line 390 in thread.h was also last touched in 1604cfb, committed to blead in January of this year.

Relevant lines:

386 #ifndef PERL_SET_CONTEXT
387 #  define PERL_SET_CONTEXT(t) \
388     STMT_START {                        \
389         int _eC_;                       \
390         if ((_eC_ = pthread_setspecific(PL_thr_key, (void *)(t))))  \
391             Perl_croak_nocontext("panic: pthread_setspecific (%d) [%s:%d]", \
392                                  _eC_, __FILE__, __LINE__); \
393     } STMT_END
394 #endif /* PERL_SET_CONTEXT */

But I don't see the Woverflow build-time warnings reported in #19098.

@schwern, @khwilliamson or anyone, can you take a look?

Thank you very much.
Jim Keenan

[Leont]

Line 461 in hv.h was last touched in this commit:

I think you need to add a -w to your git blames, that commit was only a whitespace change. That line was last meaningfully changed in d1db91c from 2005.

[hvds]

1604cfb was (mostly) a whitespace-only commit; to see the real last change for these you need something like git blame -w.

For the "may be used uninitialized", the issue is not with the macro itself, but how it is being used - the warning says we may be invoking this macro before the my_perl structure is initialized.

In this case, my best guess is that it is complaining about the attempt to invoke Perl_croak_nocontext within MUTEX_INIT invoked via PERL_SYS_INIT3 in main() - MUTEX_INIT should probably be using a safer croak that checks my_perl, and takes some alternative action if it isn't set up (possibly just exit(1)); we should also be ensuring we set my_perl back to 0 when we free it during destruction.

[jkeenan]

Okay, so the emergence of these warnings is not so much a question of the particular commit as it is of the fact that gcc12 finds the C-coding problematic. The practical question remains: How can we silence these warnings?

[Leont]

The practical question remains: How can we silence these warnings?

It's puzzling. That line doesn't contain any array subscripting, and does deal with a struct hek.

That 768614336404564650 equals 0xAAAAAAAAAAAAAAA, that's probably not a coincidence but I'm not sure what it means

[hvds]

@jkeenan could you try the following?

• cd to your perl source directory
• make sure you have at least configured to build with this gcc (so we have cflags)
• write the following to a file test.c:

#include "perl.h"

void test_hek(HEK *hek) {
    (void)share_hek_hek(hek);
}

• run the following:

`sh cflags test.o` -E test.c | perl -nle 'next unless /^void test_hek/ .. 0; print unless /^#/'

This should show what the preprocessor expands it too. For me it shows:

% `sh cflags test.o` -E test.c | perl -nle 'next unless /^void test_hek/ .. 0; print unless /^#/'
void test_hek(HEK *hek) {
 (void)(++(((struct shared_he *)(((char *)hek) - 
      __builtin_offsetof (
      struct shared_he
      , 
      shared_he_hek
      )
      )) ->shared_he_he.he_valu.hent_refcount), hek);
}
% 

It seems plausible that gcc treats __builtin_offsetof as indistinguishable from array subscripting for the purposes of this warning, but it would be worth confirming whether you get something similar to the above.

[jkeenan]

$ sh ./Configure -des -Dusedevel -Duseithreadds -Dcc=gcc12
...

$ ls -l cflags
-rwxr-xr-x  1 jkeenan  jkeenan  2796 Sep  7 11:00 cflags

$ `sh cflags test.o` -E test.c | perl -nle 'next unless /^void test_hek/ .. 0; print unless /^#/'
void test_hek(HEK *hek) {
    (void)(++(((struct shared_he *)(((char *)hek) - 
         __builtin_offsetof (
         struct shared_he
         , 
         shared_he_hek
         )
         )) ->shared_he_he.he_valu.hent_refcount), hek);
}

$ `sh cflags test.o` -E test.c | perl -nle 'next unless /^void test_hek/ .. 0; print unless /^#/' 1>$P5P_DIR/myhvtest.txt 2>&1

$ cd $P5P_DIR

$ diff -w myhvtest.txt hvhvtest.txt
$ [no output]

Modulo the whitespace caused by redirecting output to a file, the output is what you expected.

[hvds]

Ok; it's a shame it isn't giving more contextual information - if you could group these Warray-bounds to count how many appear for each individual source file, that would give an additional clue. (To avoid having to do a complete non-parallelizing build for that, consider doing a full build and then repeatedly remove and rebuild one of the object files corresponding to those listed below, eg: rm hv.o ; make hv.o.)

Looking at the references to share_hek_hek to group similar calls, I get:

./hv.c:    (void)share_hek_hek(shared);
./hv.c:    (void)share_hek_hek(shared);
./op.c:    (void)share_hek_hek(hek);
./scope.c: (void)share_hek_hek(hek);

./sv.c:    SvPV_set(dsv, HEK_KEY(share_hek_hek(SvSHARED_HEK_FROM_PV(SvPVX_const(ssv)))));
./sv.c:    SvPV_set(sv, (char *)HEK_KEY(share_hek_hek(hek)));
./sv.c:    SvPV_set(sv, (char *)HEK_KEY(share_hek_hek(hek)));
./scope.c: CvNAME_HEK_set(*svp, share_hek_hek(hek));
./pad.c:   CvNAME_HEK_set( cv , share_hek_hek(CvNAME_HEK(proto)));

#ifndef USE_ITHREADS
./hv.c:    HeKEY_hek(entry) = share_hek_hek(chain->refcounted_he_hek);

#ifdef PERL_ANY_COW
./sv.c:    new_pv = HEK_KEY(share_hek_hek(SvSHARED_HEK_FROM_PV(SvPVX_const(ssv))));

.. so to get 6 warnings, perhaps it is not objecting to those in void context (assuming your build sets PERL_ANY_COW and not USE_ITHREADS). That would imply 4 warnings from sv.c, and one each from scope.c and pad.c.

[hvds]

@jkeenan For the other warning, please could you check if the branch safe-croak shuts it up?

@Leont @tonycoz do one of you guys know if it would be possible/safe to write choke_nocontext (9243901) to fall back to libc fprintf, perhaps by putting it in its own file along with some #undef fprintf etc?

[jkeenan]

Ok; it's a shame it isn't giving more contextual information - if you could group these Warray-bounds to count how many appear for each individual source file, that would give an additional clue. (To avoid having to do a complete non-parallelizing build for that, consider doing a full build and then repeatedly remove and rebuild one of the object files corresponding to those listed below, eg: rm hv.o ; make hv.o.)

Looking at the references to share_hek_hek to group similar calls, I get:

./hv.c:    (void)share_hek_hek(shared);
./hv.c:    (void)share_hek_hek(shared);
./op.c:    (void)share_hek_hek(hek);
./scope.c: (void)share_hek_hek(hek);

./sv.c:    SvPV_set(dsv, HEK_KEY(share_hek_hek(SvSHARED_HEK_FROM_PV(SvPVX_const(ssv)))));
./sv.c:    SvPV_set(sv, (char *)HEK_KEY(share_hek_hek(hek)));
./sv.c:    SvPV_set(sv, (char *)HEK_KEY(share_hek_hek(hek)));
./scope.c: CvNAME_HEK_set(*svp, share_hek_hek(hek));
./pad.c:   CvNAME_HEK_set( cv , share_hek_hek(CvNAME_HEK(proto)));

#ifndef USE_ITHREADS
./hv.c:    HeKEY_hek(entry) = share_hek_hek(chain->refcounted_he_hek);

#ifdef PERL_ANY_COW
./sv.c:    new_pv = HEK_KEY(share_hek_hek(SvSHARED_HEK_FROM_PV(SvPVX_const(ssv))));

.. so to get 6 warnings, perhaps it is not objecting to those in void context (assuming your build sets PERL_ANY_COW and not USE_ITHREADS). That would imply 4 warnings from sv.c, and one each from scope.c and pad.c.

Here's perl -V output:

$ cat perl-V-freebsd12-gcc12.txt 
Summary of my perl5 (revision 5 version 35 subversion 4) configuration:
  Commit id: 034242a8a539b024648b18318271eabddf40ca48
  Platform:
    osname=freebsd                          archname=amd64-freebsd
    osvers=12.2-stable
    uname='freebsd perlmonger.nycbug.org 12.2-stable freebsd 12.2-stable r368982 generic amd64 '
    config_args='-des -Dusedevel -Dcc=gcc12'
    hint=recommended                        use64bitall=define
    useposix=true                           uselongdouble=undef
    d_sigaction=define                      usemymalloc=n
    useithreads=undef                       default_inc_excludes_dot=define
    usemultiplicity=undef
    use64bitint=define
  Compiler:
    cc='gcc12'
    ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
    optimize='-O2'
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='12.0.0 20210822 (experimental)'
    gccosandvers=''                         longdblkind=3
    intsize=4                               ivtype='long'
    longsize=8                              ivsize=8
    ptrsize=8                               nvtype='double'
    doublesize=8                            nvsize=8
    byteorder=12345678                      Off_t='off_t'
    doublekind=3                            lseeksize=8
    d_longlong=define                       alignbytes=8
    longlongsize=8                          prototype=define
    d_longdbl=define
    longdblsize=16
  Linker and Libraries:
    ld='gcc12'
    ldflags ='-Wl,-E  -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib
    libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=                                   libperl=libperl.a
    so=so                                   gnulibc_version=''
    useshrplib=false
  Dynamic Linking:
    dlsrc=dl_dlopen.xs                      ccdlflags=' '
    dlext=so                                cccdlflags='-DPIC -fPIC'
    d_dlsymun=undef
    lddlflags='-shared  -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES                               USE_LARGE_FILES
    PERLIO_LAYERS                           USE_LOCALE
    PERL_COPY_ON_WRITE                      USE_LOCALE_COLLATE
    PERL_DONT_CREATE_GVSV                   USE_LOCALE_CTYPE
    PERL_MALLOC_WRAP                        USE_LOCALE_NUMERIC
    PERL_OP_PARENT                          USE_LOCALE_TIME
    PERL_PRESERVE_IVUV                      USE_PERLIO
    PERL_USE_DEVEL                          USE_PERL_ATOF
    USE_64_BIT_ALL
    USE_64_BIT_INT
  Built under freebsd
  Compiled at Sep  8 2021 22:00:44
  %ENV:
  @INC:
    lib
    /usr/local/lib/perl5/site_perl/5.35.4/amd64-freebsd
    /usr/local/lib/perl5/site_perl/5.35.4
    /usr/local/lib/perl5/5.35.4/amd64-freebsd
    /usr/local/lib/perl5/5.35.4

I've sent you the full make output via email.

Thanks.
jimk

[hvds]

I've sent you the full make output via email.

Thanks Jim; quite different from what I had anticipated, it's actually warning about only two invocations, pad.c:S_cv_clone (twice), and op.c:Perl_newMYSUB (four times: twice for op.c, twice more for opmini.c).

In both those invocations the arg is derived at some point as CvNAME_HEK of a CV (but we take avoiding action if the CV is not CvNAMED, so we're not dealing with the case where that returns 0).

I'm no closer to understanding what it is actually complaining about, but it might be possible to start trying to create a cut-down testcase that also generates the warning. At this point I wouldn't rule out the possibility that it's a gcc bug.

[hvds]

At this point I wouldn't rule out the possibility that it's a gcc bug.

I was able to build gcc-11.2.0 locally, and it does not warn on this; I'll try reporting it to the gcc team and see what they say.

[hvds]

@jkeenan For the other warning, please could you check if the branch safe-croak shuts it up?

@Leont @tonycoz do one of you guys know if it would be possible/safe to write choke_nocontext (9243901) to fall back to libc fprintf, perhaps by putting it in its own file along with some #undef fprintf etc?

Ok, while I think the mutex-init croak is a bug, it's not actually what is being warned about here. It reduces to this:

% gcc-11.2.0 -c -O2 -Wmaybe-uninitialized test.c
test.c: In function 'init':
test.c:15:9: warning: 'my_perl' may be used uninitialized [-Wmaybe-uninitialize]
   15 |         pthread_setspecific(PL_thr_key, PL_current_context);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test.c:10:12: note: by argument 2 of type 'const void *' to 'pthread_setspecific' declared here
   10 | extern int pthread_setspecific (unsigned int key, const void *pointer);
      |            ^~~~~~~~~~~~~~~~~~~
% cat test.c
typedef struct { char c[1]; } PerlInterpreter;
extern PerlInterpreter *PL_curinterp ;

typedef unsigned int perl_key;
extern perl_key PL_thr_key;

extern _Thread_local void *PL_current_context;

extern void *malloc (long unsigned int size);
extern int pthread_setspecific (unsigned int key, const void *pointer);

void perl_alloc(void) {
    PerlInterpreter *my_perl = (PerlInterpreter*)malloc((sizeof(PerlInterpreter)));
    PL_current_context = (void *)my_perl;
    pthread_setspecific(PL_thr_key, PL_current_context);
}   
% 

So I suspect this is actually about the nature of thread local storage, which I know nothing about. @nwc10 can you take a look if this could possible be a valid complaint?