Problem compiling under ppc64le

[morvael]

For a long time I'm compiling each version of Netty (incl. tcnative) on my own, to create the missing epoll library for ppc64le. Upgraded from 4.1.86.Final to 4.1.94.Final (tcnative 2.0.61.Final) and it stopped compiling, because of Werror flag, which then breaks compilation after this warning:

     [exec] /usr/bin/cc -DBORINGSSL_IMPLEMENTATION -I../third_party/googletest/include -I../crypto/../include -I../crypto/fipsmodule/../../include -Werror -Wformat=2 -Wsign-compare -Wmissing-field-initializers -Wwrite-strings -Wvla -Wshadow -ggdb -Wall -fvisibility=hidden -fno-common -Wno-free-nonheap-object -Wimplicit-fallthrough -Wformat-signedness -Wmissing-prototypes -Wold-style-definition -Wstrict-prototypes -D_XOPEN_SOURCE=700 -O3 -fno-omit-frame-pointer -DOPENSSL_C11_ATOMIC -fPIC -MD -MT crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o -MF crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o.d -o crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o -c ../crypto/fipsmodule/bcm.c
     [exec] In file included from ../crypto/fipsmodule/bcm.c:31:
     [exec] In function ‘OPENSSL_memset’,
     [exec]     inlined from ‘bn_big_endian_to_words.constprop’ at ../crypto/fipsmodule/bn/bytes.c:76:7,
     [exec]     inlined from ‘BN_bin2bn.constprop’ at ../crypto/fipsmodule/bn/bytes.c:115:3:
     [exec] ../crypto/fipsmodule/../internal.h:857:10: error: ‘memset’ specified size 18446744073709551608 exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
     [exec]    return memset(dst, c, n);
     [exec]           ^~~~~~~~~~~~~~~~~
     [exec] cc1: all warnings being treated as errors
     [exec] ninja: build stopped: subcommand failed.

OS:
CentOS Linux release 8.5.2111
Linux 4.18.0-348.7.1.el8_5.ppc64le #1 SMP Wed Dec 22 13:41:57 UTC 2021 ppc64le ppc64le ppc64le GNU/Linux

GCC:
gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-4)

[normanmaurer]

Just ignore this warning via -W flag ? Seems like your compiler is too strict here to compile BoringSSL

[morvael]

Guess that requires editing build scripts, I'll try to find it. Thanks.

[morvael]

Deleted all Werror I have found in pom.xml of the boringssl-static project that fails to compile, yet the error above still appears. From where it does pull that Werror flag?

[normanmaurer]

Maybe just add -Wno-error=stringop-overflow=

[morvael]

Cool, but where as it seems it ignored all my pom.xml changes.

[normanmaurer]

Add to the pom.xml. Maybe your compiler set some defaults.

[morvael]

I think the problem is in the created build script that I have no control over in the pom:

    [mkdir] Created dir: /home/dderwi/netty/netty-tcnative/boringssl-static/target/boringssl-chromium-stable/build
     [exec] -- The C compiler identification is GNU 8.5.0
     [exec] -- Detecting C compiler ABI info
     [exec] -- Detecting C compiler ABI info - done
     [exec] -- Check for working C compiler: /usr/bin/cc - skipped
     [exec] -- Detecting C compile features
     [exec] -- Detecting C compile features - done
     [exec] -- The CXX compiler identification is GNU 8.5.0
     [exec] -- Detecting CXX compiler ABI info
     [exec] -- Detecting CXX compiler ABI info - done
     [exec] libunwind not found. Disabling unwind tests.
     [exec] -- Check for working CXX compiler: /usr/bin/c++ - skipped
     [exec] -- Detecting CXX compile features
     [exec] -- Detecting CXX compile features - done
     [exec] -- Found Perl: /usr/bin/perl (found version "5.26.3") 
     [exec] -- Checking for module 'libunwind-generic'
     [exec] --   Package 'libunwind-generic', required by 'virtual:world', not found
     [exec] -- The ASM compiler identification is GNU
     [exec] -- Found assembler: /usr/bin/cc
     [exec] -- Looking for pthread.h
     [exec] -- Looking for pthread.h - found
     [exec] -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
     [exec] -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
     [exec] -- Looking for pthread_create in pthreads
     [exec] -- Looking for pthread_create in pthreads - not found
     [exec] -- Looking for pthread_create in pthread
     [exec] -- Looking for pthread_create in pthread - found
     [exec] -- Found Threads: TRUE  
     [exec] -- Configuring done
     [exec] -- Generating done
     [exec] -- Build files have been written to: /home/dderwi/netty/netty-tcnative/boringssl-static/target/boringssl-chromium-stable/build
     [exec] [1/275] Generating ghashp8-ppc.S
     [exec] [2/275] Generating aesp8-ppc.S
     [exec] [3/275] Building ASM object crypto/fipsmodule/CMakeFiles/fipsmodule.dir/ghashp8-ppc.S.o
     [exec] [4/275] Building C object crypto/fipsmodule/CMakeFiles/fipsmodule.dir/fips_shared_support.c.o
     [exec] [5/275] Building ASM object crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesp8-ppc.S.o
     [exec] [6/275] Building C object crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o
     [exec] FAILED: crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o 
     [exec] /usr/bin/cc -DBORINGSSL_IMPLEMENTATION -I../third_party/googletest/include -I../crypto/../include -I../crypto/fipsmodule/../../include -Werror -Wformat=2 -Wsign-compare -Wmissing-field-initializers -Wwrite-strings -Wvla -Wshadow -ggdb -Wall -fvisibility=hidden -fno-common -Wno-free-nonheap-object -Wimplicit-fallthrough -Wformat-signedness -Wmissing-prototypes -Wold-style-definition -Wstrict-prototypes -D_XOPEN_SOURCE=700 -O3 -fno-omit-frame-pointer -DOPENSSL_C11_ATOMIC -fPIC -MD -MT crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o -MF crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o.d -o crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.o -c ../crypto/fipsmodule/bcm.c
     [exec] In file included from ../crypto/fipsmodule/bcm.c:31:
     [exec] In function ‘OPENSSL_memset’,
     [exec]     inlined from ‘bn_big_endian_to_words.constprop’ at ../crypto/fipsmodule/bn/bytes.c:76:7,
     [exec]     inlined from ‘BN_bin2bn.constprop’ at ../crypto/fipsmodule/bn/bytes.c:115:3:
     [exec] ../crypto/fipsmodule/../internal.h:857:10: error: ‘memset’ specified size 18446744073709551608 exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
     [exec]    return memset(dst, c, n);
     [exec]           ^~~~~~~~~~~~~~~~~
     [exec] cc1: all warnings being treated as errors
     [exec] ninja: build stopped: subcommand failed.

[morvael]

It's as if I should edit generate boringssl-static/target/boringssl-chromium-stable/build script in the middle of the build to add those flags there.

[morvael]

I think I need to raise this with boringssl project.

[normanmaurer]

if you can provide a way for me to reproduce I might be able to help. I think it should be possible to fix this within our build scripts

[morvael]

I think that would require providing a machine with this architecture, OS, and software versions? Architecture is the most problematic, I once tried qemu ppc64le on my x86-64 and it was slowing down to a crawl... Or maybe it would trigger as well on x86-64, as only software/OS causes it?

[morvael]

Let's see first what boringssl people will say about it.

https://bugs.chromium.org/p/boringssl/issues/detail?id=632

[morvael]

Still it's puzzling, I understand the flags to compiler go via those *Flags variables but I can't see my added or removed flags affect the build:

                        <mkdir dir="${boringsslHome}" />
                        <exec executable="cmake" failonerror="true" dir="${boringsslHome}" resolveexecutable="true">
                          <arg value="-DCMAKE_POSITION_INDEPENDENT_CODE=TRUE" />
                          <arg value="-DCMAKE_BUILD_TYPE=Release" />
                          <arg value="-DCMAKE_ASM_FLAGS=${cmakeAsmFlags}" />
                          <arg value="-DCMAKE_C_FLAGS_RELEASE=${cmakeCFlags}" />
                          <arg value="-DCMAKE_CXX_FLAGS_RELEASE=${cmakeCxxFlags}" />
                          <arg value="-DCMAKE_SYSTEM_PROCESSOR=x86_64" />
                          <arg value="-DCMAKE_OSX_ARCHITECTURES=x86_64" />
                          <arg value="${cmakeOsxDeploymentTarget}" />
                          <arg value="-GNinja" />
                          <arg value="${boringsslSourceDir}" />
                        </exec>
                        <if>
                          <!-- may be called ninja-build or ninja -->
                          <!-- See https://github.com/netty/netty-tcnative/issues/475 -->
                          <available file="ninja-build" filepath="${env.PATH}" />
                          <then>
                            <property name="ninjaExecutable" value="ninja-build" />
                          </then>
                          <else>
                            <property name="ninjaExecutable" value="ninja" />
                          </else>
                        </if>
                        <exec executable="${ninjaExecutable}" failonerror="true" dir="${boringsslHome}" resolveexecutable="true" />

[normanmaurer]

Please show me the diff

[morvael]

I think I have found it by correct backtracking. It seems to compile. Will soon show my diff. Thank you for persevering with amateurish me.

[morvael]

So while removing Werror flags was easy (just search & replace), adding the no-error one in the right place was more tricky as it required finding the right place :) I have simply added it in wrong place yesterday (too many cflags variables all over the pom).

[morvael]

diff --git a/boringssl-static/pom.xml b/boringssl-static/pom.xml
index f013c97..0895cf7 100644
--- a/boringssl-static/pom.xml
+++ b/boringssl-static/pom.xml
@@ -200,7 +200,7 @@
                               <!-- On *nix, add ASM flags to disable executable stack -->
                               <property name="cmakeAsmFlags" value="-Wa,--noexecstack" />
                               <!-- Use -DOPENSSL_C11_ATOMIC so we replace most of the locking code with atomics-->
-                              <property name="cmakeCFlags" value="-O3 -fno-omit-frame-pointer -DOPENSSL_C11_ATOMIC" />
+                              <property name="cmakeCFlags" value="-O3 -fno-omit-frame-pointer -DOPENSSL_C11_ATOMIC -Wno-error=stringop-overflow" />
                               <!-- We need to define __STDC_CONSTANT_MACROS and __STDC_FORMAT_MACROS when building boringssl on centos 6 -->
                               <property name="cmakeCxxFlags" value="-O3 -fno-omit-frame-pointer -Wno-error=maybe-uninitialized -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS" />
                             </then>

Can this be added to upstream tcnative? Maybe even make it tied to this specific architecture/OS?

[normanmaurer]

I think we can just add it without tie to to a specific platform. Want to do a PR ?

[morvael]

I finally have to do it for the first time, never did, and always end up like a pleb who doesn't know how to make a PR :)

[morvael]

So give me a while please...

[normanmaurer]

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request

[morvael]

Yes, was reading through that. I understand I need the fork variant, as I'm not able to create my own branch here?

[normanmaurer]

YesAm 20.07.2023 um 11:36 schrieb morvael ***@***.***>: Yes, was reading through that. I understand I need the fork variant, as I'm not able to create my own branch here? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[morvael]

#803

[davidben]

Note BoringSSL no longer supports ppcle64:
https://boringssl-review.googlesource.com/c/boringssl/+/56389

[normanmaurer]

@davidben thanks for clarify.

[morvael]

I do not think I use it in any way, I just need tcnative project to compile, to later compile netty, and finally get epoll native jar with the linux-ppcle_64 classifier.

@normanmaurer can I delete the forked repo now?