scons python modules malforms additional compiler flags in some cases

[srakitnican]

Because of following code flags = flags.replace('-Wall', '').replace('-Werror', ''), something like -Werror=format-security becomes =format-security.

https://github.com/CauldronDevelopmentLLC/cbang/search?utf8=✓&q=-Werror&type=Code

[jcoffland]

Fixed

[srakitnican]

Thanks.

Please note you have one more reference like this in CAMotics sources, I assume it could give the same issue: https://github.com/CauldronDevelopmentLLC/CAMotics/blob/83e0d40acef7f241919153b3746b49b0d8e0e8bc/src/cairo/SConscript#L7

[srakitnican]

Hmm, I think I've tried similar fix but then it was throwing different kind of error. Something like -Werror=format-security requires -Wall flag. So in my fix I was removing everything following -Werror that wasn't a space as well, not ideal but it worked for this case.

flags = re.sub(r'-Werro([^\s]+|r)', '', flags)

[jcoffland]

I'm wary of replacing all specific -Werror=... type options.

[srakitnican]

But newest code with Fedora flags fails to build. I have to tamper with logic again.

g++ -o build/boost/libs/iostreams/src/bzip2.o -c -std=gnu++98 -ggdb -I/usr/include/v8-3.14/ -O2 -g -pipe -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -DDEBUG -D_REENTRANT -DHAVE_EXPAT -DHAVE_PTHREADS -DHAVE_LIBSQLITE -DHAVE_OPENSSL -DHAVE_V8 -DDEBUG_LEVEL=1 -DUSING_CBANG -Iinclude -Isrc -Isrc/boost src/boost/libs/iostreams/src/bzip2.cpp
cc1plus: error: -Wformat-security ignored without -Wformat [-Werror=format-security]
g++ -o build/boost/libs/iostreams/src/file_descriptor.o -c -std=gnu++98 -ggdb -I/usr/include/v8-3.14/ -O2 -g -pipe -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -DDEBUG -D_REENTRANT -DHAVE_EXPAT -DHAVE_PTHREADS -DHAVE_LIBSQLITE -DHAVE_OPENSSL -DHAVE_V8 -DDEBUG_LEVEL=1 -DUSING_CBANG -Iinclude -Isrc -Isrc/boost src/boost/libs/iostreams/src/file_descriptor.cpp
cc1plus: error: -Wformat-security ignored without -Wformat [-Werror=format-security]
cc1plus: some warnings being treated as errors
scons: *** [build/boost/libs/iostreams/src/file_descriptor.o] Error 1
cc1plus: some warnings being treated as errors
scons: *** [build/boost/libs/iostreams/src/bzip2.o] Error 1
scons: building terminated because of errors.

[jcoffland]

You could install bzip2-devel to work around this.

[jcoffland]

Also, adding ccflags=-Wformat to the scons command line might help.

[srakitnican]

This happens with bzip2-devel installed. Adding -Wformat definitely helps. Thanks.

[jcoffland]

C! is supposed to use the system bzip2 if it's available. Can you post the errors you find in config.log that are related to it's attempts to configure the system bzip2 so I can debug? You should see lines like this:

Checking for C header file bzlib.h... no
Checking for C library bz2... no

Perhaps the first line is a yes. If it looks like this:

Checking for C header file bzlib.h... (cached) no
Checking for C library bz2... (cached) no

Then delete the scons cache and rebuild:

rm -rf .scon*
scons

Then search for bz2 in config.log and paste what you find here.

[srakitnican]

I am using mock to build the package, mock recreates build tree every time it is started. It seems that system library is found:

Checking for C header file bzlib.h... yes
Checking for C library bz2... yes

scons: Configure: Checking for C header file bzlib.h... 
.sconf_temp/conftest_4.c <-
  |
  |#include <bzlib.h>
  |
  |
gcc -o .sconf_temp/conftest_4.o -c -std=c99 -ggdb -Wall -Werror -I/usr/include/v8-3.14/ -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -DDEBUG -D_REENTRANT .sconf_temp/conftest_4.c
scons: Configure: yes

scons: Configure: Checking for C library bz2... 
.sconf_temp/conftest_5.c <-
  |
  |
  |
  |int
  |main() {
  |  
  |return 0;
  |}
  |
gcc -o .sconf_temp/conftest_5.o -c -std=c99 -ggdb -Wall -Werror -I/usr/include/v8-3.14/ -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -DDEBUG -D_REENTRANT .sconf_temp/conftest_5.c
gcc -o .sconf_temp/conftest_5 -rdynamic -pthread .sconf_temp/conftest_5.o -lz -lpthread -lbz2
scons: Configure: yes

$ cat .sconf_temp/conftest_4.c 

#include <bzlib.h>

$ cat .sconf_temp/conftest_5.c 



int
main() {
  
return 0;
}

But still:

g++ -o build/boost/libs/iostreams/src/bzip2.o -c -std=gnu++98 -ggdb -I/usr/include/v8-3.14/ -O2 -g -pipe -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -DDEBUG -D_REENTRANT -DHAVE_EXPAT -DHAVE_PTHREADS -DHAVE_LIBSQLITE -DHAVE_OPENSSL -DHAVE_V8 -DDEBUG_LEVEL=1 -DUSING_CBANG -Iinclude -Isrc -Isrc/boost src/boost/libs/iostreams/src/bzip2.cpp
cc1plus: error: -Wformat-security ignored without -Wformat [-Werror=format-security]
g++ -o build/boost/libs/iostreams/src/file_descriptor.o -c -std=gnu++98 -ggdb -I/usr/include/v8-3.14/ -O2 -g -pipe -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -DDEBUG -D_REENTRANT -DHAVE_EXPAT -DHAVE_PTHREADS -DHAVE_LIBSQLITE -DHAVE_OPENSSL -DHAVE_V8 -DDEBUG_LEVEL=1 -DUSING_CBANG -Iinclude -Isrc -Isrc/boost src/boost/libs/iostreams/src/file_descriptor.cpp
cc1plus: error: -Wformat-security ignored without -Wformat [-Werror=format-security]
cc1plus: some warnings being treated as errors
scons: *** [build/boost/libs/iostreams/src/file_descriptor.o] Error 1
cc1plus: some warnings being treated as errors
scons: *** [build/boost/libs/iostreams/src/bzip2.o] Error 1
scons: building terminated because of errors.

[jcoffland]

Ahh... I didn't notice that the problem here is not with bzip2 but with the bzip2 that is built into boost. Try installing boost-devel.

[srakitnican]

boost-devel is installed also, I don't see it searching exactly for boost so I have no idea if it finds it or not. Any hint of what I should be looking for?

Here is the full build log: build.log.txt

[jcoffland]

I was wrong. C! now always uses its built-in boost. From your build log I see that you (or the package builder) are adding -Wformat-security on the command line. You have to also add -Wformat to avoid this error. I believe you indicated above that adding -Wformat fixed the problem. Or is it getting stripped out by the "remove warnings" bit in src/boost/SConstruct?

[srakitnican]

Adding -Wformat does indeed fix this issue, but is not ideal solution. I am using a rpmbuild macro to pass in the standard set of build flags that are set by Fedora people. Shall this set of flags change in the future in a way that doesn't favor this issue, this has a potential of breaking again.

[jcoffland]

But it's the Fedora flags that are in error. According to the compiler error, you cannot have -Wformat-security with out -Wformat. The fact that they omitted this is wrong. At least for this compiler.

cc1plus: error: -Wformat-security ignored without -Wformat [-Werror=format-security]

[srakitnican]

But Fedora flags are turning this on with -Wall that gets stripped by the build system, but leaving -Werror=format-security, leading to this issue.

[jcoffland]

Ok, I see. So if we are stripping out -Wall then we need to also remove -Werror=.* like you originally suggested. I've committed changes.