Skip to content

Commit

Permalink
Add Boost.Build variants for UBSan-enabled builds [skip appveyor]
Browse files Browse the repository at this point in the history
The build variants enable group of checks offered by clang
UndefinedBehaviorSanitizer detector.

Add sanitizers suppression file in .ci/blacklist.supp based on
copy of the file from Boost.Beast.

Update Travis CI:
  - Add .ci/build-and-test.sh script as handy proxy for b2 command.
  - Add build jobs for each of the three new UBSan variants.
  - Display COMPILER and VARIANT first as these two are most important
    details while inspecting the build matrix
    (TOOLSET can be derived from COMPILER).
  • Loading branch information
mloskot committed Jun 21, 2018
1 parent e48abf0 commit 14a7c20
Show file tree
Hide file tree
Showing 5 changed files with 167 additions and 27 deletions.
40 changes: 40 additions & 0 deletions .ci/blacklist.supp
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# LLVM/clang sanitizers suppression file for Boost.GIL
#
# Copied from https://github.com/boostorg/beast/blob/develop/tools/blacklist.supp
#
# Remember that this blacklist file is GLOBAL to all sanitizers
# Be therefore extremely careful when considering to add a sanitizer
# filter here instead of using a runtime suppression
#
# Remember also that filters here quite literally completely
# remove instrumentation altogether, so filtering here means
# that sanitizers such as tsan will false positive on problems
# introduced by code filtered here.
#
# The main use for this file is ubsan, as it's the only sanitizer
# without a runtime suppression facility.
#
# Be ESPECIALLY careful when filtering out entire source files!
# Try if at all possible to filter only functions using fun:regex
# Remember you must use mangled symbol names with fun:regex

# boost/lexical_cast.hpp:1625:43: runtime error: downcast of address 0x7fbb4fffbce8 which does not point to an object of type 'buffer_t' (aka 'parser_buf<std::basic_streambuf<char, char_traits<char> >, char>')
# Fixed in Boost 1.63.0 https://svn.boost.org/trac/boost/ticket/12889
#
fun:*shl_input_streamable*

## The well known ubsan failure in libstdc++ extant for years :)
# Line 96:24: runtime error: load of value 4294967221, which is not a valid value for type 'std::_Ios_Fmtflags'
#
#fun:*_Ios_Fmtflags*

# boost/any.hpp:259:16: runtime error: downcast of address 0x000004392e70 which does not point to an object of type 'any::holder<int>'
#
#fun:*any_cast*

# basic_string.h:409:51: runtime error: unsigned integer overflow: 3 - 9 cannot be represented in type 'unsigned long' in std::string::_S_compare(unsigned long, unsigned long)
fun:*_S_compare*
fun:*string::compare*

# stl_tree.h:702:24: runtime error: downcast of address 0x000001271b68 with insufficient space for an object of type
fun:*_Rb_tree*
40 changes: 40 additions & 0 deletions .ci/build-and-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Boost.GIL proxy for b2 command on Travis CI (can run locally too)
set -eu

if [ ! -z ${TRAVIS+x} ] && [ "$TRAVIS" == "true" ] && [ ! -z ${DOC+x} ]; then
echo "Documentation build requested, skipping tests build"
exit 0
fi

if [ -z ${TOOLSET+x} ]; then
echo "Missing environment TOOLSET"
exit 1
fi

if [ -z ${VARIANT+x} ]; then
echo "Missing environment VARIANT"
exit 1
fi

if [ -z ${B2_OPTIONS+x} ]; then
B2_OPTIONS=""
fi

if [ ! -z ${TRAVIS+x} ] && [ "$TRAVIS" == "true" ]; then
JOBS="2"
elif [[ $(uname -s) == "Linux" ]]; then
JOBS=$(lscpu -p | grep -v '^#' | sort -u -t, -k 2,4 | wc -l)
elif [[ $(uname) == "Darwin" ]]; then
JOBS=$(sysctl -n hw.physicalcpu)
else
JOBS=1
fi

echo "Running ./b2 -j $JOBS $B2_OPTIONS toolset=$TOOLSET variant=$VARIANT"

set -euv
./b2 -j $JOBS $B2_OPTIONS toolset=$TOOLSET variant=$VARIANT libs/gil/test
./b2 -j $JOBS $B2_OPTIONS toolset=$TOOLSET variant=$VARIANT libs/gil/toolbox/test
./b2 -j $JOBS $B2_OPTIONS toolset=$TOOLSET variant=$VARIANT libs/gil/numeric/test
./b2 -j $JOBS $B2_OPTIONS toolset=$TOOLSET variant=$VARIANT libs/gil/io/test//simple
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,6 @@ CMakeLists.txt text eol=lf
*.html text
*.txt text
*.yml text

# sanitizers suppression
*.supp text eol=lf
77 changes: 51 additions & 26 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ branches:
env:
global:
- secure: "UHit2f6Hq2pkHvx8rfrQvFacYiQKVO3vrCbNuDi/VSAIzQjRnqCqE06y4vpXLMsXf62TvBeCBStIuI8g+HP8B+f39oGb/9Om+yIgN/yes47R4sLFKFbRiOS6sfCIefJp7Kx7GSFf81xWxStpIU4QaSsk8Dlt5xyurTWXFSde+lQ="

matrix:
- BOGUS_JOB=true

Expand All @@ -28,7 +27,7 @@ matrix:
- env: BOGUS_JOB=true
include:
- os: linux
env: TOOLSET=gcc COMPILER=g++-5 CXXSTD=c++11 VARIANT=debug
env: COMPILER=g++-5 VARIANT=debug TOOLSET=gcc CXXSTD=c++11
addons:
apt:
packages:
Expand All @@ -37,7 +36,7 @@ matrix:
- ubuntu-toolchain-r-test

- os: linux
env: TOOLSET=gcc COMPILER=g++-5 CXXSTD=c++11 VARIANT=release
env: COMPILER=g++-5 VARIANT=release TOOLSET=gcc CXXSTD=c++11
addons:
apt:
packages:
Expand All @@ -46,7 +45,7 @@ matrix:
- ubuntu-toolchain-r-test

- os: linux
env: TOOLSET=gcc COMPILER=g++-6 CXXSTD=c++11 VARIANT=debug
env: COMPILER=g++-6 VARIANT=debug TOOLSET=gcc CXXSTD=c++11
addons:
apt:
packages:
Expand All @@ -55,7 +54,7 @@ matrix:
- ubuntu-toolchain-r-test

- os: linux
env: TOOLSET=gcc COMPILER=g++-6 CXXSTD=c++11 VARIANT=release
env: COMPILER=g++-6 VARIANT=release TOOLSET=gcc CXXSTD=c++11
addons:
apt:
packages:
Expand All @@ -64,7 +63,7 @@ matrix:
- ubuntu-toolchain-r-test

- os: linux
env: TOOLSET=gcc COMPILER=g++-7 CXXSTD=c++11 VARIANT=debug
env: COMPILER=g++-7 VARIANT=debug TOOLSET=gcc CXXSTD=c++11
addons:
apt:
packages:
Expand All @@ -73,7 +72,7 @@ matrix:
- ubuntu-toolchain-r-test

- os: linux
env: TOOLSET=gcc COMPILER=g++-7 CXXSTD=c++11 VARIANT=release
env: COMPILER=g++-7 VARIANT=release TOOLSET=gcc CXXSTD=c++11
addons:
apt:
packages:
Expand All @@ -82,7 +81,7 @@ matrix:
- ubuntu-toolchain-r-test

- os: linux
env: TOOLSET=clang COMPILER=clang++-3.9 CXXSTD=c++11 VARIANT=debug
env: COMPILER=clang++-3.9 VARIANT=debug TOOLSET=clang CXXSTD=c++11
addons:
apt:
packages:
Expand All @@ -92,7 +91,7 @@ matrix:
- llvm-toolchain-precise-3.9

- os: linux
env: TOOLSET=clang COMPILER=clang++-3.9 CXXSTD=c++11 VARIANT=release
env: COMPILER=clang++-3.9 VARIANT=release TOOLSET=clang CXXSTD=c++11
addons:
apt:
packages:
Expand All @@ -101,11 +100,41 @@ matrix:
- ubuntu-toolchain-r-test
- llvm-toolchain-precise-3.9

- os: linux
env: COMPILER=clang++-5.0 VARIANT=ubsan_integer TOOLSET=clang CXXSTD=c++11 UBSAN_OPTIONS='print_stacktrace=1'
addons:
apt:
packages:
- clang-5.0
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-5.0

- os: linux
env: COMPILER=clang++-5.0 VARIANT=ubsan_undefined TOOLSET=clang CXXSTD=c++11 UBSAN_OPTIONS='print_stacktrace=1'
addons:
apt:
packages:
- clang-5.0
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-5.0

- os: linux
env: COMPILER=clang++-5.0 VARIANT=ubsan_nullability TOOLSET=clang CXXSTD=c++11 UBSAN_OPTIONS='print_stacktrace=1'
addons:
apt:
packages:
- clang-5.0
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-5.0

- os: osx
env: TOOLSET=clang COMPILER=clang++ CXXSTD=c++11 VARIANT=debug
env: COMPILER=clang++ VARIANT=debug TOOLSET=clang CXXSTD=c++11

- os: osx
env: TOOLSET=clang COMPILER=clang++ CXXSTD=c++11 VARIANT=release
env: COMPILER=clang++ VARIANT=release TOOLSET=clang CXXSTD=c++11

- env: DOC=1
addons:
Expand Down Expand Up @@ -135,26 +164,22 @@ script:
if [ "$DOC" ]; then
echo "using doxygen ;" > ~/user-config.jam
./b2 libs/gil/doc
else
echo "using $TOOLSET : : $COMPILER : <cxxflags>-std=$CXXSTD ;" > ~/user-config.jam
./b2 libs/gil/test toolset=$TOOLSET variant=$VARIANT
./b2 libs/gil/toolbox/test toolset=$TOOLSET variant=$VARIANT
./b2 libs/gil/numeric/test toolset=$TOOLSET variant=$VARIANT
./b2 libs/gil/io/test//simple toolset=$TOOLSET variant=$VARIANT
fi
- |-
echo "using $TOOLSET : : $COMPILER : <cxxflags>-std=$CXXSTD ;" > ~/user-config.jam
- travis_retry libs/gil/.ci/build-and-test.sh

after_success:
# Upload docs only when building upstream.
- |
if [ "$DOC" -a \
"$TRAVIS_REPO_SLUG" = "boostorg/gil" -a \
"$TRAVIS_PULL_REQUEST" = "false" ]; then
export GH_TOKEN
cd libs/gil
.ci/upload_docs.sh
# Upload docs only when building upstream.
- |
if [ "$DOC" -a \
"$TRAVIS_REPO_SLUG" = "boostorg/gil" -a \
"$TRAVIS_PULL_REQUEST" = "false" ]; then
export GH_TOKEN
cd libs/gil
.ci/upload_docs.sh
fi
notifications:
email:
on_success: always
Expand Down
34 changes: 33 additions & 1 deletion Jamfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,41 @@
# Boost.GIL (Generic Image Library)
#
# Copyright (c) 2013-2017 Vinnie Falco (vinnie dot falco at gmail dot com)
# Copyright (c) 2018 Mateusz Loskot <mateusz@loskot.net>
#
# Use, modification and distribution is subject to the Boost Software License,
# Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at
# http://www.boost.org/LICENSE_1_0.txt)

# TODO: Remove -Wno-unused below once tests have been cleaned up, and no longer use of assert() macro, etc.

variant ubsan_integer
: release
:
<cxxflags>"-std=c++11 -Wno-unused -fstrict-aliasing -fno-omit-frame-pointer -fsanitize=integer -fno-sanitize-recover=integer -fsanitize-blacklist=libs/gil/.ci/blacklist.supp"
<linkflags>"-fsanitize=integer"
<debug-symbols>on
<define>BOOST_USE_ASAN=1
;

variant ubsan_nullability
: release
:
<cxxflags>"-std=c++11 -Wno-unused -fstrict-aliasing -fno-omit-frame-pointer -fsanitize=nullability -fno-sanitize-recover=nullability -fsanitize-blacklist=libs/gil/.ci/blacklist.supp"
<linkflags>"-fsanitize=nullability"
<debug-symbols>on
<define>BOOST_USE_ASAN=1
;

variant ubsan_undefined
: release
:
<cxxflags>"-std=c++11 -Wno-unused -fstrict-aliasing -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=undefined -fsanitize-blacklist=libs/gil/.ci/blacklist.supp"
<linkflags>"-fsanitize=undefined"
<debug-symbols>on
<define>BOOST_USE_ASAN=1
;

project boost-gil
:
requirements
Expand All @@ -17,7 +47,9 @@ project boost-gil
<toolset>intel:<debug-symbols>off
<toolset>gcc:<cxxflags>"-std=c++11 -pedantic -fstrict-aliasing -Wconversion -Wextra -Wfloat-equal -Wshadow -Wsign-promo -Wstrict-aliasing -Wunused-parameter "
<toolset>darwin:<cxxflags>"-std=c++11 -pedantic -fstrict-aliasing -Wconversion -Wextra -Wfloat-equal -Wshadow -Wsign-promo -Wstrict-aliasing -Wunused-parameter "
<toolset>clang:<cxxflags>"-std=c++11 -pedantic -fstrict-aliasing -Wconversion -Wextra -Wfloat-equal -Wshadow -Wsign-promo -Wstrict-aliasing -Wunused-parameter -Wsign-conversion"
# variant filter for clang is necessary to allow ubsan_* variants declare distinct set of <cxxflags>
<toolset>clang,<variant>debug:<cxxflags>"-std=c++11 -pedantic -fstrict-aliasing -Wconversion -Wextra -Wfloat-equal -Wshadow -Wsign-promo -Wstrict-aliasing -Wunused-parameter -Wsign-conversion"
<toolset>clang,<variant>release:<cxxflags>"-std=c++11 -pedantic -fstrict-aliasing -Wconversion -Wextra -Wfloat-equal -Wshadow -Wsign-promo -Wstrict-aliasing -Wunused-parameter -Wsign-conversion"
;

build-project example ;
Expand Down

0 comments on commit 14a7c20

Please sign in to comment.