Skip to content

[ML] Use hardened compiler options to build 3rd party libraries #453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 8, 2019
Merged

[ML] Use hardened compiler options to build 3rd party libraries #453

merged 2 commits into from
Apr 8, 2019

Conversation

droberts195
Copy link
Contributor

This change adds the stack protector, relro and fortify source
options to the commands used to build the 3rd party libraries
we use on Linux.

(We already used these options when building our own code.)

@droberts195
[ML] Use hardened compiler options to build 3rd party libraries
e1489a2 
This change adds the stack protector, relro and fortify source
options to the commands used to build the 3rd party libraries
we use on Linux.

(We already used these options when building our own code.)
droberts195
droberts195 commented Apr 5, 2019
View reviewed changes
dev-tools/docker/linux_image/Dockerfile
ENV PATH /usr/local/gcc73/bin:/usr/bin:/bin:/usr/sbin:/sbin

# For compiling in C++14 mode
# For compiling in C++14 mode with hardening and optimisation
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that optimisation was not added in this commit. If CFLAGS is not specified then the configure scripts use a default value for it, and this invariably includes optimisation. It's only because we are now specifying CFLAGS that we have to explicitly add the optimisation option now.

dimitris-athanasiou
dimitris-athanasiou reviewed Apr 7, 2019
View reviewed changes
build-setup/linux.md
```
./b2 -j6 --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS
sudo env PATH="$PATH" ./b2 install --prefix=/usr/local/gcc73 --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS
./b2 -j6 --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS define=_FORTIFY_SOURCE=2 cxxflags=-std=gnu++14 cxxflags=-fstack-protector linkflags=-Wl,-z,relro linkflags=-Wl,-z,now
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we have a second define here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It’s fine to have two (or more) defines here. They become multiple -D arguments on the compiler command line, which is a very usual thing to have.

build-setup/linux.md
./b2 -j6 --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS
sudo env PATH="$PATH" ./b2 install --prefix=/usr/local/gcc73 --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS
./b2 -j6 --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS define=_FORTIFY_SOURCE=2 cxxflags=-std=gnu++14 cxxflags=-fstack-protector linkflags=-Wl,-z,relro linkflags=-Wl,-z,now
sudo env PATH="$PATH" ./b2 install --prefix=/usr/local/gcc73 --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS define=_FORTIFY_SOURCE=2 cxxflags=-std=gnu++14 cxxflags=-fstack-protector linkflags=-Wl,-z,relro linkflags=-Wl,-z,now
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

dev-tools/vagrant/linux/provision.sh
echo " Building..."
./b2 -j$NUMCPUS --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS > b2_make.log 2>&1
./b2 install --prefix=/usr/local/gcc73 --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS > b2_make_install.log 2>&1
./b2 -j$NUMCPUS --layout=versioned --disable-icu pch=off optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS define=_FORTIFY_SOURCE=2 cxxflags=-std=gnu++14 cxxflags=-fstack-protector linkflags=-Wl,-z,relro linkflags=-Wl,-z,now > b2_make.log 2>&1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

second define in this and the following line is well (if it is an issue)

dimitris-athanasiou
dimitris-athanasiou approved these changes Apr 8, 2019
View reviewed changes
Copy link
Contributor

@dimitris-athanasiou dimitris-athanasiou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@droberts195 droberts195 merged commit 6dc74bf into elastic:master Apr 8, 2019
@droberts195 droberts195 deleted the harden_3rd_party branch April 8, 2019 07:56
droberts195 added a commit to droberts195/ml-cpp that referenced this pull request Apr 8, 2019
@droberts195
[ML] Use hardened compiler options to build 3rd party libraries
edb8506 
This change adds the stack protector, relro and fortify source
options to the commands used to build the 3rd party libraries
we use on Linux.

(We already used these options when building our own code.)

Backport of elastic#453
@droberts195 droberts195 mentioned this pull request Apr 8, 2019
Merged
droberts195 added a commit that referenced this pull request Apr 8, 2019
@droberts195
[ML] Use hardened compiler options to build 3rd party libraries (#455)
0122f28 
This change adds the stack protector, relro and fortify source
options to the commands used to build the 3rd party libraries
we use on Linux.

(We already used these options when building our own code.)

Backport of #453
droberts195 added a commit to droberts195/ml-cpp that referenced this pull request Apr 24, 2019
@droberts195
[ML] Use hardening options for gcc 7.3 build
e18ff54 
This is a follow on to elastic#453

The stack protector, relro and fortify source options
need to be used when building gcc, because we redistribute
two libraries that are built as part of it, namely
libgcc_s.so.1 and libstdc++.so.6.
@droberts195 droberts195 mentioned this pull request Apr 24, 2019
Merged
droberts195 added a commit that referenced this pull request Apr 25, 2019
@droberts195
[ML] Use hardening options for gcc 7.3 build (#470)
235e692 
This is a follow on to #453

The stack protector, relro and fortify source options
need to be used when building gcc, because we redistribute
two libraries that are built as part of it, namely
libgcc_s.so.1 and libstdc++.so.6.
droberts195 added a commit to droberts195/ml-cpp that referenced this pull request Apr 25, 2019
@droberts195
[ML] Use hardening options for gcc 7.3 build
5799df9 
This is a follow on to elastic#453

The stack protector, relro and fortify source options
need to be used when building gcc, because we redistribute
two libraries that are built as part of it, namely
libgcc_s.so.1 and libstdc++.so.6.

Backport of elastic#470
@droberts195 droberts195 mentioned this pull request Apr 25, 2019
Merged
droberts195 added a commit that referenced this pull request Apr 25, 2019
@droberts195
[ML] Use hardening options for gcc 7.3 build (#471)
9a87be5 
This is a follow on to #453

The stack protector, relro and fortify source options
need to be used when building gcc, because we redistribute
two libraries that are built as part of it, namely
libgcc_s.so.1 and libstdc++.so.6.

Backport of #470
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dimitris-athanasiou dimitris-athanasiou dimitris-athanasiou approved these changes

Assignees
No one assigned
Labels
>enhancement :ml v7.2.0 v8.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@droberts195 @dimitris-athanasiou