Skip to content

Commit

Permalink
chromium: use official build settings (NixOS#101467)
Browse files Browse the repository at this point in the history
LLD: https://lld.llvm.org/
When you link a large program on a multicore machine, you can expect that LLD runs more than twice as fast as the GNU gold linker. Your mileage may vary, though.
Link-time optimization (LTO) is supported by default.
Some default settings have been tuned for the 21st century. For example, the stack is marked as non-executable by default to tighten security.

LTO & ThinLTO: https://clang.llvm.org/docs/ThinLTO.html
LTO (Link Time Optimization) achieves better runtime performance through whole-program analysis and cross-module optimization. However, monolithic LTO implements this by merging all input into a single module, which is not scalable in time or memory, and also prevents fast incremental compiles. ThinLTO is a new approach that is designed to scale like a non-LTO build, while retaining most of the performance achievement of full LTO.

PGO: https://llvm.org/docs/HowToBuildWithPGO.html https://blog.chromium.org/2020/08/chrome-just-got-faster-with-profile.html
Allows your compiler to better optimize code for how it actually runs. Users report that applying this to Clang and LLVM can decrease overall compile time by 20%.
Because PGO uses real usage scenarios that match the workflows of Chrome users around the world, the most common tasks get prioritized and made faster. Delivers up to 10% faster page loads.

CFI: https://clang.llvm.org/docs/ControlFlowIntegrity.html https://www.chromium.org/developers/testing/control-flow-integrity
Aborts the program upon detecting certain forms of undefined behavior that can potentially allow attackers to subvert the program’s control flow. These schemes have been optimized for performance, allowing developers to enable them in release builds.
By default, a program compiled with CFI will crash with SIGILL if it detects a CFI violation.

Additionally:
Use minizip instead of zlib. Chromium says zlib but actually uses minizip.
Remove old unused workarounds.
Make shell scripts POSIX compliant.
Update documentation URLs.
Prepare for using system libraries.
  • Loading branch information
TredwellGit committed Oct 24, 2020
1 parent 7be0ab8 commit 2bb0110
Showing 1 changed file with 30 additions and 45 deletions.
75 changes: 30 additions & 45 deletions pkgs/applications/networking/browsers/chromium/common.nix
Expand Up @@ -5,7 +5,7 @@
, libevent, expat, libjpeg, snappy
, libpng, libcap
, xdg_utils, yasm, nasm, minizip, libwebp
, libusb1, pciutils, nss, re2, zlib
, libusb1, pciutils, nss, re2

, python2Packages, perl, pkgconfig
, nspr, systemd, kerberos
Expand Down Expand Up @@ -46,8 +46,6 @@ buildFun:

with stdenv.lib;

# see http://www.linuxfromscratch.org/blfs/view/cvs/xsoft/chromium.html

let
jre = jre8; # TODO: remove override https://github.com/NixOS/nixpkgs/pull/89731

Expand All @@ -63,7 +61,7 @@ let
mkGnFlags =
let
# Serialize Nix types into GN types according to this document:
# https://chromium.googlesource.com/chromium/src/+/master/tools/gn/docs/language.md
# https://source.chromium.org/gn/gn/+/master:docs/language.md
mkGnString = value: "\"${escape ["\"" "$" "\\"] value}\"";
sanitize = value:
if value == true then "true"
Expand All @@ -75,6 +73,7 @@ let
toFlag = key: value: "${key}=${sanitize value}";
in attrs: concatStringsSep " " (attrValues (mapAttrs toFlag attrs));

# https://source.chromium.org/chromium/chromium/src/+/master:build/linux/unbundle/replace_gn_files.py
gnSystemLibraries = [
"ffmpeg"
"flac"
Expand All @@ -85,9 +84,6 @@ let
"opus"
"snappy"
"zlib"
# "re2" # fails with linker errors
# "harfbuzz-ng" # in versions over 63 harfbuzz and freetype are being built together
# so we can't build with one from system and other from source
];

opusWithCustomModes = libopus.override {
Expand All @@ -99,11 +95,9 @@ let
libevent expat libjpeg snappy
libpng libcap
xdg_utils minizip libwebp
libusb1 re2 zlib
libusb1 re2
ffmpeg libxslt libxml2
nasm
# harfbuzz # in versions over 63 harfbuzz and freetype are being built together
# so we can't build with one from system and other from source
];

# build paths and release info
Expand Down Expand Up @@ -132,6 +126,7 @@ let
};

nativeBuildInputs = [
llvmPackages.lldClang.bintools
ninja which python2Packages.python perl pkgconfig
python2Packages.ply python2Packages.jinja2 nodejs
gnutar python2Packages.setuptools
Expand Down Expand Up @@ -163,11 +158,24 @@ let
];

postPatch = ''
# remove unused third-party
for lib in ${toString gnSystemLibraries}; do
if [ -d "third_party/$lib" ]; then
find "third_party/$lib" -type f \
\! -path "third_party/$lib/chromium/*" \
\! -path "third_party/$lib/google/*" \
\! -path "third_party/harfbuzz-ng/utils/hb_scoped.h" \
\! -regex '.*\.\(gn\|gni\|isolate\)' \
-delete
fi
done
# Required for patchShebangs (unsupported interpreter directive, basename: invalid option -- '*', etc.):
substituteInPlace native_client/SConstruct \
--replace "#! -*- python -*-" ""
substituteInPlace third_party/harfbuzz-ng/src/src/update-unicode-tables.make \
--replace "/usr/bin/env -S make -f" "/usr/bin/make -f"
substituteInPlace native_client/SConstruct --replace "#! -*- python -*-" ""
if [ -e third_party/harfbuzz-ng/src/src/update-unicode-tables.make ]; then
substituteInPlace third_party/harfbuzz-ng/src/src/update-unicode-tables.make \
--replace "/usr/bin/env -S make -f" "/usr/bin/make -f"
fi
# We want to be able to specify where the sandbox is via CHROME_DEVEL_SANDBOX
substituteInPlace sandbox/linux/suid/client/setuid_sandbox_host.cc \
Expand Down Expand Up @@ -195,48 +203,26 @@ let
sed -i -e '/libpci_loader.*Load/s!"\(libpci\.so\)!"${pciutils}/lib/\1!' \
gpu/config/gpu_info_collector_linux.cc
sed -i -re 's/([^:])\<(isnan *\()/\1std::\2/g' \
chrome/browser/ui/webui/engagement/site_engagement_ui.cc
sed -i -e '/#include/ {
i #include <algorithm>
:l; n; bl
}' gpu/config/gpu_control_list.cc
# Allow to put extensions into the system-path.
sed -i -e 's,/usr,/run/current-system/sw,' chrome/common/chrome_paths.cc
patchShebangs .
# use our own nodejs
mkdir -p third_party/node/linux/node-linux-x64/bin
ln -s $(which node) third_party/node/linux/node-linux-x64/bin/node
ln -s "$(command -v node)" third_party/node/linux/node-linux-x64/bin/node
# Allow building against system libraries in official builds
sed -i 's/OFFICIAL_BUILD/GOOGLE_CHROME_BUILD/' tools/generate_shim_headers/generate_shim_headers.py
# remove unused third-party
# in third_party/crashpad third_party/zlib contains just a header-adapter
for lib in ${toString gnSystemLibraries}; do
find -type f -path "*third_party/$lib/*" \
\! -path "*third_party/crashpad/crashpad/third_party/zlib/*" \
\! -path "*third_party/$lib/chromium/*" \
\! -path "*third_party/$lib/google/*" \
\! -path "*base/third_party/icu/*" \
\! -path "*base/third_party/libevent/*" \
\! -regex '.*\.\(gn\|gni\|isolate\|py\)' \
-delete
done
'' + optionalString stdenv.isAarch64 ''
substituteInPlace build/toolchain/linux/BUILD.gn \
--replace 'toolprefix = "aarch64-linux-gnu-"' 'toolprefix = ""'
'' + optionalString stdenv.cc.isClang ''
mkdir -p third_party/llvm-build/Release+Asserts/bin
ln -s ${stdenv.cc}/bin/clang third_party/llvm-build/Release+Asserts/bin/clang
ln -s ${stdenv.cc}/bin/clang++ third_party/llvm-build/Release+Asserts/bin/clang++
ln -s ${llvmPackages.llvm}/bin/llvm-ar third_party/llvm-build/Release+Asserts/bin/llvm-ar
'';

gnFlags = mkGnFlags ({
use_lld = false;
use_gold = stdenv.buildPlatform.is64bit; # ld.gold outs-of-memory on i686
gold_path = "${stdenv.cc}/bin";
custom_toolchain = "//build/toolchain/linux/unbundle:default";
host_toolchain = "//build/toolchain/linux/unbundle:default";
is_official_build = true;
is_debug = false;

proprietary_codecs = false;
Expand Down Expand Up @@ -295,8 +281,7 @@ let
# This is to ensure expansion of $out.
libExecPath="${libExecPath}"
python build/linux/unbundle/replace_gn_files.py \
--system-libraries ${toString gnSystemLibraries}
python build/linux/unbundle/replace_gn_files.py --system-libraries ${toString gnSystemLibraries}
${gnChromium}/bin/gn gen --args=${escapeShellArg gnFlags} out/Release | tee gn-gen-outputs.txt
# Fail if `gn gen` contains a WARNING.
Expand Down

0 comments on commit 2bb0110

Please sign in to comment.