How to pass CC or CFLAGS #1072

juanitotc · 2017-12-23T11:42:46Z

Is there a way to pass CC or CFLAGS so that they are used to build for the downloaded/built-in libs and ghb/handbrake-cli?

bradleysepos · 2017-12-23T12:51:56Z

It would be helpful to know what you're trying to do. See ./configure --help for some compiler options.

juanitotc · 2017-12-23T14:46:42Z

I tried:

CC="gcc -flto -fuse-linker-plugin -mtune=generic -Os -pipe" CXX="g++ -flto -fuse-linker-plugin -mtune=generic -Os -pipe -fno-exceptions -fno-rtti" ./configure --prefix=/usr/local

and

CFLAGS="-flto -fuse-linker-plugin -mtune=generic -Os -pipe" CXXFLAGS="-flto -fuse-linker-plugin -mtune=generic -Os -pipe -fno-exceptions -fno-rtti" ./configure --prefix=/usr/local

..but it didn't seem to use either CC or CFLAGS during the build

bradleysepos · 2017-12-23T15:57:44Z

Ah. You can probably do:

./configure
cd build
for CONTRIB in ffmpeg libbluray libdvdread libdvdnav libvpx x265; do
    make -j=$(nproc) "${CONTRIB}" CFLAGS='-flto -fuse-linker-plugin -mtune=generic -Os -pipe' CXXFLAGS='-flto -fuse-linker-plugin -mtune=generic -Os -pipe -fno-exceptions -fno-rtti'
done
make -j=$(nproc)

juanitotc · 2017-12-24T06:18:47Z

Hmm, thanks - I should have thought of that...

There seems to be a problem with libvpx - with plain "make", the build succeeds.

With make CFLAGS='-flto -fuse-linker-plugin -mtune=generic -Os -pipe' CXXFLAGS='-flto -fuse-linker-plugin -mtune=generic -Os -pipe -fno-exceptions -fno-rtti', the build fails with many errors of the form:

mkdir -p vpx_ports/
/usr/src/HandBrake-1.0.7/build/contrib/libvpx/libvpx-1.5.0/build/make/gen_asm_deps.sh
--build-pfx= --depfile=vpx_ports/emms.asm.d -f elf64 -I./ -I"/usr/src/HandBrake-1.0.7/build/contrib/libvpx/libvpx-1.5.0"/ vpx_ports/emms.asm > vpx_ports/emms.asm.d
mkdir -p ./
/usr/local/bin/gcc -flto -fuse-linker-plugin -mtune=generic -Os -pipe -M vpx_config.c | sed -e 's;^([a-zA-Z0-9_]).o;vpx_config.c.o vpx_config.c.d;' > vpx_config.c.d
mkdir -p vp9/encoder/x86/
/usr/local/bin/gcc -flto -fuse-linker-plugin -mtune=generic -Os -pipe -flto -fuse-linker-plugin -mtune=generic -Os -pipe -M vp9/encoder/x86/vp9_error_intrin_avx2.c | sed -e 's;^([a-zA-Z0-9_]).o;vp9/encoder/x86/vp9_error_intrin_avx2.c.o vp9/encoder/x86/vp9_error_intrin_avx2.c.d;' > vp9/encoder/x86/vp9_error_intrin_avx2.c.d
vp9/encoder/x86/vp9_error_intrin_avx2.c:13:24: fatal error: ./vp9_rtcd.h: No such file or directory
#include "./vp9_rtcd.h"
^
compilation terminated.

bradleysepos · 2017-12-24T10:00:06Z

I doubt anyone will offer much help with custom compilation options; that's definitely something that you should figure out if you want a custom build. At minimum, I think you should try compiling the latest master branch instead of 1.0.x, which is stated to be replaced with the former very soon.

You may want to look at https://github.com/HandBrake/HandBrake/blob/master/contrib/libvpx/module.defs and similar to see how HandBrake's build system works. You can see how to override environment variables like CFLAGS (<contrib>.CONFIGURE.env.CFLAGS) in the x264 module: https://github.com/HandBrake/HandBrake/blob/master/contrib/x264/module.defs

StatusCode404 · 2021-06-22T06:18:25Z

@juanitotc I know this is a 4 year old thread, but just wanted to chime in here that I've just gone through the endeavour and that using CFLAGS and CXXFLAGS are NOT the recommended way of passing compiler flags for HandBrake due to its idiosyncracies.
This is described here... https://github.com/griff/HandBrake/blob/master/doc/BUILD-Linux

Handbrake is just a front-end to a LOT of "crontrib". To see how each contrib module is built, you can leverage the "make" reports for each contrib in the build or destination directory, before making them.

e.g.

Let's say you're building HandBrake in a folder called "build", then:

$  cd ./build
$  make report.help
  AVAILABLE MAKEFILE VARS REPORTS
  ----------------------------------------------------------------
  report.main            global general vars
  report.gcc             global gcc vars (inherited by module GCC)
  report.var             usage: make report.var name=VARNAME
  x265.report            X265-scoped vars
  x265_8.report          X265_8-scoped vars
  x265_10.report         X265_10-scoped vars
  x265_12.report         X265_12-scoped vars
  libdav1d.report        LIBDAV1D-scoped vars
  ffmpeg.report          FFMPEG-scoped vars
  libdvdread.report      LIBDVDREAD-scoped vars
  libdvdnav.report       LIBDVDNAV-scoped vars
  libbluray.report       LIBBLURAY-scoped vars
  nvenc.report           NVENC-scoped vars
  libhb.report           LIBHB-scoped vars
  test.report            TEST-scoped vars
  gtk.report             GTK-scoped vars
  pkg.report             PKG-scoped vars

On each line, first column above, you'll see each report. you can then access the reports by
$ make <report_name>

Where you replace <report_name> with the report you want.

It is important to note, there's a hierarchy and inheritance to the above even within each report.
report.gcc can be taken as the root for gcc flags.

In my case, I chose to configure the build using "speed" previously...
$ ./configure --build=build --optimize=speed

Which maps to
GCC.args.O.speed
in the report.gcc

Another important key in that report is
GCC.args.extra
which basically 'may' append extra compiler option flags after the former. As you know with gcc, if there's a conflict between options, the last one is used. Since we can't tell easily enough if the many modules are using one or the other or both, I tend to ensure whatever is in the first, is also in the latter. But the latter can contain more! You can see the defaults by checking the report.

You can override the above by creating a text file configuration called "custom.defs" in the root of the handbrake source folder (if you git cloned it, then the top folder of HandBrake where you basically do your git pull commands).

/HandBrake$ ls -h
AUTHORS.markdown  CODE_OF_CONDUCT.md  CONTRIBUTING.md  download  gtk      macosx         pkg              scripts      THANKS.markdown
build             configure           COPYING          gccFDO    libhb    make           preset           SECURITY.md  TRANSLATION.markdown
build2            contrib             custom.defs    graphics  LICENSE  NEWS.markdown  README.markdown  test         win

I do FDO (feedback-directed optimisation aka FDO aka PGO) in mine so I usually build first with custom.defs defined as

$ cat custom.defs 
GCC.args.O.speed = -march=native -O3 -pipe -fprofile-generate=../gccFDO -fprofile-update=prefer-atomic

GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -fprofile-generate=../gccFDO -fprofile-update=prefer-atomic

Then run HandBrake transcoding several videos with varying codecs for a couple of days to generate profiles.
Then I use the generated profiles by using...

$ cat custom.defs 
GCC.args.O.speed = -march=native -O3 -pipe -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training

GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training

on a brand new build directory.

You can fine tune the compiler flags and optimisation for each module by using the reporting process I described above for each module and overriding the keys by quoting them in the custom_defs file with the values you want, just like the example above for the GCC.args defaults.

For all of the above to work, remember not to have exported CFLAGS or CXXFLAGS. You can check what flags you have setup in your bash session by:
$ export -p | grep FLAGS

LTO + FDO:
Link Time Optimisation LTO with FDO are excellent together as can be easilly be researched on google for many programs and benchmarks.
Unfortunately when setting LTO as the default in GCC.args.* using -flto or setting LTO for the FFMPEG module; fails the whole build.
LTO can be added however to all other modules!
This is my custom.defs...

$ cat custom.defs
GCC.args.O.speed = -march=native -O3 -pipe -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
X265.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
X265.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
X265_8.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
X265_8.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
X265_10.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
X265_10.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
X265_12.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
X265_12.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBHB.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBHB.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBDAV1D.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBDAV1D.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
GTK.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
GTK.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBDVDREAD.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBDVDREAD.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBDVDNAV.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBDVDNAV.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBBLURAY.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
LIBBLURAY.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
TEST.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
TEST.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
NVENC.GCC.args.O.speed = -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training
NVENC.GCC.args.extra = -mfpmath=sse -march=native -O3 -pipe -flto -fprofile-use=../gccFDO -fprofile-correction -fprofile-partial-training

20% Improvrement with LTO + PGO from above!
My benchmarks of transcoding a 4k x265 source to 1440p x265 using:

nlmeans filter ultralight
mkv
10bit x265
RF=24
Constant Quality
Variable Framerate
DTS-MA 7.1 to AAC 7.1 512kbits
on ivy-bridge i7-3770 (old cpu but with AVX) was ~20% faster over stock HandBrake from Ubuntu 20.04 repos!!!

StatusCode404 · 2021-07-07T15:17:37Z

I edited the above custom.defs that I use to include "-fprofile-partial-training"

StatusCode404 · 2021-07-07T15:24:55Z

I have added the above to unix-stackexchange here: https://unix.stackexchange.com/a/655387/134742

StatusCode404 · 2021-07-21T09:49:03Z

Updated the training custom.defs options by adding:
-fprofile-update=prefer-atomic

That will help ensure no clobbering happens of profile data from multiple threads.

StatusCode404 · 2021-08-01T02:49:27Z

Added new answer for v1.4.0 in stackexchange...
https://unix.stackexchange.com/a/662803/134742

It provides the process for both GCC and LLVM/CLANG/LLD in Linux.

bradleysepos closed this as completed Dec 24, 2017

sr55 mentioned this issue Jun 25, 2021

Ability to add Custom build flags. #2934

Closed

StatusCode404 mentioned this issue Jul 21, 2021

Add build system documentation. HandBrake/HandBrake-docs#32

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to pass CC or CFLAGS #1072

How to pass CC or CFLAGS #1072

juanitotc commented Dec 23, 2017

bradleysepos commented Dec 23, 2017

juanitotc commented Dec 23, 2017

bradleysepos commented Dec 23, 2017

juanitotc commented Dec 24, 2017

bradleysepos commented Dec 24, 2017

StatusCode404 commented Jun 22, 2021 •

edited

StatusCode404 commented Jul 7, 2021

StatusCode404 commented Jul 7, 2021

StatusCode404 commented Jul 21, 2021

StatusCode404 commented Aug 1, 2021 •

edited

How to pass CC or CFLAGS #1072

How to pass CC or CFLAGS #1072

Comments

juanitotc commented Dec 23, 2017

bradleysepos commented Dec 23, 2017

juanitotc commented Dec 23, 2017

bradleysepos commented Dec 23, 2017

juanitotc commented Dec 24, 2017

bradleysepos commented Dec 24, 2017

StatusCode404 commented Jun 22, 2021 • edited

StatusCode404 commented Jul 7, 2021

StatusCode404 commented Jul 7, 2021

StatusCode404 commented Jul 21, 2021

StatusCode404 commented Aug 1, 2021 • edited

StatusCode404 commented Jun 22, 2021 •

edited

StatusCode404 commented Aug 1, 2021 •

edited