Clang produces much larger binaries than GCC

[DoDoENT]

Since now GCC is deprecated and Clang is new default, we would like to see Clang building binaries that are of similar size (when using same optimisation parameters).

Currently, binaries produced by Clang are up to 15% larger than binaries produced by GCC.

Here are sizes of our barcode library when compiled with GCC 4.9 and when compiled with default Clang in r11 (both use gnustl_static as c++ library):

$ ls -lh libs_clang/*
libs_clang/arm64-v8a:
total 6616
-rwxr-xr-x  1 dodo  staff   3.2M Mar 15 14:12 libBlinkBarcode.so

libs_clang/armeabi-v7a:
total 4904
-rwxr-xr-x  1 dodo  staff   2.4M Mar 15 14:12 libBlinkBarcode.so
[dodo@mujcek: androidJni (feature/ndk-r11)]$ ls -lh libs_gcc/*
libs_gcc/arm64-v8a:
total 6096
-rwxr-xr-x  1 dodo  staff   3.0M Mar 15 14:07 libBlinkBarcode.so

libs_gcc/armeabi-v7a:
total 4304
-rwxr-xr-x  1 dodo  staff   2.1M Mar 15 14:07 libBlinkBarcode.so

Can this be looked into in r11b or in r12?

Compile options used for Clang (armeabi-v7a target):

 -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -Wno-invalid-command-line-argument -Wno-unused-command-line-argument -no-canonical-prefixes -fno-integrated-as -target armv7-none-linux-androideabi -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fno-exceptions -fno-rtti -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -mfpu=neon -Wno-parentheses-equality -Wno-invalid-command-line-argument -Wno-unused-command-line-argument -Qunused-arguments -Wno-unknown-warning-option -Wno-psabi -Wno-unknown-pragmas -Werror -Wno-error=deprecated-declarations -Wno-error=cpp -Wall -Os -Wconversion -Wno-sign-conversion -Wno-write-strings -Wno-pragmas -Wvla -Wformat-security -Wextra -Wno-unused-parameter -Wno-ignored-qualifiers -Wno-type-limits -fvisibility=hidden -ffunction-sections -fstack-protector-all -Wa,--noexecstack -Wformat -Werror=format-security  -std=c++11 -Wno-overloaded-virtual -fvisibility-inlines-hidden  -fexceptions

and GCC (armeabi-v7a target - some warning flags are different or missing because those are not supported under GCC):

-fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -fno-exceptions -fno-rtti -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -mfpu=neon -Wno-psabi -Wno-unknown-pragmas -Werror -Wno-error=deprecated-declarations -Wno-error=cpp -Wall -Os -Wconversion -Wno-sign-conversion -Wno-write-strings -Wno-pragmas -Wvla -Wformat-security -Wextra -Wno-unused-parameter -Wno-ignored-qualifiers -Wno-type-limits -fvisibility=hidden -ffunction-sections -fstack-protector-all -Wa,--noexecstack -Wformat -Werror=format-security  -std=c++11 -Wno-overloaded-virtual -fvisibility-inlines-hidden  -fexceptions

Generally, you can see that both Clang and GCC use -Os optimisation flag.

If achieving the same binary size as with GCC is not possible, we would like at least then better performance of binary produced by Clang. We need to somehow explain to our clients why our library will be 15% larger when we start using Clang.

[DanAlbert]

Do you have evidence of performance not being better for these? This might just be down to inlining decisions.

[DoDoENT]

@DanAlbert, I haven't measured performance because I am currently just playing around with clang while doing other things. On SGS6 scanning performance of our OCR system is comparable to binary generated by GCC. I still haven't tested on slower phone and enabled timings to determine if there is noticeable performance improvement. So far I can only see larger binary size. I opened this issue because when creating a next release with Clang and r11, I need to somehow justify the larger binary to my clients.

Are there any flags besides -Os I could try with clang to see if it will produce smaller binary? Our code uses STL and template classes a lot.

[stephenhines]

I see that your flags show -g for both clang and gcc. Can you try comparing the stripped sizes of your binaries? Do you actually ship (or intend to ship) binaries that still retain debuginfo? It is also peculiar to see some combinations of flags like both -fno-exceptions and -fexceptions in the same command line. You should probably only specify the one you need. -fno-exceptions might also help cut down on the overall size.

[DoDoENT]

@stephenhines, I've compared the sizes of stripped binaries (the ones produced inside lib folder. I need -g to ensure debug symbols will appear in unstripped binaries inside obj folder - I use those unstripped binaries to retrace binary stack traces reported by clients.

I do have some conflicting flags, but this is due to build system - I have a macro which adds common flags for all files and then specific files "overwrite" those flags by appending new ones (compiler always uses the last one specified in such cases). Unfortunately, I need to compile with -fexceptions because a 3rd party library I am using requires it.

I only noticed that when same compile flags are used with both Clang and GCC, Clang produces larger binary. I think this is an issue.

[DoDoENT]

Our PhotoMath native library has difference of over 1 MB between GCC and Clang version:

$ ls -l libs_clang/arm64-v8a/libBlinkPhotomath.so 
-rwxr-xr-x  1 dodo  staff  14514568 Mar 16 13:04 libs_clang/arm64-v8a/libBlinkPhotomath.so
$ ls -l libs_gcc/arm64-v8a/libBlinkPhotomath.so 
-rwxr-xr-x  1 dodo  staff  13433848 Mar 16 13:10 libs_gcc/arm64-v8a/libBlinkPhotomath.so

This is becoming a serious issue for us.

[DanAlbert]

From some analysis I did when we made this decision for the platform:

It seems our 32-bit ARM devices are typically 1MB larger (out of 600MB-900MB total), whereas volantis is slightly smaller, and fugu is 15MB smaller. Strangely, sprout_b is also slightly smaller.

I think the ARM32 numbers are small enough to not bother anyone, especially given that it looks like it will be a win for the new devices since they're aarch64.

But yes, the group that did grow is most likely the vast majority of your users. Just wanted to provide the data that this isn't actually always the case. Having examples of cases where we do have the problem is very helpful though, so we'll point someone at your libraries when they look in to this.

[DoDoENT]

Thanks @DanAlbert, we are still building against GNU STL. We still haven't tried libc++ because lot of dependencies we need are still compiled against GNU STL. Is your data based on clang+gnustl vs gcc+gnustl or clang+libc++ vs gcc+gnustl or are you referring to pure C/C++ code (i.e. no STL)?

[DanAlbert]

The Android platform uses libc++ exclusively, so that was a Clang+libc++ vs GCC+libc++. We only have to keep one copy of libc++.so though, so that cuts down the size a but compared to the relative cost for an app.

Note that it's a shared libc++, and it's actually a different source base than the one in the NDK. the one in the NDK needs a lot of work. Your libs would most likely be larger with libc++ right now because there's no clean way to get the demangler out if your app.

Taking libc++ so it's more reliable and we're comfortable recommending it as the default is something we're going to be working on this year, but until we really dig in it's hard to make a schedule estimate.

[fornwall]

Taking libc++ so it's more reliable and we're comfortable recommending it as the default is something we're going to be working on this year, but until we really dig in it's hard to make a schedule estimate.

So for now (NDK r11) you recommend the GNU STL runtime as a default for most NDK users (regardless if one uses gcc or clang)?

[DoDoENT]

@DanAlbert, indeed. Clang+libc++ combination produced over 2 MB larger binary for our app than GCC+stdc++ combination (6.5 MB instead of 4.5 MB). The Clang+libc++ was only marginally faster (less than 5%).

[DoDoENT]

@fornwall, I would recommend using GNU STL because we just found out an issue #31 with libc++ on Android 2.3.

[bog-dan-ro]

I can confirm this behavior, Qt libs compiled with clang are +10% larger using the same compiler flags.
And yes, I compared stripped libs.

[DanAlbert]

@fornwall, correct.

[DanAlbert]

btw we're trying to get some people to look in to code size issues. One thing that was found quickly was that GCC still defaults to -Bsymbolic when linking. This results in a smaller PLT, but means that symbols cannot be preempted (such as a replacement implementation of operator new). It probably can't account for the full 10% size increase, but it will affect the size a bit.

FYI, this is also being discussed on #133.

[enh]

is this bug actually different from the -Os vs -Oz issue, or is it obsolete?

[chri7325]

Unfortunately for us, this is the only reason why we can't move to clang. We really really want to move to it for better C++ support but our final binaries end up being way too big. We also use Qt for our Qt api and if they are forced to clang along with us with this much larger size, then any app would most likely go over the 100mb limit.

[DanAlbert]

Probably should have duped these the other way around, but #133 has a lot more data on it so I'm closing this in favor of that one.