Symbols blow up binary size to an unreasonable degree on x86_64-unknown-linux-gnu

[crumblingstatue]

Standard hello world program generated with cargo new --bin mytest

fn main() {
    println!("Hello, world!");
}

Built with cargo build --release, the resulting binary size is 4270408 bytes.
This sounds unreasonably large for a hello world program.

After strip --strip-all, we get 449224.

The non-stripped version is almost 10 times larger than the stripped version. Why?

This is with rustc 1.23.0-nightly (fa26421f5 2017-11-15), installed through rustup.

$ rustc -vV
rustc 1.23.0-nightly (fa26421f5 2017-11-15)
binary: rustc
commit-hash: fa26421f56e385b1055e65b29a55b36bb2eae23e
commit-date: 2017-11-15
host: x86_64-unknown-linux-gnu
release: 1.23.0-nightly
LLVM version: 4.0

Here is stable rustc 1.21.0 (3b72af97e 2017-10-09) for good measure:
Normal: 4059968
Stripped: 404096

This time, the initial size is lower, but the stripped size is even smaller, making the non-stripped version slightly more than 10 times larger.

(As a sidenote, I've been noticing that binary sizes for the same program have been steadily increasing with every new rustc, at least on x86_64-unknown-linux-gnu, but that's a separate issue, I suppose)

[kennytm]

On x86_64-apple-darwin:

Version	Size	After strip -x
1.23.0-nightly (fa26421f5 2017-11-15)	525348	377348
1.23.0-nightly (ff0f5de3b 2017-11-14)	525348	(not checked)
1.22.0-beta.3 (cc6ed0640 2017-11-13)	520164	373140
1.21.0 (3b72af97e 2017-10-09)	452684	310548
1.15.1 (021bd294c 2017-02-08)	390608	268696
1.14.0 (e8a012324 2016-12-16)	408304	285392
1.13.0 (021bd294c 2017-02-08)	348872	277232

The size is definitely growing but not something that increases by 10×.

---

On x86_64-pc-windows-msvc:

Version	Size
1.23.0-nightly (fa26421f5 2017-11-15)	134144
1.21.0 (3b72af97e 2017-10-09)	130048

Probably a Linux-only problem.

[crumblingstatue]

@kennytm updated OP with rustc -vV info.

[kennytm]

Repro'd on Linux with stable. (Not sure about the BSDs)

Version	Size	After strip
1.21.0 (3b72af97e 2017-10-09)	4062904	404184
1.15.1 (021bd294c 2017-02-08)	3465968	(not checked)
1.14.0 (e8a012324 2016-12-16)	2068056	372256
1.13.0 (2c6933acc 2016-11-07)	647864	364064
1.12.0 (cfcb716cf 2016-07-03)	652216	(not checked)
1.10.0 (cfcb716cf 2016-07-03)	647760	(not checked)
1.5.0 (3d7cd77e4 2015-12-04)	557208	(not checked)
1.0.0 (a59de37e9 2015-05-13)	579688	(not checked)

The initial blow up problem comes from 1.14.0.

---

Comparing the readelf -S output of 1.13.0 and 1.14.0, we find that 1.14.0 has 3 more sections, .debug_pubnames, .debug_macinfo and .debug_pubtypes. The existing debug info sections are also much larger.

readelf -S -W 1.13.0

$ readelf -S -W 1.13.0
There are 42 section headers, starting at offset 0x9d838:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .interp           PROGBITS        0000000000000270 000270 00001c 00   A  0   0  1
  [ 2] .note.ABI-tag     NOTE            000000000000028c 00028c 000020 00   A  0   0  4
  [ 3] .note.gnu.build-id NOTE            00000000000002ac 0002ac 000024 00   A  0   0  4
  [ 4] .gnu.hash         GNU_HASH        00000000000002d0 0002d0 0000b4 00   A  5   0  8
  [ 5] .dynsym           DYNSYM          0000000000000388 000388 0009c0 18   A  6   3  8
  [ 6] .dynstr           STRTAB          0000000000000d48 000d48 00065a 00   A  0   0  1
  [ 7] .gnu.version      VERSYM          00000000000013a2 0013a2 0000d0 02   A  5   0  2
  [ 8] .gnu.version_r    VERNEED         0000000000001478 001478 000110 00   A  6   4  8
  [ 9] .rela.dyn         RELA            0000000000001588 001588 003558 18   A  5   0  8
  [10] .rela.plt         RELA            0000000000004ae0 004ae0 000768 18  AI  5  27  8
  [11] .init             PROGBITS        0000000000005248 005248 00001a 00  AX  0   0  4
  [12] .plt              PROGBITS        0000000000005270 005270 000500 10  AX  0   0 16
  [13] .plt.got          PROGBITS        0000000000005770 005770 000018 00  AX  0   0  8
  [14] .text             PROGBITS        0000000000005790 005790 03fc69 00  AX  0   0 16
  [15] .fini             PROGBITS        00000000000453fc 0453fc 000009 00  AX  0   0  4
  [16] .rodata           PROGBITS        0000000000045440 045440 005031 00   A  0   0 64
  [17] .eh_frame_hdr     PROGBITS        000000000004a474 04a474 00148c 00   A  0   0  4
  [18] .eh_frame         PROGBITS        000000000004b900 04b900 006254 00   A  0   0  8
  [19] .gcc_except_table PROGBITS        0000000000051b54 051b54 002578 00   A  0   0  4
  [20] .tdata            PROGBITS        0000000000254d90 054d90 0000f0 00 WAT  0   0 16
  [21] .init_array       INIT_ARRAY      0000000000254e80 054e80 000010 00  WA  0   0  8
  [22] .fini_array       FINI_ARRAY      0000000000254e90 054e90 000008 00  WA  0   0  8
  [23] .jcr              PROGBITS        0000000000254e98 054e98 000008 00  WA  0   0  8
  [24] .data.rel.ro      PROGBITS        0000000000254ea0 054ea0 002ee8 00  WA  0   0 32
  [25] .dynamic          DYNAMIC         0000000000257d88 057d88 000220 10  WA  6   0  8
  [26] .got              PROGBITS        0000000000257fa8 057fa8 000040 08  WA  0   0  8
  [27] .got.plt          PROGBITS        0000000000258000 058000 000290 08  WA  0   0  8
  [28] .data             PROGBITS        00000000002582a0 0582a0 0001e1 00  WA  0   0 32
  [29] .bss              NOBITS          00000000002584a0 058481 000fd8 00  WA  0   0 32
  [30] .comment          PROGBITS        0000000000000000 058481 000079 01  MS  0   0  1
  [31] .debug_aranges    PROGBITS        0000000000000000 0584fa 0023a0 00      0   0  1
  [32] .debug_info       PROGBITS        0000000000000000 05a89a 00b63f 00      0   0  1
  [33] .debug_abbrev     PROGBITS        0000000000000000 065ed9 00186a 00      0   0  1
  [34] .debug_line       PROGBITS        0000000000000000 067743 002b12 00      0   0  1
  [35] .debug_frame      PROGBITS        0000000000000000 06a258 006a60 00      0   0  8
  [36] .debug_str        PROGBITS        0000000000000000 070cb8 007aee 01  MS  0   0  1
  [37] .debug_loc        PROGBITS        0000000000000000 0787a6 010c95 00      0   0  1
  [38] .debug_ranges     PROGBITS        0000000000000000 08943b 002220 00      0   0  1
  [39] .shstrtab         STRTAB          0000000000000000 09d69d 000198 00      0   0  1
  [40] .symtab           SYMTAB          0000000000000000 08b660 008a30 18     41 1199  8
  [41] .strtab           STRTAB          0000000000000000 094090 00960d 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

readelf -S -W 1.14.0

$ readelf -S -W 1.14.0
There are 45 section headers, starting at offset 0x1f8318:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .interp           PROGBITS        0000000000000270 000270 00001c 00   A  0   0  1
  [ 2] .note.ABI-tag     NOTE            000000000000028c 00028c 000020 00   A  0   0  4
  [ 3] .note.gnu.build-id NOTE            00000000000002ac 0002ac 000024 00   A  0   0  4
  [ 4] .gnu.hash         GNU_HASH        00000000000002d0 0002d0 0000b4 00   A  5   0  8
  [ 5] .dynsym           DYNSYM          0000000000000388 000388 0009c0 18   A  6   3  8
  [ 6] .dynstr           STRTAB          0000000000000d48 000d48 00065a 00   A  0   0  1
  [ 7] .gnu.version      VERSYM          00000000000013a2 0013a2 0000d0 02   A  5   0  2
  [ 8] .gnu.version_r    VERNEED         0000000000001478 001478 000110 00   A  6   4  8
  [ 9] .rela.dyn         RELA            0000000000001588 001588 0035a0 18   A  5   0  8
  [10] .rela.plt         RELA            0000000000004b28 004b28 000768 18  AI  5  27  8
  [11] .init             PROGBITS        0000000000005290 005290 00001a 00  AX  0   0  4
  [12] .plt              PROGBITS        00000000000052b0 0052b0 000500 10  AX  0   0 16
  [13] .plt.got          PROGBITS        00000000000057b0 0057b0 000018 00  AX  0   0  8
  [14] .text             PROGBITS        00000000000057d0 0057d0 041f29 00  AX  0   0 16
  [15] .fini             PROGBITS        00000000000476fc 0476fc 000009 00  AX  0   0  4
  [16] .rodata           PROGBITS        0000000000047740 047740 0052b1 00   A  0   0 64
  [17] .eh_frame_hdr     PROGBITS        000000000004c9f4 04c9f4 0014d4 00   A  0   0  4
  [18] .eh_frame         PROGBITS        000000000004dec8 04dec8 00645c 00   A  0   0  8
  [19] .gcc_except_table PROGBITS        0000000000054324 054324 002700 00   A  0   0  4
  [20] .tdata            PROGBITS        0000000000256d50 056d50 0000f0 00 WAT  0   0 16
  [21] .init_array       INIT_ARRAY      0000000000256e40 056e40 000010 00  WA  0   0  8
  [22] .fini_array       FINI_ARRAY      0000000000256e50 056e50 000008 00  WA  0   0  8
  [23] .jcr              PROGBITS        0000000000256e58 056e58 000008 00  WA  0   0  8
  [24] .data.rel.ro      PROGBITS        0000000000256e60 056e60 002f28 00  WA  0   0 32
  [25] .dynamic          DYNAMIC         0000000000259d88 059d88 000220 10  WA  6   0  8
  [26] .got              PROGBITS        0000000000259fa8 059fa8 000040 08  WA  0   0  8
  [27] .got.plt          PROGBITS        000000000025a000 05a000 000290 08  WA  0   0  8
  [28] .data             PROGBITS        000000000025a2a0 05a2a0 0001e1 00  WA  0   0 32
  [29] .bss              NOBITS          000000000025a4a0 05a481 000ff8 00  WA  0   0 32
  [30] .comment          PROGBITS        0000000000000000 05a481 000079 01  MS  0   0  1
  [31] .debug_aranges    PROGBITS        0000000000000000 05a4fa 0023a0 00      0   0  1
  [32] .debug_pubnames   PROGBITS        0000000000000000 05c89a 01a19a 00      0   0  1
  [33] .debug_info       PROGBITS        0000000000000000 076a34 0716cd 00      0   0  1
  [34] .debug_abbrev     PROGBITS        0000000000000000 0e8101 002763 00      0   0  1
  [35] .debug_line       PROGBITS        0000000000000000 0ea864 02c556 00      0   0  1
  [36] .debug_frame      PROGBITS        0000000000000000 116dc0 006a60 00      0   0  8
  [37] .debug_str        PROGBITS        0000000000000000 11d820 056459 01  MS  0   0  1
  [38] .debug_loc        PROGBITS        0000000000000000 173c79 010c95 00      0   0  1
  [39] .debug_macinfo    PROGBITS        0000000000000000 18490e 000008 00      0   0  1
  [40] .debug_pubtypes   PROGBITS        0000000000000000 184916 00ab8a 00      0   0  1
  [41] .debug_ranges     PROGBITS        0000000000000000 18f4a0 0567a0 00      0   0  1
  [42] .shstrtab         STRTAB          0000000000000000 1f814c 0001c7 00      0   0  1
  [43] .symtab           SYMTAB          0000000000000000 1e5c40 008da8 18     44 1238  8
  [44] .strtab           STRTAB          0000000000000000 1ee9e8 009764 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

Detailed breakdown of size delta for each section

Section	1.13 size	1.14 size	Δ
.interp	28	28	-
.note.ABI-tag	32	32	-
.note.gnu.build-id	36	36	-
.gnu.hash	180	180	-
.dynsym	2496	2496	-
.dynstr	1626	1626	-
.gnu.version	208	208	-
.gnu.version_r	272	272	-
.rela.dyn	13656	13728	+0.53%
.rela.plt	1896	1896	-
.init	26	26	-
.plt	1280	1280	-
.plt.got	24	24	-
.text	261225	270121	+3.41%
.fini	9	9	-
.rodata	20529	21169	+3.12%
.eh_frame_hdr	5260	5332	+1.37%
.eh_frame	25172	25692	+2.07%
.gcc_except_table	9592	9984	+4.09%
.tdata	240	240	-
.init_array	16	16	-
.fini_array	8	8	-
.jcr	8	8
.data.rel.ro	12008	12072	+0.53%
.dynamic	544	544	-
.got	64	64	-
.got.plt	656	656	-
.data	16	16	-
.bss	4056	4088	+0.79%
.comment	121	121	-
.debug_aranges	9120	9120	-
.debug_pubnames	-	106906	new
.debug_info	46655	464589	+895.80%
.debug_abbrev	6250	10083	+61.33%
.debug_line	11026	181590	+1546.95%
.debug_frame	27232	27232	-
.debug_str	31470	355369	+1022.88%
.debug_loc	68757	68757	-
.debug_macinfo	8	8	new
.debug_pubtypes	-	43914	new
.debug_ranges	8736	354208	+3954.58%
.shstrtab	408	455	+11.52%
.symtab	35376	36264	+2.51%
.strtab	38413	38756	+0.89%

[kennytm]

#43392 seems to be the same problem. Since Windows and macOS places the debug symbols externally (*.pdb and *.dSYM respectively), we don't see the size blow up on those platforms.

[est31]

This seems like a very important thing. Size affects linkage time, affects compilation speed.

[kennytm]

From the 1.14.0 release note I believe the cause of the 1.13.0 → 1.14.0 jump is introduced by #37280, where libstd starts to be built with -C debuginfo=1. However the underlying cause is still #43392, that release builds copy the debuginfo from libstd.

[michaelwoerister]

Making the assumption that this is caused by libstd debuginfo, the size difference in percent should become smaller for larger programs, that is, running strip on a big crate should not reduce its size by 90%. Has anybody tried this?

[Michael-F-Bryan]

On mdbook, a non-trivial sized project with about 5.5 kloc of Rust.

$ rustc --version
rustc 1.24.0-nightly (687d3d15b 2018-01-02)
$ cargo build --release
$ du -h ./target/release/mdbook
12M	./target/release/mdbook
$ strip ./target/release/mdbook 
$ du -h ./target/release/mdbook
6.3M	./target/release/mdbook

So stripping debug info goes from 12M to 6.3M.

[whitequark]

The C/C++ toolchain (whether GCC or Clang) on Linux solves this with -gsplit-dwarf, which results in similar products as the *.pdb or *.dSYM on Windows and macOS; to anyone interested in implementing the support, it would be as simple as setting the appropriate LLVM codegen option (as seen in clang):

llvm::TargetOptions &Options;
// ...
if (CodeGenOpts.EnableSplitDwarf)
  Options.MCOptions.SplitDwarfFile = CodeGenOpts.SplitDwarfFile;

[kennytm]

Oh interesting. cc @main-- rust-lang/rfcs#2154.

[main--]

Thanks. I actually knew about this "debug fission" but was not aware that LLVM already supports it. The nice thing about the old .gnu_debuglink is that it's so easy to implement - simply read your debug sections from somewhere else. Debug fission is certainly a more modern approach though. Note that even though that GCC page estimates a 70% reduction from dwo files, it only moves .debug_info, .debug_types, and .debug_str. Most notably, .debug_line remains in the original object (and finally the binary).

How good is toolchain support? Of course gdb reads info from dwo files but GNU addr2line does not. It's either just not implemented yet or not intended to work like that (which would be very disappointing since you need .debug_info for inlining info) - either way, it's a bad sign.

I'm still having a hard time finding pretty much any documentation at all about debug fissure (beyond that one GCC page) but these points lead me to believe that it's not the best approach for what I'm trying to do in rust-lang/rfcs#2154.

[whitequark]

GNU addr2line does not

But... if .debug_line remains in the binary, why would addr2line need to read .dwo files? Is it to handle inlined functions? (I am not familiar with that part of DWARF.)

[main--]

Is it to handle inlined functions?

Yes. .debug_line does not contain inlining info, you need .debug_info for that.

[matklad]

Making the assumption that this is caused by libstd debuginfo, the size difference in percent should become smaller for larger programs, that is, running strip on a big crate should not reduce its size by 90%. Has anybody tried this?

@michaelwoerister this is kind of true, but only kind of... Measurements in https://users.rust-lang.org/t/rust-binary-sizes-once-again/16287/2 for a crate with "with rocket, serde, wiringpi and pam-auth" should reduction 6.5M -> 1.6M. It is not 90%, but, still, is very significant.

Note also an interesting observation that LTO does not affect stripped binaries, but, for non-striped, goes from 6.5M -> 4.7M.

I wonder if there's any actionable items on this issue? Could there be some linker flag for "please, don't link-in the debuginfo into the final binary"?

[michaelwoerister]

That's a good idea, @matklad. ld actually has a --strip-debug flag. We should pass that if the output binary does not request debuginfo.

[michaelwoerister]

If anyone wants to give it a try, the relevant spot in the source code is here:

rust/src/librustc_trans/back/linker.rs

Lines 283 to 285 in 15add36

	fn debuginfo(&mut self) {
	// Don't do anything special here for GNU-style linkers.
	}

This method should add any linker flags relevant to debuginfo.

[axos88]

I also stumbled upon this cross-compiling from OS X to arm-linux: https://users.rust-lang.org/t/rust-binary-sizes-once-again/16287/2