Stripping BOLTed binaries may result in misaligned PT_LOADs

[amharc]

Consider the following simple main.cc file:

int main() {}

Running:

$ clang++ main.cc -o main -Wl,-q
$ llvm-bolt main -o main.bolted
$ llvm-strip -S main.bolted -o main.bolted.stripped

Results in a misaligned PT_LOAD:

  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
...
  LOAD           0x003028 0x0000000000200000 0x0000000000200000 0x200384 0x200384 R E 0x200000

which leads to program crashes at startup:

$ ./main.bolted.stripped
[1]    153575 segmentation fault  ./main.bolted.stripped
$ /lib64/ld-linux-x86-64.so.2 ./main.bolted.stripped
./main.bolted.stripped: error while loading shared libraries: ./main.bolted.stripped: ELF load command address/offset not properly aligned

This is because the new PT_PHDR header was placed at the same offset as the new PT_LOAD containing the modified .text section:

$ llvm-readelf -l -h main.bolted
...
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x200000 0x0000000000200000 0x0000000000200000 0x000310 0x000310 R   0x8
...
  LOAD           0x200000 0x0000000000200000 0x0000000000200000 0x200384 0x200384 R E 0x200000

Which confuses llvm-strip, as it thinks that the PT_LOAD is a child of the PT_PHDR and thus it will disregard the alignment requirements of the (alleged) child.

[llvmbot]

@llvm/issue-subscribers-bolt

[maksfb]

@amharc, do you consider it to be an llvm-bolt issue or llvm-strip issue?

[amharc]

I'm not actually sure: on one hand, my system's version of GNU binutils strip fails to strip a BOLTed binary altogether with error messages like:

strip: main.bolted.stripped.gnu: section `.text' can't be allocated in segment 4
LOAD: .text .eh_frame .eh_frame_hdr
strip: main.bolted.stripped.gnu: section `.eh_frame' can't be allocated in segment 4
LOAD: .text .eh_frame .eh_frame_hdr
strip: main.bolted.stripped.gnu: section `.eh_frame_hdr' can't be allocated in segment 4
LOAD: .text .eh_frame .eh_frame_hdr

and I wasn't able to find a conclusive reference documenting what a strip tool should do in such a case, and making llvm-bolt update the alignment requirements for the new PT_PHDR header would solve the problem.

But on the other hand a reasonable solution would indeed be to teach llvm-strip to propagate child segments' alignment requirements back to the parents before assigning offsets to segments.

[llvmbot]

@llvm/issue-subscribers-tools-llvm-objcopy-strip

Author: Krzysztof Pszeniczny (amharc)

Consider the following simple `main.cc` file: ```c++ int main() {} ```

Running:

$ clang++ main.cc -o main -Wl,-q
$ llvm-bolt main -o main.bolted
$ llvm-strip -S main.bolted -o main.bolted.stripped

Results in a misaligned PT_LOAD:

  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
...
  LOAD           0x003028 0x0000000000200000 0x0000000000200000 0x200384 0x200384 R E 0x200000

which leads to program crashes at startup:

$ ./main.bolted.stripped
[1]    153575 segmentation fault  ./main.bolted.stripped
$ /lib64/ld-linux-x86-64.so.2 ./main.bolted.stripped
./main.bolted.stripped: error while loading shared libraries: ./main.bolted.stripped: ELF load command address/offset not properly aligned

This is because the new PT_PHDR header was placed at the same offset as the new PT_LOAD containing the modified .text section:

$ llvm-readelf -l -h main.bolted
...
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x200000 0x0000000000200000 0x0000000000200000 0x000310 0x000310 R   0x8
...
  LOAD           0x200000 0x0000000000200000 0x0000000000200000 0x200384 0x200384 R E 0x200000

Which confuses llvm-strip, as it thinks that the PT_LOAD is a child of the PT_PHDR and thus it will disregard the alignment requirements of the (alleged) child.

[jh7370]

What version of llvm-strip are you using? There was a commit about 8 months ago to the LLVM main branch (so should be in the most recent release, I believe) that should address this issue, if I understand it correctly.

[llvmbot]

@llvm/issue-subscribers-bolt

[maksfb]

@amharc, do you consider it to be an llvm-bolt issue or llvm-strip issue?

[amharc]

I'm not actually sure: on one hand, my system's version of GNU binutils strip fails to strip a BOLTed binary altogether with error messages like:

strip: main.bolted.stripped.gnu: section `.text' can't be allocated in segment 4
LOAD: .text .eh_frame .eh_frame_hdr
strip: main.bolted.stripped.gnu: section `.eh_frame' can't be allocated in segment 4
LOAD: .text .eh_frame .eh_frame_hdr
strip: main.bolted.stripped.gnu: section `.eh_frame_hdr' can't be allocated in segment 4
LOAD: .text .eh_frame .eh_frame_hdr

and I wasn't able to find a conclusive reference documenting what a strip tool should do in such a case, and making llvm-bolt update the alignment requirements for the new PT_PHDR header would solve the problem.

But on the other hand a reasonable solution would indeed be to teach llvm-strip to propagate child segments' alignment requirements back to the parents before assigning offsets to segments.

[llvmbot]

@llvm/issue-subscribers-tools-llvm-objcopy-strip

Author: Krzysztof Pszeniczny (amharc)

Consider the following simple `main.cc` file: ```c++ int main() {} ```

Running:

$ clang++ main.cc -o main -Wl,-q
$ llvm-bolt main -o main.bolted
$ llvm-strip -S main.bolted -o main.bolted.stripped

Results in a misaligned PT_LOAD:

  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
...
  LOAD           0x003028 0x0000000000200000 0x0000000000200000 0x200384 0x200384 R E 0x200000

which leads to program crashes at startup:

$ ./main.bolted.stripped
[1]    153575 segmentation fault  ./main.bolted.stripped
$ /lib64/ld-linux-x86-64.so.2 ./main.bolted.stripped
./main.bolted.stripped: error while loading shared libraries: ./main.bolted.stripped: ELF load command address/offset not properly aligned

This is because the new PT_PHDR header was placed at the same offset as the new PT_LOAD containing the modified .text section:

$ llvm-readelf -l -h main.bolted
...
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x200000 0x0000000000200000 0x0000000000200000 0x000310 0x000310 R   0x8
...
  LOAD           0x200000 0x0000000000200000 0x0000000000200000 0x200384 0x200384 R E 0x200000

Which confuses llvm-strip, as it thinks that the PT_LOAD is a child of the PT_PHDR and thus it will disregard the alignment requirements of the (alleged) child.

[jh7370]

What version of llvm-strip are you using? There was a commit about 8 months ago to the LLVM main branch (so should be in the most recent release, I believe) that should address this issue, if I understand it correctly.