patchelf.rb fails on go bottling

[iMichka]

brew config output

Linux CI / not necessary

brew doctor output

Linux CI / not necessary

---

• The brew doctor above contains no "Warning" lines.

What were you trying to do (and why)?

Bottling go 1.16

What happened (include all command output)?

2021-02-24T13:07:05.0421541Z �[34m==>�[0m �[1mbrew bottle --verbose --json go --keep-old�[0m
2021-02-24T13:08:18.6409734Z �[31m==>�[0m �[1m�[31mFAILED�[0m�[0m
2021-02-24T13:08:18.6423380Z �[34m==>�[0m �[1mInstalling patchelf...�[0m
2021-02-24T13:08:18.6425435Z /home/linuxbrew/.linuxbrew/bin/brew install patchelf
2021-02-24T13:08:18.6427118Z �[34m==>�[0m �[1mDownloading https://linuxbrew.bintray.com/bottles/patchelf-0.12_4.x86_64_linux.bottle.tar.gz�[0m
2021-02-24T13:08:18.6435621Z �[34m==>�[0m �[1mDownloading from https://akamai.bintray.com/56/56a0ff9acc9ab6c5d7f53d6f8ecce4878fd8c547a581960519504cf9e9dfac51?__gda__=exp=1614172746~hmac=244b54c7e503cb364eace097d882356e0ddff8e99b4d142f80efb8edcf1ddb63&response-content-disposition=attachment%3Bfilename%3D%22patchelf-0.12_4.x86_64_linux.bottle.tar.gz%22&response-content-type=application%2Fgzip&requestInfo=U2FsdGVkX1_jdUz69RdsWjsrSR4X_jmDg5dlt5h7Nb-yBuj-maRRdBkqSm_1Jz_-prCs2RbsOBOjQfpIjTD9WyBV3JbEQNepyLa4KE79tbIwbLbeqSBg516kV6h6N5EpkhIhtuYYsPUnKnNAfTw3Xw&response-X-Checksum-Sha1=584ea8b7f356dc8ebe6ca45f05c81fe8ebbe06d9&response-X-Checksum-Sha2=56a0ff9acc9ab6c5d7f53d6f8ecce4878fd8c547a581960519504cf9e9dfac51�[0m
2021-02-24T13:08:18.6442358Z �[34m==>�[0m �[1mPouring patchelf-0.12_4.x86_64_linux.bottle.tar.gz�[0m
2021-02-24T13:08:18.6443157Z /home/linuxbrew/.linuxbrew/Cellar/patchelf/0.12_4: 7 files, 942.3KB
2021-02-24T13:08:18.6444110Z �[34m==>�[0m �[1mBottling go--1.16.x86_64_linux.bottle.tar.gz...�[0m
2021-02-24T13:08:18.6445073Z tar cf /github/home/bottles/go--1.16.x86_64_linux.bottle.tar go/1.16
2021-02-24T13:08:18.6445811Z gzip -f go-bottle.tar
2021-02-24T13:08:18.6446721Z �[34m==>�[0m �[1mDetecting if go--1.16.x86_64_linux.bottle.tar.gz is relocatable...�[0m
2021-02-24T13:08:18.6447840Z �[33mWarning:�[0m String '/home/linuxbrew/.linuxbrew/Cellar' still exists in these files:
2021-02-24T13:08:18.6448905Z �[31m/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/pkg/linux_amd64/runtime/cgo.a�[0m
2021-02-24T13:08:18.6450027Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/cgo' at offset �[1m0x5777�[0m
2021-02-24T13:08:18.6451524Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/cgo' at offset �[1m0x75cd�[0m
2021-02-24T13:08:18.6452717Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/cgo' at offset �[1m0xa6ec�[0m
2021-02-24T13:08:18.6453884Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/cgo' at offset �[1m0xe1be�[0m
2021-02-24T13:08:18.6455041Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/cgo' at offset �[1m0x10aeb�[0m

snip

2021-02-24T13:08:18.9431325Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/profbuf.go' at offset �[1m0x1b352f�[0m
2021-02-24T13:08:18.9432584Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/rwmutex.go' at offset �[1m0x1b3578�[0m
2021-02-24T13:08:18.9433833Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/select.go' at offset �[1m0x1b35c1�[0m
2021-02-24T13:08:18.9435084Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/signal_amd64.go' at offset �[1m0x1b3609�[0m
2021-02-24T13:08:18.9436364Z  �[1m-->�[0m match '/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/runtime/signal_linux_amd64.go' at offset �[1m0x1b3657�[0m
2021-02-24T13:08:18.9437157Z Only the first 100 matches were output.
2021-02-24T13:08:18.9437510Z 
2021-02-24T13:08:18.9438176Z �[31mError:�[0m undefined method `header' for nil:NilClass
2021-02-24T13:08:18.9439644Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/bundle/ruby/2.6.0/gems/patchelf-1.3.0/lib/patchelf/alt_saver.rb:793:in `block in write_replaced_sections'
2021-02-24T13:08:18.9441343Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/bundle/ruby/2.6.0/gems/patchelf-1.3.0/lib/patchelf/alt_saver.rb:792:in `each'
2021-02-24T13:08:18.9442983Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/bundle/ruby/2.6.0/gems/patchelf-1.3.0/lib/patchelf/alt_saver.rb:792:in `write_replaced_sections'
2021-02-24T13:08:18.9444683Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/bundle/ruby/2.6.0/gems/patchelf-1.3.0/lib/patchelf/alt_saver.rb:626:in `rewrite_sections_library'
2021-02-24T13:08:18.9446352Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/bundle/ruby/2.6.0/gems/patchelf-1.3.0/lib/patchelf/alt_saver.rb:469:in `rewrite_sections'
2021-02-24T13:08:18.9447942Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/bundle/ruby/2.6.0/gems/patchelf-1.3.0/lib/patchelf/alt_saver.rb:88:in `save!'
2021-02-24T13:08:18.9449480Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/bundle/ruby/2.6.0/gems/patchelf-1.3.0/lib/patchelf/patcher.rb:191:in `save'
2021-02-24T13:08:18.9450843Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/os/linux/elf.rb:175:in `save_using_patchelf_rb'
2021-02-24T13:08:18.9451981Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/os/linux/elf.rb:105:in `patch!'
2021-02-24T13:08:18.9453168Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/keg_relocate.rb:48:in `change_rpath'
2021-02-24T13:08:18.9454595Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/keg_relocate.rb:16:in `block (2 levels) in relocate_dynamic_linkage'
2021-02-24T13:08:18.9455971Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/pathname.rb:356:in `ensure_writable'
2021-02-24T13:08:18.9457334Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/keg_relocate.rb:15:in `block in relocate_dynamic_linkage'
2021-02-24T13:08:18.9458685Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/keg_relocate.rb:14:in `each'
2021-02-24T13:08:18.9460015Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/keg_relocate.rb:14:in `relocate_dynamic_linkage'
2021-02-24T13:08:18.9461415Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg_relocate.rb:65:in `replace_placeholders_with_locations'
2021-02-24T13:08:18.9462781Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/bottle.rb:396:in `block (2 levels) in bottle_formula'
2021-02-24T13:08:18.9464001Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils.rb:424:in `ignore_interrupts'
2021-02-24T13:08:18.9465218Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/bottle.rb:394:in `block in bottle_formula'
2021-02-24T13:08:18.9466363Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/lock_file.rb:35:in `with_lock'
2021-02-24T13:08:18.9467852Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:332:in `block in lock'
2021-02-24T13:08:18.9469187Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/lock_file.rb:35:in `with_lock'
2021-02-24T13:08:18.9470229Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:330:in `lock'
2021-02-24T13:08:18.9471341Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/bottle.rb:301:in `bottle_formula'
2021-02-24T13:08:18.9472521Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/bottle.rb:89:in `block in bottle'
2021-02-24T13:08:18.9473715Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/bottle.rb:88:in `each'
2021-02-24T13:08:18.9474856Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/bottle.rb:88:in `bottle'
2021-02-24T13:08:18.9475920Z /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:122:in `<main>'
2021-02-24T13:08:18.9476405Z 
2021-02-24T13:08:18.9476954Z �[34m==>�[0m �[1mbrew linkage --test go�[0m
2021-02-24T13:08:20.2809156Z �[34m==>�[0m �[1mbrew install --only-dependencies --include-test go�[0m

What did you expect to happen?

Bottling should not fail

Step-by-step reproduction instructions (by running brew commands)

See commands used in the logs above

[iMichka]

cc @sjackman

I added HOMEBREW_PATCHELF_RB_WRITE: 0 to my build and got the same error.

Also, patchelf is being installed on CI, but it should not.

[sjackman]

I added HOMEBREW_PATCHELF_RB_WRITE: 0 to my build and got the same error.

HOMEBREW_NO_PATCHELF_RB_WRITE=1

All of the boolean env vars respond only to being set or not being set. Their value is ignored. The empty string may be treated as being unset, but I wouldn’t rely on that behaviour.

Also, patchelf is being installed on CI, but it should not.

Mike flagged this known issue previously.

[sjackman]

@rmNULL @david942j We've bene using patchelf.rb for some months now, and it's been working great! We've just experienced our first failure in bottling go. Would either of you be interested in looking into this issue?

@iMichka Could you provide the ELF file that produces this error?

[iMichka]

Here the detail about the file.

I found it out by using patchelf:

patchelf: cannot find section '.interp'. The input file is most likely statically linked

file /home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/debug/elf/testdata/go-relocation-test-gcc930-ranges-no-rela-x86-64

/home/linuxbrew/.linuxbrew/Cellar/go/1.16/libexec/src/debug/elf/testdata/go-relocation-test-gcc930-ranges-no-rela-x86-64: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter *empty*, not stripped

I can probably workaround the issue by not installing this debug/testdata file.

I think either ignoring these cases or at least having a better error message in patchelf.rb would help.

[danielnachun]

Weird, the ELF is dynamically linked but has no interpreter? Definitely seems like files that should be ignored or deleted if unneeded.

I'm wondering if we should eventually have an audit that warns us about ELFs missing the .interp section (so we know those binaries are static or weird like this one), and another to catch .debug sections to tell us when debugging symbols haven't been stripped (since file only returns stripped if all symbols have been stripped, not just debugging ones).

[iMichka]

My initial attempt to delete the debug files before bottling worked and I could ship go.
But it looks like some other formulae need these files (octant for example):
https://github.com/Homebrew/linuxbrew-core/runs/1974182054

[Bo98]

I wouldn't delete all of debug. Just the testdata subdirectory, and maybe the *_test.go files if you really want.

[iMichka]

I wouldn't delete all of debug. Just the testdata subdirectory

Thanks that worked

[david942j]

So it Looks like not a bug of patchelf.rb but a bad ELF encountered? If so I can help modify patchelf.rb to throw a better error description in such cases.

[iMichka]

Yes, I think the best is to have a better error message. At least a line that tells the name of the file that was broken.

It's not the first time this happens, there was already another weird ELF file in a formula that was used for test/debug purposes by upstream, and that was clearly not ment to be shipped to our users.

[sjackman]

It may be a bug in Homebrew if it's calling patchelf.rb for an ELF that should not be patched, like for example calling set_interpreter for an ELF that does not have a .interp section. Improved error messages are always appreciated.

[david942j]

One thing worth to mention is an ELF executable without .interp section is actually legitimate. Section headers are usually used for object files (i.e. .o files), and after objects linked those section headers are not necessary. Nevertheless, at least for common compilers (or to be precisely, linkers) those section headers are kept after linking, so it's usually fine to fetch information from sections in a linked ELF.

And for Unix patchelf, I would say it shouldn't use section headers for fetching the interpreter name (and file command shouldn't do so as well).

patchelf.rb currently supports two modes, one is unix-patchelf-compat mode that performs exact same as what patchelf does, so an error occurs when .interp section is not present is expected. One thing might help to improve is I can slightly modify patchelf.rb to just do not modify the section header when there is no one, maybe after then we are able to process this go binary.

BTW, the other mode of patchelf.rb won't have this issue because it properly handles the cases when there is no section header ;)

[sjackman]

Thanks for your input, @david942j!

I wouldn't delete all of debug. Just the testdata subdirectory

Thanks that worked

It looks we have a workaround that got us past this issue. There were some test data ELF files in the go package that weren't needed, so simply deleting them got us past this issue. So no changes are needed for patchelf.rb this time around. Sorry for the noise, and thanks for your help!