cmd/link: latest Mac otool does not understand where Go functions end

[cherrymui]

What version of Go are you using (go version)?

tip, also Go 1.10, 1.9, 1.8, 1.7

What operating system and processor architecture are you using (go env)?

darwin/amd64 macOS 10.13.3

What did you do?

$ cat xxx.go
package main; func main(){}
$ go build xxx.go
$ objdump -d -macho xxx &>/tmp/s # wait for ~10 minutes or more (!)
$ ll -h /tmp/s
-rw-r--r--  1 cherryyz  wheel   1.1G Apr  5 11:18 /tmp/s
$ otool -t -V xxx # also not terminated for a while, I didn't wait for 10 minutes
(ctrl-c)

It appears that it doesn't know where a function ends. So it disassembles each function (mostly) to the end of TEXT. And again for the next function, disassemble to the end of TEXT, and again... This results in a giant disassembly output.

__rt0_amd64:
 1043e10:       48 8b 3c 24     movq    (%rsp), %rdi
 ... 
 104b40d:       eb b1   jmp     main.init
__rt0_amd64_darwin:
 1046f90:       e9 7b ce ff ff  jmp     __rt0_amd64
 ...
 104b40d:       eb b1   jmp     main.init
_callRet:
 1044460:       48 83 ec 28     subq    $40, %rsp
 ...
 104b40d:       eb b1   jmp     main.init
_cmpbody:
 1001410:       48 39 fe        cmpq    %rdi, %rsi
 ...
 104b40d:       eb b1   jmp     main.init
_exit1:
 1046fc0:       bf 00 00 00 00  movl    $0, %edi
 ...

I remember I definitely did use otool to disassemble our binary before, and it worked. But it apparently doesn't work now for all recent versions of Go. Apparently otool is using LLVM-based objdump now. This is probably not the case before, when it worked for our binaries. I don't know when they switched. Maybe this is a bug in the LLVM-based objdump, I don't know. Or maybe there is something we can do to work around.

FWIW, lldb does disassemble our binary correctly.

$ objdump -version
Apple LLVM version 9.0.0 (clang-900.0.39.2)
  Optimized build.
  Default target: x86_64-apple-darwin17.4.0
  Host CPU: broadwell

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
$ otool --version
llvm-otool(1): Apple Inc. version cctools-900
Apple LLVM version 9.0.0 (clang-900.0.39.2)
  Optimized build.
  Default target: x86_64-apple-darwin17.4.0
  Host CPU: broadwell

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64

[minux]

Did we set the size of a function (as the symbol size) in the symbol table?

[cherrymui]

Did we set the size of a function (as the symbol size) in the symbol table?

Where should we put it? I didn't see a "size" entry in Mach-O symbol table.

[rsc]

I don't believe the Mac symbol table has any entry for sizes.

Does this happen with host linking too?
If not, maybe try to examine what's different about the symbol table?
I wonder if maybe we're accidentally not writing the symbol table in address order?

[cherrymui]

This also happens with host linking. Host-linked binary has symbol table in address order, but apparently it doesn't help.

I'll try to find out something from LLVM source.

[cherrymui]

Reading llvm-objdump source code, apparently it thinks all our function symbols are in DATA section, because our __text section does not have S_ATTR_PURE_INSTRUCTIONS bit set. Therefore it doesn't think the next function symbol is function, so it keeps going.

http://llvm.org/doxygen/MachOObjectFile_8cpp_source.html#l01945

 bool MachOObjectFile::isSectionText(DataRefImpl Sec) const {
   uint32_t Flags = getSectionFlags(*this, Sec);
   return Flags & MachO::S_ATTR_PURE_INSTRUCTIONS;
 }
 
 bool MachOObjectFile::isSectionData(DataRefImpl Sec) const {
   uint32_t Flags = getSectionFlags(*this, Sec);
   unsigned SectionType = Flags & MachO::SECTION_TYPE;
   return !(Flags & MachO::S_ATTR_PURE_INSTRUCTIONS) &&
          !(SectionType == MachO::S_ZEROFILL ||
            SectionType == MachO::S_GB_ZEROFILL);
 }

Setting the bit indeed makes otool happy.

[gopherbot]

Change https://golang.org/cl/105256 mentions this issue: cmd/link: set S_ATTR_PURE_INSTRUCTIONS bit for text section on darwin