Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: statically compiling windows/amd64 binary on linux fails on 1.5 #13070

Closed
jessfraz opened this issue Oct 27, 2015 · 41 comments

Comments

Projects
None yet
8 participants
@jessfraz
Copy link
Contributor

commented Oct 27, 2015

Previously on go 1.4.3 this used to work:

export CC=x86_64-w64-mingw32-gcc
export CGO_ENABLED=1
export GOOS=windows
export GOARCH=amd64

go build -o bundles/1.9.0-dev/cross/windows/amd64/docker-1.9.0-dev.exe -a -tags 'netgo static_build sqlite_omit_load_extension apparmor selinux daemon journald btrfs_noversion libdm_no_deferred_remove' -installsuffix netgo -ldflags '-w -extldflags "-static" -extld=x86_64-w64-mingw32-gcc' ./docker

Now when trying to run the same thing I get:

# github.com/docker/docker/docker
/usr/local/go/pkg/tool/linux_amd64/link: running x86_64-w64-mingw32-gcc failed: exit status 1
/usr/bin/x86_64-w64-mingw32-ld: i386 architecture of input file `/tmp/go-link-623415307/000000.o' is incompatible with i386:x86-64 output
collect2: error: ld returned 1 exit status

That's a super gross command I know, but I can make a small go program and try to compile statically for windows on linux and see the same thing.

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2015

Also this is go 1.5.1

@davecheney

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2015

Two questions

  1. Isn't cross compilation always statically linked, ie, no cgo by default unless you work really hard at it
  2. The error message seems to be saying your trying to link a 32 bit .o using a 64 bit linker, the mingw32 would agree
@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2015

well its weird because the x86_64-w64 in the prefix is supposed to mean 64bit, its poorly named trying to find link to what I read when I originally wrote the binary compile script

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2015

so from what I can tell for 32bit i686-w64-mingw32-gcc is used and for 64bit x86_64-w64-mingw32-gcc is used.

@mwhudson

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2015

Still, /tmp/go-link-623415307/000000.o, which I think is a file produced by cgo, seems to be 32 bit when it needs to be 64-bit. Can you run go build -x and stick the output in a gist?

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2015

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2015

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2015

FWIW this is the output with go 1.4.3 https://gist.github.com/anonymous/5c991873a89319fc958f

@mwhudson

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2015

Hm, well one thing that's iirc changed is that there is external linking support for windows now. Can you try adding -v to -ldflags? And maybe -linkmode=internal too?

@davecheney

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2015

Hm, well one thing that's iirc changed is that there is external linking support for windows now. Can you try adding -v to -ldflags? And maybe -linkmode=internal too?

That would explain it. The previous 1.4 behaviour is "schmeh, i'll ignore your suggestion for linking and do it myself".

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2015

yay that works omg thanks for the speedy help \o/

@jessfraz jessfraz closed this Oct 27, 2015

@mwhudson

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

This still looks like a bug in something or other, fwiw: external linking should work when cross compiling.

@davecheney

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

@jfrazelle could you run file /tmp/go-link-623415307/000000.o or similar (not the microsoft tools) to solve the argument about which arch this binary is? There might be something bone headed in the go tool that is passing -m32 or something when it should not.

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

totally will do!

On Tue, Oct 27, 2015 at 6:52 PM, Dave Cheney notifications@github.com
wrote:

@jfrazelle https://github.com/jfrazelle could you run file
/tmp/go-link-623415307/000000.o or similar (not the microsoft tools) to
solve the argument about which arch this binary is? There might be
something bone headed in the go tool that is passing -m32 or something
when it should not.


Reply to this email directly or view it on GitHub
#13070 (comment).

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

this is probably a dumb question but it is deleting that file before i can run file on it... is there something i need to pass??

@davecheney

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

-work

On Wed, 28 Oct 2015, 13:13 Jess Frazelle notifications@github.com wrote:

this is probably a dumb question but it is deleting that file before i can
run file on it... is there something i need to pass??


Reply to this email directly or view it on GitHub
#13070 (comment).

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

wow i swear i knew that :shame:

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

so its still deleting it but keeping the go-build-xxxx dir instead of the link dir :(

@mwhudson

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

add -tmpdir /home/goobjs or something like that to -ldflags (after creating the directory!)

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

---> Making bundle: binary (in bundles/1.9.0-dev/binary)
Building: bundles/1.9.0-dev/binary/docker-1.9.0-dev.exe
# github.com/docker/docker/docker
/usr/local/go/pkg/tool/linux_amd64/link: running x86_64-w64-mingw32-gcc failed: exit status 1
/usr/bin/x86_64-w64-mingw32-ld: i386 architecture of input file `/butts/000000.o' is incompatible with i386:x86-64 output
collect2: error: ld returned 1 exit status

root@dfd7314e5304:/go/src/github.com/docker/docker# ^C
root@dfd7314e5304:/go/src/github.com/docker/docker# file /butts/000000.o
/butts/000000.o: Targa image data - Map 1 x
@davecheney

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

Ain't nobody gonna have time to link that!

On Wed, 28 Oct 2015, 13:40 Jess Frazelle notifications@github.com wrote:

---> Making bundle: binary (in bundles/1.9.0-dev/binary)
Building: bundles/1.9.0-dev/binary/docker-1.9.0-dev.exe

github.com/docker/docker/docker

/usr/local/go/pkg/tool/linux_amd64/link http://github.com/docker/docker/docker/usr/local/go/pkg/tool/linux_amd64/link: running x86_64-w64-mingw32-gcc failed: exit status 1
/usr/bin/x86_64-w64-mingw32-ld: i386 architecture of input file `/butts/000000.o' is incompatible with i386:x86-64 output
collect2: error: ld returned 1 exit status

root@dfd7314e5304:/go/src/github.com/docker/docker# ^C
root@dfd7314e5304:/go/src/github.com/docker/docker# file /butts/000000.o
/butts/000000.o: Targa image data - Map 1 x


Reply to this email directly or view it on GitHub
#13070 (comment).

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

Lol I was thinking I have no idea what this means

@davecheney

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

I'm wondering if that .o is corrupt.

On Wed, 28 Oct 2015, 14:21 Jess Frazelle notifications@github.com wrote:

Lol I was thinking I have no idea what this means


Reply to this email directly or view it on GitHub
#13070 (comment).

@minux

This comment has been minimized.

Copy link
Member

commented Oct 28, 2015

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

uploaded to https://s3.j3ss.co/tmp/linker-artifacts.tar.gz

On Wed, Oct 28, 2015 at 12:14 PM, Minux Ma notifications@github.com wrote:

Could you please make that 00000.o available somewhere so that i can take a
look? Thanks.


Reply to this email directly or view it on GitHub
#13070 (comment).

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Oct 28, 2015

Your object file has 1 section (.rsrc) with some resources (icons, bitmaps and similar) in it:

C:\dev\go\src>objdump -x u:\butts\000000.o

u:\butts\000000.o:     file format pe-i386
u:\butts\000000.o
architecture: i386, flags 0x00000039:
HAS_RELOC, HAS_DEBUG, HAS_SYMS, HAS_LOCALS
start address 0x00000000

Characteristics 0x104
        line numbers stripped
        32 bit words

Time/Date               Thu Jan 01 11:00:00 1970
Magic                   0000
MajorLinkerVersion      0
MinorLinkerVersion      0
SizeOfCode              00000000
SizeOfInitializedData   00000000
SizeOfUninitializedData 00000000
AddressOfEntryPoint     00000000
BaseOfCode              00000000
BaseOfData              00000000
ImageBase               00000000
SectionAlignment        00000000
FileAlignment           00000000
MajorOSystemVersion     0
MinorOSystemVersion     0
MajorImageVersion       0
MinorImageVersion       0
MajorSubsystemVersion   0
MinorSubsystemVersion   0
Win32Version            00000000
SizeOfImage             00000000
SizeOfHeaders           00000000
CheckSum                00000000
Subsystem               00000000        (unspecified)
DllCharacteristics      00000000
SizeOfStackReserve      00000000
SizeOfStackCommit       00000000
SizeOfHeapReserve       00000000
SizeOfHeapCommit        00000000
LoaderFlags             00000000
NumberOfRvaAndSizes     00000000

The Data Directory
Entry 0 00000000 00000000 Export Directory [.edata (or where ever we found it)]
Entry 1 00000000 00000000 Import Directory [parts of .idata]
Entry 2 00000000 00000000 Resource Directory [.rsrc]
Entry 3 00000000 00000000 Exception Directory [.pdata]
Entry 4 00000000 00000000 Security Directory
Entry 5 00000000 00000000 Base Relocation Directory [.reloc]
Entry 6 00000000 00000000 Debug Directory
Entry 7 00000000 00000000 Description Directory
Entry 8 00000000 00000000 Special Directory
Entry 9 00000000 00000000 Thread Storage Directory [.tls]
Entry a 00000000 00000000 Load Configuration Directory
Entry b 00000000 00000000 Bound Import Directory
Entry c 00000000 00000000 Import Address Table Directory
Entry d 00000000 00000000 Delay Import Directory
Entry e 00000000 00000000 CLR Runtime Header
Entry f 00000000 00000000 Reserved

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .rsrc         0005aaa7  00000000  00000000  0000003c  2**2
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
SYMBOL TABLE:
[  0](sec  1)(fl 0x00)(ty   0)(scl   3) (nx 0) 0x00000000 .rsrc


RELOCATION RECORDS FOR [.rsrc]:
OFFSET   TYPE              VALUE
00000158 rva32             .rsrc
00000168 rva32             .rsrc
00000178 rva32             .rsrc
00000188 rva32             .rsrc
00000198 rva32             .rsrc
000001a8 rva32             .rsrc
000001b8 rva32             .rsrc
000001c8 rva32             .rsrc



C:\dev\go\src>

Alex

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Oct 28, 2015

If I use simple "go build" command on a simple cgo program, I can see completely different output{

C:\dev\src\a>type foo.go
package main

import "C"

func main() {
        println("hello")
}

C:\dev\src\a>go build -ldflags="-tmpdir=c:\tmp\aaa" foo.go

C:\dev\src\a>dir c:\tmp\aaa
 Volume in drive C has no label.
 Volume Serial Number is F483-CD2E

 Directory of c:\tmp\aaa

29/10/2015  09:28 AM    <DIR>          .
29/10/2015  09:28 AM    <DIR>          ..
29/10/2015  09:29 AM             2,107 000000.o
29/10/2015  09:29 AM            11,723 000001.o
29/10/2015  09:29 AM           884,460 go.o
               3 File(s)        898,290 bytes
               2 Dir(s)  100,695,478,272 bytes free

C:\dev\src\a>objdump -h c:\tmp\aaa\000000.o

c:\tmp\aaa\000000.o:     file format pe-i386

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00000000  00000000  00000000  00000000  2**2
                  ALLOC, LOAD, READONLY, CODE
  1 .data         00000000  00000000  00000000  00000000  2**2
                  ALLOC, LOAD, DATA
  2 .rdata$zzz    00000028  00000000  00000000  0000017c  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .bss          00000000  00000000  00000000  00000000  2**2
                  ALLOC
  4 .debug_aranges 00000030  00000000  00000000  000001a4  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING
  5 .debug_info   000002a5  00000000  00000000  000001d4  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING
  6 .debug_abbrev 00000032  00000000  00000000  00000479  2**0
                  CONTENTS, READONLY, DEBUGGING
  7 .debug_line   0000003a  00000000  00000000  000004ab  2**0
                  CONTENTS, READONLY, DEBUGGING
  8 .debug_str    00000000  00000000  00000000  00000000  2**0
                  READONLY, DEBUGGING

C:\dev\src\a>objdump -h c:\tmp\aaa\000001.o

c:\tmp\aaa\000001.o:     file format pe-i386

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         000001f0  00000000  00000000  000001f4  2**4
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
  1 .data         00000000  00000000  00000000  00000000  2**2
                  ALLOC, LOAD, DATA
  2 .rdata        00000084  00000000  00000000  000003e4  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .rdata$zzz    00000064  00000000  00000000  00000468  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .eh_frame     00000140  00000000  00000000  000004cc  2**2
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
  5 .bss          00000000  00000000  00000000  00000000  2**2
                  ALLOC
  6 .debug_aranges 000000b0  00000000  00000000  0000060c  2**3
                  CONTENTS, RELOC, READONLY, DEBUGGING
  7 .debug_info   00000f0c  00000000  00000000  000006bc  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING
  8 .debug_abbrev 00000499  00000000  00000000  000015c8  2**0
                  CONTENTS, READONLY, DEBUGGING
  9 .debug_line   000002f6  00000000  00000000  00001a61  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING
 10 .debug_str    00000018  00000000  00000000  00001d57  2**0
                  CONTENTS, READONLY, DEBUGGING
 11 .debug_loc    0000006f  00000000  00000000  00001d6f  2**0
                  CONTENTS, READONLY, DEBUGGING

C:\dev\src\a>

That is all on Windows with whatever version of mingw I have.

Alex

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 29, 2015

Ya sorry the rsrc tool has an issue filed I guess akavel/rsrc#11

@jessfraz

This comment has been minimized.

Copy link
Contributor Author

commented Oct 29, 2015

We embed an image we can probably remove it ... windows... cute whale but idk if worth the trouble ;)

@jhowardmsft

This comment has been minimized.

Copy link

commented Oct 29, 2015

@jfrazelle Ah, we can't remove the use of akavel/rsrc as it's also the thing that manifests docker.exe for Windows 10, otherwise version APIs don't work.

(PS We hit the same thing - I emailed golang-nuts: https://groups.google.com/forum/#!topic/golang-nuts/pZB_QQQk2Z4)

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Oct 29, 2015

@jhowardmsft go1.5 started using external linker on windows. I suspect some functionality we had with internal linker is broken. For example we cannot include dwarf information anymore (see issue #10776). It sounds like we don't link *.syso files properly either.

What others said, you can use -linkmode=internal ldflag when building executable. This should revert your build process to what 1.4 use to do.

Alex

@jhowardmsft

This comment has been minimized.

Copy link

commented Oct 30, 2015

@alexbrainman Yes, thank you so much. That's great - I can compile again using 1.5.1 with that workaround on Windows.

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Oct 30, 2015

I am pretty sure the problem is akavel/rsrc/issues/11 (hopefully @akavel will fix the issue).

Alex

@minux

This comment has been minimized.

Copy link
Member

commented Oct 30, 2015

@akavel akavel referenced this issue Nov 2, 2015

Closed

64 bit issue #11

@akavel

This comment has been minimized.

Copy link
Contributor

commented Nov 2, 2015

If I change rsrc to make 2 .syso files, one for 32-bit and one for 64-bit ARCH, (how) can I then tell the go tool which one of them it should choose for linking? Would naming them foo_amd64.syso and foo_386.syso work? I understand this is not assumed to be a regression; given that, has it now become impossible to have a project with a COFF .syso file be go-gettable on any ARCH? or if it is, how? I'd be grateful for any help here!

@jhowardmsft

This comment has been minimized.

Copy link

commented Nov 2, 2015

I was thinking about this over the weekend. Building both .syso files would be problematic for linking as obviously only one file is needed. Perhaps an rsrc --arch=amd64 .... option?

@akavel

This comment has been minimized.

Copy link
Contributor

commented Nov 2, 2015

@jhowardmsft Yeah, that's what I'll try to do as a quick workaround (?) but the problem is, such a project becomes then somewhat strongly tied to exactly 1 architecture (amd64 in this case), making it significantly more annoying to build on e.g. 386, or potentially some, uhm, Windows RT? or some other arm stuff in the future? Raspberry Pi maybe?

As another workaround, I'm thinking if maybe doing something like shown below might maybe work (? untested), but boy is it ugly:

$ find amd64 386 -t f
amd64/rsrc.syso
amd64/dummy.go
386/rsrc.syso
386/dummy.go
$ cat rsrc_amd64.go
import _ "myproj/amd64"
$ cat rsrc_386.go
import _ "myproj/386"
@jhowardmsft

This comment has been minimized.

Copy link

commented Nov 2, 2015

Oh yuck! ;) Yeah, that is kind of horrible. I am wondering if that would work unless rsrc.syso had an rsrc_ARCH.syso name. Does go use that as a build tag on an _ARCH postfix for syso files? I'm really not sure if it does or not.

@mwhudson

This comment has been minimized.

Copy link
Contributor

commented Nov 2, 2015

I think the GOOS_GOARCH naming convention works for syso files too (it's
how the race detector works)

On 3 November 2015 at 07:03, Mateusz Czapliński notifications@github.com
wrote:

If I change rsrc to make 2 .syso files, one for 32-bit and one for 64-bit
ARCH, (how) can I then tell the go tool which one of them it should choose
for linking? Would naming them foo_amd64.syso and foo_386.syso work?
I understand this is not assumed to be a regression, so given that, did it
now become impossible to have a project with a COFF .syso file be
go-gettable on any ARCH? or if it is, how? I'd be grateful for any help
here!


Reply to this email directly or view it on GitHub
#13070 (comment).

@minux

This comment has been minimized.

Copy link
Member

commented Nov 2, 2015

@akavel

This comment has been minimized.

Copy link
Contributor

commented Nov 2, 2015

@mwhudson @minux Ok, that's great then, thanks! Now just to know if a quick .Machine=pe.IMAGE_FILE_MACHINE_AMD64 will be enough, or some more serious investigation will be needed...

@golang golang locked and limited conversation to collaborators Nov 4, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.