build: NetBSD support #611

Closed
gopherbot opened this Issue Feb 19, 2010 · 29 comments

Comments

Projects
None yet
7 participants
@gopherbot

by s1061123:

Hi golang gurus,

I love NetBSD and use as my OS, however, unfortunately, 
it might be too minor to support golang officially ;)

NetBSD cannot goes well as FreeBSD because NetBSD binary requires original
ELF header as follows:
http://www.netbsd.org/docs/kernel/elf-notes.html
So original linkers, such as "6l", require some hacks for NetBSD.

Currently I am going to hack it because of my love to NetBSD ;), but
I would like to know whether someone doing same things or not.

If none does it, could let me know 'how to commit the code'?
I am a newbie of opensource, so I am not getting used to.

Thanks,
Tomofumi
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Feb 19, 2010

Comment 1 by giles.lean:

I've been working (slowly) on a NetBSD port.  My method has been to run all.bash and
fix whatever breaks; I haven't got as far as 6l and 8l yet.
Perhaps we can collaborate on this work?  I can put my existing patches on the web
somewhere, or mail them.
Best regards,
Giles

Comment 1 by giles.lean:

I've been working (slowly) on a NetBSD port.  My method has been to run all.bash and
fix whatever breaks; I haven't got as far as 6l and 8l yet.
Perhaps we can collaborate on this work?  I can put my existing patches on the web
somewhere, or mail them.
Best regards,
Giles
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Feb 20, 2010

Comment 2 by giles.lean:

I forgot to add:
> If none does it, could let me know 'how to commit the code'?
On the top level page http://golang.org/ in the left hand side panel there is an item
"How to write code" which links to http://golang.org/doc/code.html, which in turn
describes the process for contributing code.
I've contributed a couple of minor fixes previously; the process wasn't difficult.
I imagine if we produce a clean looking NetBSD port which passes the test suite, it
would have a good chance of being picked up.  But that I suspect is some time away.
First, there's some porting to do, and I think 6l and 8l might be the easy parts. 
Figuring out the best choices for threading and semaphore interfaces inside the
runtime package is my current area of work.  So far as I know, none of the interfaces
used in the OS X (Darwin), FreeBSD, or Linux ports are available natively in NetBSD,
and using (say) the Linux implementation via NetBSD kernel emulation seems like a bad
idea.
FYI I'm working on NetBSD-5.0.  Given the threading changes introduced in 5.0
starting there seemed best.  I'll be happy if 4.0 can also be supported, but would
value a "better" 5.0 port over a less good port which also supported 4.0.

Comment 2 by giles.lean:

I forgot to add:
> If none does it, could let me know 'how to commit the code'?
On the top level page http://golang.org/ in the left hand side panel there is an item
"How to write code" which links to http://golang.org/doc/code.html, which in turn
describes the process for contributing code.
I've contributed a couple of minor fixes previously; the process wasn't difficult.
I imagine if we produce a clean looking NetBSD port which passes the test suite, it
would have a good chance of being picked up.  But that I suspect is some time away.
First, there's some porting to do, and I think 6l and 8l might be the easy parts. 
Figuring out the best choices for threading and semaphore interfaces inside the
runtime package is my current area of work.  So far as I know, none of the interfaces
used in the OS X (Darwin), FreeBSD, or Linux ports are available natively in NetBSD,
and using (say) the Linux implementation via NetBSD kernel emulation seems like a bad
idea.
FYI I'm working on NetBSD-5.0.  Given the threading changes introduced in 5.0
starting there seemed best.  I'll be happy if 4.0 can also be supported, but would
value a "better" 5.0 port over a less good port which also supported 4.0.
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Feb 20, 2010

Comment 3 by s1061123:

Hi Giles,
First, thank you for your quick reply.
I'm very happy to know you are working with it. My progress is also slow, too.
>Perhaps we can collaborate on this work?  I can put my existing patches on the web
>somewhere, or mail them.
I'm agree with your opinion. Currently my status is the following:
 - Target host is amd64, NetBSD-5.0 (now 5.0.2)
 - Compile golang with "FreeBSD" option
  (I change scripts not to check platform from 'uname', force build as FreeBSD)
 - golang library unit-test is failed because generated binary by 6l does not have
NetBSD specific ELF section.
Attached log might be easy to understand my status.
Currently I do not take care of thread and semaphores and my target is "hello world."
I will attach the diff after my workspace is updated.
If you do not mind, please attach diffs and then let's get on the same page.
Thanks,
Tomofumi
----- Log
kagaribi% pwd
/home/tom/usr/go/src
kagaribi% ./all.bash
rm -f *.o *.6 6.out lib9.a
(snip)
gmake[1]: Entering directory `/home/tom/usr/go/src/pkg/archive/tar'
gotest
gmake[2]: Entering directory `/home/tom/usr/go/src/pkg/archive/tar'
rm -f _test/archive/tar.a _gotest_.6
gmake[2]: Leaving directory `/home/tom/usr/go/src/pkg/archive/tar'
gmake[2]: Entering directory `/home/tom/usr/go/src/pkg/archive/tar'
6g -o _gotest_.6 common.go reader.go writer.go    reader_test.go writer_test.go
rm -f _test/archive/tar.a
gopack grc _test/archive/tar.a _gotest_.6
gmake[2]: Leaving directory `/home/tom/usr/go/src/pkg/archive/tar'
/home/tom/usr/bin/gotest: line 164: ./6.out: cannot execute binary file
gmake[1]: *** [test] Error 126
gmake[1]: Leaving directory `/home/tom/usr/go/src/pkg/archive/tar'
gmake: *** [archive/tar.test] Error 2

Comment 3 by s1061123:

Hi Giles,
First, thank you for your quick reply.
I'm very happy to know you are working with it. My progress is also slow, too.
>Perhaps we can collaborate on this work?  I can put my existing patches on the web
>somewhere, or mail them.
I'm agree with your opinion. Currently my status is the following:
 - Target host is amd64, NetBSD-5.0 (now 5.0.2)
 - Compile golang with "FreeBSD" option
  (I change scripts not to check platform from 'uname', force build as FreeBSD)
 - golang library unit-test is failed because generated binary by 6l does not have
NetBSD specific ELF section.
Attached log might be easy to understand my status.
Currently I do not take care of thread and semaphores and my target is "hello world."
I will attach the diff after my workspace is updated.
If you do not mind, please attach diffs and then let's get on the same page.
Thanks,
Tomofumi
----- Log
kagaribi% pwd
/home/tom/usr/go/src
kagaribi% ./all.bash
rm -f *.o *.6 6.out lib9.a
(snip)
gmake[1]: Entering directory `/home/tom/usr/go/src/pkg/archive/tar'
gotest
gmake[2]: Entering directory `/home/tom/usr/go/src/pkg/archive/tar'
rm -f _test/archive/tar.a _gotest_.6
gmake[2]: Leaving directory `/home/tom/usr/go/src/pkg/archive/tar'
gmake[2]: Entering directory `/home/tom/usr/go/src/pkg/archive/tar'
6g -o _gotest_.6 common.go reader.go writer.go    reader_test.go writer_test.go
rm -f _test/archive/tar.a
gopack grc _test/archive/tar.a _gotest_.6
gmake[2]: Leaving directory `/home/tom/usr/go/src/pkg/archive/tar'
/home/tom/usr/bin/gotest: line 164: ./6.out: cannot execute binary file
gmake[1]: *** [test] Error 126
gmake[1]: Leaving directory `/home/tom/usr/go/src/pkg/archive/tar'
gmake: *** [archive/tar.test] Error 2
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Feb 20, 2010

Comment 4 by giles.lean:

I have put up a web page with notes about my approach, my email address, and links to
the patches I have.I suspect we're better taking the discussion out of the issue
tracker now.
My web page is at:
http://netherstone.net/golang/netbsd/
Until the DNS for netherstone.net updates, the IP address is 65.181.139.106:
http://65.181.139.106/golang/netbsd/
Regards,
Giles
<giles.lean@pobox.com>

Comment 4 by giles.lean:

I have put up a web page with notes about my approach, my email address, and links to
the patches I have.I suspect we're better taking the discussion out of the issue
tracker now.
My web page is at:
http://netherstone.net/golang/netbsd/
Until the DNS for netherstone.net updates, the IP address is 65.181.139.106:
http://65.181.139.106/golang/netbsd/
Regards,
Giles
<giles.lean@pobox.com>
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Feb 20, 2010

Comment 5 by giles.lean:

[ Reposted to fix a critical typo in a domain name: the domain I'm using is in .org,
not .net. ]
I have put up a web page with notes about my approach, my email address, and links to
the patches I have.I suspect we're better taking the discussion out of the issue
tracker now.
My web page is at:
http://netherstone.org/golang/netbsd/
Until the DNS for netherstone.org updates, the IP address is 65.181.139.106:
http://65.181.139.106/golang/netbsd/
Regards,
Giles
<giles.lean@pobox.com>

Comment 5 by giles.lean:

[ Reposted to fix a critical typo in a domain name: the domain I'm using is in .org,
not .net. ]
I have put up a web page with notes about my approach, my email address, and links to
the patches I have.I suspect we're better taking the discussion out of the issue
tracker now.
My web page is at:
http://netherstone.org/golang/netbsd/
Until the DNS for netherstone.org updates, the IP address is 65.181.139.106:
http://65.181.139.106/golang/netbsd/
Regards,
Giles
<giles.lean@pobox.com>
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Feb 22, 2010

Comment 6 by giles.lean:

I've created a blog to track my progress; the pre-existing main page is still there
(and still looks like it was created about 1996 :-) and the blog's here:
http://netherstone.org/golang/netbsd/blog/
I think it will be easier (and more obvious for anyone wanting to track my progress)
to add a new posting when I have a new patch or newer version of an old patch.  We'll
see.
As before; all help is welcome, and anyone wanting to assist should feel free to
pitch in.

Comment 6 by giles.lean:

I've created a blog to track my progress; the pre-existing main page is still there
(and still looks like it was created about 1996 :-) and the blog's here:
http://netherstone.org/golang/netbsd/blog/
I think it will be easier (and more obvious for anyone wanting to track my progress)
to add a new posting when I have a new patch or newer version of an old patch.  We'll
see.
As before; all help is welcome, and anyone wanting to assist should feel free to
pitch in.
@rsc

This comment has been minimized.

Show comment Hide comment
@rsc

rsc Mar 6, 2010

Contributor

Comment 7:

Labels changed: added priority-low, removed priority-medium.

Status changed to LongTerm.

Contributor

rsc commented Mar 6, 2010

Comment 7:

Labels changed: added priority-low, removed priority-medium.

Status changed to LongTerm.

@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Aug 17, 2010

Comment 8 by joerg.sonnenberger:

I have a mostly working Go port for NetBSD current/amd64. There are some technical
reasons why Go won't work on 5.1 or earlier for amd64 at least. I haven't had time to
look at i386 yet.

Comment 8 by joerg.sonnenberger:

I have a mostly working Go port for NetBSD current/amd64. There are some technical
reasons why Go won't work on 5.1 or earlier for amd64 at least. I haven't had time to
look at i386 yet.
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Aug 17, 2010

Comment 9 by joerg.sonnenberger:

I have a mostly working Go port for NetBSD current/amd64. There are some technical
reasons why Go won't work on 5.1 or earlier for amd64 at least. I haven't had time to
look at i386 yet.

Comment 9 by joerg.sonnenberger:

I have a mostly working Go port for NetBSD current/amd64. There are some technical
reasons why Go won't work on 5.1 or earlier for amd64 at least. I haven't had time to
look at i386 yet.
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Aug 17, 2010

Comment 10 by joerg.sonnenberger:

I have a mostly working Go port for NetBSD current/amd64. There are some technical
reasons why Go won't work on 5.1 or earlier for amd64 at least. I haven't had time to
look at i386 yet.

Comment 10 by joerg.sonnenberger:

I have a mostly working Go port for NetBSD current/amd64. There are some technical
reasons why Go won't work on 5.1 or earlier for amd64 at least. I haven't had time to
look at i386 yet.
@gopherbot

This comment has been minimized.

Show comment Hide comment
@gopherbot

gopherbot Aug 17, 2010

Comment 11 by joerg.sonnenberger:

I have a mostly working Go port for NetBSD current/amd64. There are some technical
reasons why Go won't work on 5.1 or earlier for amd64 at least. I haven't had time to
look at i386 yet.

Comment 11 by joerg.sonnenberger:

I have a mostly working Go port for NetBSD current/amd64. There are some technical
reasons why Go won't work on 5.1 or earlier for amd64 at least. I haven't had time to
look at i386 yet.
@rsc

This comment has been minimized.

Show comment Hide comment
@rsc

rsc Dec 9, 2011

Contributor

Comment 12:

Labels changed: added priority-someday, removed priority-low.

Contributor

rsc commented Dec 9, 2011

Comment 12:

Labels changed: added priority-someday, removed priority-low.

@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Aug 18, 2012

Member

Comment 14:

i think we can close this issue now (NetBSD support is pretty complete).
Member

minux commented Aug 18, 2012

Comment 14:

i think we can close this issue now (NetBSD support is pretty complete).
@davecheney

This comment has been minimized.

Show comment Hide comment
@davecheney

davecheney Aug 18, 2012

Contributor

Comment 15:

Can we get a builder on the dashboard first ?
Contributor

davecheney commented Aug 18, 2012

Comment 15:

Can we get a builder on the dashboard first ?
@bradfitz

This comment has been minimized.

Show comment Hide comment
@bradfitz

bradfitz Aug 19, 2012

Member

Comment 16:

If NetBSD runs well under KVM and somebody could write up a NetBSD zero-to-builder HOWTO
for me (ideally on the Wiki), then I can follow the instructions on my home machine
where the OpenBSD builder runs (er, ran... we should start it up again, now that Go 1 is
out).  Feel free to assign this bug to me if you want me to run a builder and that HOWTO
has been written.
Member

bradfitz commented Aug 19, 2012

Comment 16:

If NetBSD runs well under KVM and somebody could write up a NetBSD zero-to-builder HOWTO
for me (ideally on the Wiki), then I can follow the instructions on my home machine
where the OpenBSD builder runs (er, ran... we should start it up again, now that Go 1 is
out).  Feel free to assign this bug to me if you want me to run a builder and that HOWTO
has been written.
@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Aug 20, 2012

Member

Comment 17:

my tests suggest that math test of Go will fail on NetBSD 6.0 BETA2 if it's running in
qemu-kvm.
however, if i run NetBSD with latest stable version of virtualbox, then all the test
could pass.
Member

minux commented Aug 20, 2012

Comment 17:

my tests suggest that math test of Go will fail on NetBSD 6.0 BETA2 if it's running in
qemu-kvm.
however, if i run NetBSD with latest stable version of virtualbox, then all the test
could pass.
@4a6f656c

This comment has been minimized.

Show comment Hide comment
@4a6f656c

4a6f656c Aug 22, 2012

Contributor

Comment 18:

FTR with Go tip (865991cd86fb):
$ uname -mrs
NetBSD 6.0_BETA2 amd64
$ ./all.bash
...
ALL TESTS PASSED
---
Installed Go for netbsd/amd64 in /home/joel/src/go
Installed commands in /home/joel/src/go/bin
$ uname -mrs
NetBSD 6.0_BETA2 i386
$ ./all.bash
...
ALL TESTS PASSED
Installed Go for netbsd/386 in /home/joel/src/go
Installed commands in /home/joel/src/go/bin
Contributor

4a6f656c commented Aug 22, 2012

Comment 18:

FTR with Go tip (865991cd86fb):
$ uname -mrs
NetBSD 6.0_BETA2 amd64
$ ./all.bash
...
ALL TESTS PASSED
---
Installed Go for netbsd/amd64 in /home/joel/src/go
Installed commands in /home/joel/src/go/bin
$ uname -mrs
NetBSD 6.0_BETA2 i386
$ ./all.bash
...
ALL TESTS PASSED
Installed Go for netbsd/386 in /home/joel/src/go
Installed commands in /home/joel/src/go/bin
@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Aug 22, 2012

Member

Comment 19:

I can volunteer a NetBSD6/amd64 builder for at least a year.
Member

minux commented Aug 22, 2012

Comment 19:

I can volunteer a NetBSD6/amd64 builder for at least a year.
@bradfitz

This comment has been minimized.

Show comment Hide comment
@bradfitz

bradfitz Aug 22, 2012

Member

Comment 20:

Minux, I can run a NetBSD image on Cernio.com for $65/year (which Google can pay for),
but I have little interest in maintaining it.  Do you and/or jsing want to maintain it
if I set it up?  Then it won't have GFWC issues, either, if that's a problem (net
tests?).
Member

bradfitz commented Aug 22, 2012

Comment 20:

Minux, I can run a NetBSD image on Cernio.com for $65/year (which Google can pay for),
but I have little interest in maintaining it.  Do you and/or jsing want to maintain it
if I set it up?  Then it won't have GFWC issues, either, if that's a problem (net
tests?).
@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Aug 22, 2012

Member

Comment 21:

Brad, I'm very happy to help maintaining it.
Member

minux commented Aug 22, 2012

Comment 21:

Brad, I'm very happy to help maintaining it.
@bradfitz

This comment has been minimized.

Show comment Hide comment
@bradfitz

bradfitz Aug 22, 2012

Member

Comment 22:

Could somebody point me at a NetBSD 6 image that will run under KVM?
Cernio contact says:
"Yes, NetBSD is just fine as we are using KVM. We could install via ISO or
simply a dd of a compressed image. Let me double check a few things and get
back to you on which is easiest.
If we go the pre-installed image route, you will need to do a few things:
* Make sure you support KVM virtio devices or specify which devices you will
need (e1000 for nic works well for example)
* Make sure to support ACPI for powerbutton events."
I imagine it'll be easiest if we just give them an image, and don't make them go through
the install process for us.  It would also be good to host somewhere so we can re-setup
the builder hermetically later, from install instructions.
Member

bradfitz commented Aug 22, 2012

Comment 22:

Could somebody point me at a NetBSD 6 image that will run under KVM?
Cernio contact says:
"Yes, NetBSD is just fine as we are using KVM. We could install via ISO or
simply a dd of a compressed image. Let me double check a few things and get
back to you on which is easiest.
If we go the pre-installed image route, you will need to do a few things:
* Make sure you support KVM virtio devices or specify which devices you will
need (e1000 for nic works well for example)
* Make sure to support ACPI for powerbutton events."
I imagine it'll be easiest if we just give them an image, and don't make them go through
the install process for us.  It would also be good to host somewhere so we can re-setup
the builder hermetically later, from install instructions.
@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Aug 23, 2012

Member

Comment 23:

I do not know any.
Can cernio provide xen hosting? From my own experiences,
If NetBSD is running on KVM, all.bash cannot pass.
I've experienced very strang math pkg test failures which I cannot
explain.
Member

minux commented Aug 23, 2012

Comment 23:

I do not know any.
Can cernio provide xen hosting? From my own experiences,
If NetBSD is running on KVM, all.bash cannot pass.
I've experienced very strang math pkg test failures which I cannot
explain.
@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Aug 24, 2012

Member

Comment 24:

it seems we can't test netbsd/386 build on netbsd/amd64 system.
lots of very weird test failures (SIGILL/SIGTRAP received).
@jsing, can you reproduce this?
ps: NetBSD 6.0 BETA2 under VirtualBox 4.1.18 OSE with AMD CPU.
Member

minux commented Aug 24, 2012

Comment 24:

it seems we can't test netbsd/386 build on netbsd/amd64 system.
lots of very weird test failures (SIGILL/SIGTRAP received).
@jsing, can you reproduce this?
ps: NetBSD 6.0 BETA2 under VirtualBox 4.1.18 OSE with AMD CPU.
@4a6f656c

This comment has been minimized.

Show comment Hide comment
@4a6f656c

4a6f656c Aug 25, 2012

Contributor

Comment 25:

minix.ma: Yes, I've been able to reproduce this - turns out that the netbsd32 compat
code does not handle the same signal ABI versions that the standard kernels do. I've
just sent out a change to address this. However, I'm now seeing another issue (readily
triggered by the os/exec tests) that appears to be related to wait4 and signals...
Contributor

4a6f656c commented Aug 25, 2012

Comment 25:

minix.ma: Yes, I've been able to reproduce this - turns out that the netbsd32 compat
code does not handle the same signal ABI versions that the standard kernels do. I've
just sent out a change to address this. However, I'm now seeing another issue (readily
triggered by the os/exec tests) that appears to be related to wait4 and signals...
@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Aug 28, 2012

Member

Comment 26:

Re #23, i can confirm kvm on intel cpu works.
@jsing,
now, GOARCH=386 on amd64 NetBSD will always fail the net/http test.
(TestTransportPersistConnLeak test timeout, and i've raised the 'ulimit -n')
i don't see any failed os/exec test however.
--- FAIL: TestStressSurpriseServerCloses (6.70 seconds)
transport_test.go:440:  presumed deadlock; no HTTP client activity seen in awhile
Member

minux commented Aug 28, 2012

Comment 26:

Re #23, i can confirm kvm on intel cpu works.
@jsing,
now, GOARCH=386 on amd64 NetBSD will always fail the net/http test.
(TestTransportPersistConnLeak test timeout, and i've raised the 'ulimit -n')
i don't see any failed os/exec test however.
--- FAIL: TestStressSurpriseServerCloses (6.70 seconds)
transport_test.go:440:  presumed deadlock; no HTTP client activity seen in awhile
@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Aug 28, 2012

Member

Comment 27:

to clarify, in addition to TestStressSurpriseServerCloses test failure,
TestTransportPersistConnLeak never finishes (it seems stuck at c.Get)
and triggered the test timeout.
i've watched the number of open fds, and it is well below the limit.
Member

minux commented Aug 28, 2012

Comment 27:

to clarify, in addition to TestStressSurpriseServerCloses test failure,
TestTransportPersistConnLeak never finishes (it seems stuck at c.Get)
and triggered the test timeout.
i've watched the number of open fds, and it is well below the limit.
@rsc

This comment has been minimized.

Show comment Hide comment
@rsc

rsc Sep 12, 2012

Contributor

Comment 28:

Labels changed: added go1.1maybe.

Contributor

rsc commented Sep 12, 2012

Comment 28:

Labels changed: added go1.1maybe.

@minux

This comment has been minimized.

Show comment Hide comment
@minux

minux Oct 12, 2012

Member

Comment 29:

now that we have a netbsd/amd64 builder, this could be regarded as fixed.
(still needs a netbsd/386 builder, because netbsd/amd64 host can't pass all
of "GOARCH=386 ./all.bash" tests)

Status changed to Fixed.

Member

minux commented Oct 12, 2012

Comment 29:

now that we have a netbsd/amd64 builder, this could be regarded as fixed.
(still needs a netbsd/386 builder, because netbsd/amd64 host can't pass all
of "GOARCH=386 ./all.bash" tests)

Status changed to Fixed.

@bsiegert

This comment has been minimized.

Show comment Hide comment
@bsiegert

bsiegert Oct 12, 2012

Contributor

Comment 30:

The 386 builder is now up and running.
Contributor

bsiegert commented Oct 12, 2012

Comment 30:

The 386 builder is now up and running.

@rsc rsc added this to the Go1.1 milestone Apr 14, 2015

@rsc rsc removed the go1.1maybe label Apr 14, 2015

@gopherbot gopherbot locked and limited conversation to collaborators Jun 24, 2016

This issue was closed.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.