New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/cgo: GNU/kFreeBSD support #3533

Open
oerdnj opened this Issue Apr 15, 2012 · 16 comments

Comments

Projects
None yet
4 participants
@oerdnj

oerdnj commented Apr 15, 2012

What steps will reproduce the problem?
If possible, include a link to a program on play.golang.org.
1. Build on Debian GNU/KFreeBSD (amd64 or i386)
2. Run test suite

What is the expected output?
Tests run

What do you see instead?
Tests hang, the buildd kills build after 150 minutes of inactivity:
! : || { cd /build/buildd-golang_1-3-kfreebsd-amd64-BYZvvB/golang-1/src &&
/build/buildd-golang_1-3-kfreebsd-amd64-BYZvvB/golang-1/debian/test.bash
/build/buildd-golang_1-3-kfreebsd-amd64-BYZvvB/golang-1/pkg/tool/freebsd_amd64; }
E: Caught signal 'Terminated': terminating immediately
make[1]: *** [override_dh_auto_test] Terminated
make: *** [build-arch] Terminated
Build killed with signal TERM after 150 minutes of inactivity

Which compiler are you using (5g, 6g, 8g, gccgo)?
6g,8g

Which operating system are you using?
Debian GNU/KFreeBSD

Which version are you using?  (run 'go version')
go1 with patches to allow it to compile with GNU stack

Please provide any additional information below.
The test suite just never runs on buildd and build is killed after some minutes of
inactivity.
Full buildd logs are located here...
kfreebsd-amd64:
https://buildd.debian.org/status/fetch.php?pkg=golang&;arch=kfreebsd-amd64&ver=2%3A1-3&stamp=1334365793
kfreebsd-i386:
https://buildd.debian.org/status/fetch.php?pkg=golang&;arch=kfreebsd-i386&ver=2%3A1-3&stamp=1334366206

The patches are:
http://anonscm.debian.org/gitweb/?p=pkg-google/golang.git;a=blob;f=debian/patches/007-use_native_dynamic_linker_on_kfreebsd.patch;h=04bf8b82007cfe65f9f3ddcd87e6539aa2a20e01;hb=HEAD
http://anonscm.debian.org/gitweb/?p=pkg-google/golang.git;a=blob;f=debian/patches/011-__FreeBSD_kernel__.patch;h=45be34f1229c2ac40875ad3241f1d367d726173f;hb=HEAD
http://anonscm.debian.org/gitweb/?p=pkg-google/golang.git;a=blob;f=debian/patches/012-sigfillset_kFreeBSD.patch;h=40f01fb42b90d0dce8e9bda6ee792f1df3226d3a;hb=HEAD
http://anonscm.debian.org/gitweb/?p=pkg-google/golang.git;a=blob;f=debian/patches/013-kFreeBSD-AI_MASK.patch;h=e67a34058557c47bed4b840022a0d329aec0e12e;hb=HEAD

I will disable tests on KFreeBSD for now, but it is definitely worth investigating.
@minux

This comment has been minimized.

Member

minux commented Apr 15, 2012

Comment 1:

Debian GNU/kFreeBSD is a somewhat hybrid system, i.e. FreeBSD kernel but glibc.
Because we inherently tie libc to kernel, so if we want to support Debian GNU/kFreeBSD,
we might need to add kFreeBSD to the GOOS (but this will duplicate some code).
BTW, one of the patch is unnecessary if we support detecting dynamic linker
automatically.
@oerdnj

This comment has been minimized.

oerdnj commented Apr 16, 2012

Comment 2:

Hi,
[Debian tracking bug for this: http://bugs.debian.org/668794]
I had a comment from our KFreeBSD porter which is very relevant to this issue:
On Mon, Apr 16, 2012 at 00:04, Robert Millan <rmh@d.o> wrote:
> Hi Ondřej,
>
> I notice you disabled the golang testsuite because it hangs on
> GNU/kFreeBSD.  However, the problem is still present, and chances are
> it makes golang unusable on that platform.
>
> I gave the source a look, and it seems that on GNU/kFreeBSD golang is
> playing with thread primitives, bypassing libc.  For example it
> invokes thr_new() kernel call directly, and also calls sigprocmask()
> to reset the signal mask in code that is clearly multithreaded [1].
>
> This means that either golang intends to completely replace
> libpthread, or it intends to play along with existing libpthread.  I'm
> not sure which one applies here, but in both cases there is a problem
> that needs to be fixed in golang.
>
> So please don't disable the testsuite.  If golang can't be built on
> GNU/kFreeBSD, unless you know it's a bug in the system libraries the
> problem needs to be fixed in golang.  If nobody can spend the time to
> fix it here, then you should consider not providing the package for
> kfreebsd-*.
>
> [1] On multithreaded programs, use of sigprocmask() is reserved to
> system libraries.
>
> --
> Robert Millan
@minux

This comment has been minimized.

Member

minux commented Apr 17, 2012

Comment 3:

On Mon, Apr 16, 2012 at 00:04, Robert Millan <rmh@d.o> wrote:
> I gave the source a look, and it seems that on GNU/kFreeBSD golang is
> playing with thread primitives, bypassing libc.  For example it
Yes.
> invokes thr_new() kernel call directly, and also calls sigprocmask()
> to reset the signal mask in code that is clearly multithreaded [1].
>
> This means that either golang intends to completely replace
> libpthread, or it intends to play along with existing libpthread.  I'm
Yes, when cgo isn't used, the go runtime intentionally bypasses libc and
libpthread, as we manage threads and memory ourselves.
> not sure which one applies here, but in both cases there is a problem
> that needs to be fixed in golang.
In a word, we tie libc/kernel combination to the OS, so AFAIK, we don't
support GNU/kFreeBSD now.
@oerdnj

This comment has been minimized.

oerdnj commented Apr 18, 2012

Comment 4:

Ok, in that case, I am going to drop GNU/KFreeBSD packages from Debian at this moment.
That would be probably best course of action to not leave user with broken Go at that
platform.
@minux

This comment has been minimized.

Member

minux commented Apr 18, 2012

Comment 5:

Should we add GNU/kFreeBSD as a new GOOS?

Labels changed: added priority-later, removed priority-triage.

Status changed to Accepted.

@rsc

This comment has been minimized.

Contributor

rsc commented Apr 18, 2012

Comment 6:

Does the build succeed if you export CGO_ENABLED=0 first?
@oerdnj

This comment has been minimized.

oerdnj commented Apr 18, 2012

Comment 7:

> Does the build succeed if you export CGO_ENABLED=0 first?
The build did succeed, it's the tests which hang and are killed.
Or are you also asking if the tests not hang with CGO_ENABLED=0? I can certainly try
that (tomorrow)...
@rsc

This comment has been minimized.

Contributor

rsc commented Apr 18, 2012

Comment 8:

:: Or are you also asking if the tests not hang with CGO_ENABLED=0?
yes
@minux

This comment has been minimized.

Member

minux commented Apr 19, 2012

Comment 9:

I installed a Debian GNU/kFreeBSD 6.0 amd64 (with FreeBSD kernel 8.1.1) on a VM, and
tested that with just 
this patch:
http://anonscm.debian.org/gitweb/?p=pkg-google/golang.git;a=blob;f=debian/patches/011-__FreeBSD_kernel__.patch;h=45be34f1229c2ac40875ad3241f1d367d726173f;hb=a90cb12bf263a06fbe2e08978a6010138b672260
CGO_ENABLED=0 could pass all the tests, so at least go can be packaged for GNU/kFreeBSD
amd64
(albeit without support for cgo).
I'm willing to help do the cgo port, but I must know the answer to this question:
should kfreebsd be a new GOOS, or we treat it as variant of FreeBSD and distinguish it
from FreeBSD
by 'defined(__FreeBSD_kernel__)' only when necessary?
(my guess is that, if we don't want to merge the port, the second option is better)
@minux

This comment has been minimized.

Member

minux commented Apr 20, 2012

Comment 10:

I've made CL 6086043 to address the hardcoded dynamic linker problem,
if it's merged, you can drop patch: 007-use_native_dynamic_linker_on_kfreebsd.patch
@minux

This comment has been minimized.

Member

minux commented Apr 20, 2012

Comment 11:

I'm afraid, the only remaining option is treat kfreebsd a new GOOS.
because we have a pkg/runtime/cgo/freebsd.c, which is only for FreeBSD's libc,
as it is compiled by 8c/6c, we can't use 'defined(__FreeBSD_kernel__)' trick.
@minux

This comment has been minimized.

Member

minux commented Apr 21, 2012

Comment 12:

the cgo port must wait until glibc kfreebsd has saner libpthread implementation,
or we use libpthread's synchronization primitives instead of umtx_op when using cgo.

Labels changed: added priority-someday, removed priority-later.

@rsc

This comment has been minimized.

Contributor

rsc commented Dec 4, 2013

Comment 13:

Labels changed: added repo-main.

@rsc

This comment has been minimized.

Contributor

rsc commented Mar 3, 2014

Comment 14:

Adding Release=None to all Priority=Someday bugs.

Labels changed: added release-none.

@oerdnj oerdnj added accepted labels Mar 3, 2014

@denji

This comment has been minimized.

denji commented Apr 9, 2015

$ env CGO_ENABLED=1 GOOS=freebsd GOARCH=amd64  go build -tags "sqlite redis memcache cert" -v -o gogs_bsd64
...
# runtime/cgo
/usr/local/Cellar/go/HEAD/libexec/src/runtime/cgo/gcc_freebsd_amd64.c:37:2:
error: implicit declaration of function 'SIGFILLSET' is invalid in C99 [-Werror,-Wimplicit-function-declaration]

/usr/local/Cellar/go/HEAD/libexec/src/runtime/cgo/gcc_freebsd_amd64.c:37:13:
error: variable 'ign' is uninitialized when used here [-Werror,-Wuninitialized]

/usr/local/Cellar/go/HEAD/libexec/src/runtime/cgo/gcc_freebsd_amd64.c:32:14:
note: initialize the variable 'ign' to silence this warning
~ clang --version
Apple LLVM version 6.1.0 (clang-602.0.42) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.3.0
Thread model: posix
~ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.10.3
BuildVersion:   14D130a

@minux minux added this to the Unplanned milestone Apr 9, 2015

@minux

This comment has been minimized.

Member

minux commented Apr 9, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment