x/build: set up a qemu builder

[gopherbot]

by m@capitanio.org:

I noticed a significant improvements in the arm tool chain :-)
I think it makes now a sense to take a closer look at bugs
in different environments.

The first shot is a qemu in user mode:
qemu-arm version 0.12.5
The host is (ubuntu 10.10)
GOARCH=arm
GORUN=qemu-arm
hg identify ba58b167f1fc+ tip

I tried with disabled peep or with GOARM=5, the bugs are similar.

Some syscalls are not covered, (I would propose
a QUEMU_UM to disable tests that use them):

(epoll_create)
        netchan\
        rpc\
        websocket\

Some packages show a non-deterministic behaviour,
a few do always break. I have no clue if qemu, go or
both are buggy:

        go/printer\
        compress/flate\
        compress/gzip\
        compress/zlib\
        os\
        big\
        rpc/jsonrpc\
        sync\ #(+ run.bash)
        bytes\

After disabling them, the test-suite *passed* all remaining tests.

----

Script bugs:
        run.bash: cmd/ebnflint ???
        run.bash: doc/progs [ "$GOHOSTOS" == windows ] ???
        run.bash: doc/codelab/wiki

The test/run script fails with:
./run -v
sed: can't read ./tmp.go: No such file or directory
fail: chan/powser2.go
fail: chan/select3.go
109a110,119
> =========== chan/powser2.go
> throw: malloc/free - deadlock
> 
> [signal 0xb code=0x1 addr=0x0 pc=0x15720]
> 
> 
> =========== chan/select3.go
> qemu-arm: /build/buildd/qemu-kvm-0.12.5+noroms/cpu-exec.c:784: handle_cpu_signal:
Assertion `__ret == (abi_ulong)__ret' failed.
> $RUNFILE: line 1: PID Aborted                 $E ./$A.out
> 
158a169,174
> 
> =========== fixedbugs/bug243.go
> Unsupported setsockopt level=41 optname=26 
> qemu: Unsupported syscall: 250
> Start pollServer: epoll_create: function not implemented
> Unsupported setsockopt level=41 optname=26 
1 known bugs; 0 unexpected bugs; test output differs


The captured outputs are attached.

Attachments:

1. qemu-um-bugs.txt (26808 bytes)

[gopherbot]

Comment 1 by m@capitanio.org:

I forgot to clean up the environment. It was run
with GOMAXPROCS=2.
With GOMAXPROCS=1 everything passes except the 3 epoll
cases
        netchan\
        rpc\
        websocket\
and the os test
5g -o _gotest_.5 dir_linux.go error.go env.go exec.go file.go getwd.go path.go proc.go
stat_linux.go time.go types.go  env_unix.go file_unix.go sys_linux.go exec_unix.go  
rm -f _test/os.a
gopack grc _test/os.a _gotest_.5 
--- FAIL: os_test.TestSeek (0.0 seconds)
    #6: Seek(8589934592, 0) = 0, <nil> want 8589934592, nil
    #7: Seek(8589934592, 2) = 13, <nil> want 8589934605, nil
FAIL

[gopherbot]

Comment 2 by m@capitanio.org:

I also forgot to unset DISABLE_NET_TESTS,
so all the non-working tests are (obviously):
        net\
        netchan\
        rpc\
        websocket\
        http\
        syslog\
        os\
----
Non-working benchmarks scripts:
cd pkg && make bench
...
...
cd ../../test/garbage && make bench
make[1]: Entering directory `/data4/soft/go/go/test/garbage'
5g parser.go stats.go
5l -o parser.out parser.5
./parser.out
./parser.out: 1: Syntax error: word unexpected (expecting ")")
make[1]: *** [parser.bench] Error 2
rm parser.5
make[1]: Leaving directory `/data4/soft/go/go/test/garbage'
make: *** [../../test/garbage.bench] Error 2
and
test/bench/timing.sh

[rsc]

Comment 3:

I run using the user-mode qemu all the time, but I use
the kernel binfmt misc support instead of setting $GORUN.
With DISABLE_NET_TESTS=1 all.bash runs to completion
on my linux/amd64 system.
The benchmark script looks like it just needs to use $GORUN.

Owner changed to r...@golang.org.

Status changed to HelpWanted.

[gopherbot]

Comment 4 by m@capitanio.org:

Thanks, indeed the binfmt misc support works much better. With GOMAXPROCS=1
and DISABLE_NET_TESTS=1 it passed all the tests.

[rsc]

Comment 5:

If you'd like to prepare a patch to insert $GORUN where it's needed, feel free.

[gopherbot]

Comment 6 by m@capitanio.org:

I apologize, I was wrong, in the comment #4 the tests wasn't enabled.
Actually I can't found any qemu user-mode version that supports
epoll_create. How did you setup the binfmt?

[gopherbot]

Comment 7 by m@capitanio.org:

http://code.google.com/p/go/source/detail?r=231c6b6747
almost fixes the GOMAXPROCS>1 random issues in comment #0
The success rate is much better, but not 100%.
The pkg/go/printer test breaks always:
throw: malloc/free - deadlock

Attachments:

1. go-qemu-arm-mp2.txt (16819 bytes)

[gopherbot]

Comment 8 by stephenm@golang.org:

Labels changed: added arch-arm.

[rsc]

Comment 9:

Labels changed: added priority-low, removed priority-medium.

[rsc]

Comment 10:

Labels changed: added priority-someday, removed priority-low.

[minux]

Comment 11:

Issue #5384 has been merged into this issue.

[rsc]

Comment 12:

Labels changed: added repo-main.

[rsc]

Comment 13:

Labels changed: added release-none.