make test fails on Ubuntu 16.04

[unidesigner]

I am trying to compile current master branch of DVID on an Ubuntu 16.04 system. Starting from empty directories, I'm running into issue with make test after successfully compiling DVID with make dvid. Please see the make test output: make_test_output.txt

Any idea what could cause these problems? Thanks!

[unidesigner]

Could the problem be due to golang version 1.6 in Ubuntu 16.04, but 1.5 is recommended?

[DocSavage]

This looks like some problem with the lz4 library compiled into dvid:

`fatal error: unexpected signal during runtime execution
[signal 0xb code=0x80 addr=0x0 pc=0x8ee8b0]

runtime stack:
runtime.throw(0xbf9ae0, 0x2a)
/usr/lib/go-1.6/src/runtime/panic.go:530 +0x90
runtime.sigpanic()
/usr/lib/go-1.6/src/runtime/sigpanic_unix.go:12 +0x5a

goroutine 396 [syscall, locked to thread]:
runtime.cgocall(0x8fb860, 0xc820ee2bb0, 0x0)
/usr/lib/go-1.6/src/runtime/cgocall.go:123 +0x11b fp=0xc820ee2b60 sp=0xc820ee2b30
github.com/janelia-flyem/go/golz4._Cfunc_LZ4_decompress_safe(0xc821016005, 0xc82104c000, 0x800000008082, 0x0)
github.com/janelia-flyem/go/golz4/_obj/_cgo_gotypes.go:79 +0x41 fp=0xc820ee2bb0 sp=0xc820ee2b60
github.com/janelia-flyem/go/golz4.Uncompress(0xc821016005, 0x8082, 0x8082, 0xc82104c000, 0x8000, 0x8000, 0x0, 0x0)
/home/stephan/go/src/github.com/janelia-flyem/go/golz4/lz4.go:30 +0xe1 fp=0xc820ee2c60 sp=0xc820ee2bb0
github.com/janelia-flyem/dvid/dvid.DeserializeData(0xc821016000, 0x8087, 0x8087, 0x2b4164f5ac80db01, 0x0, 0x0, 0x0, 0xeb4f4c19c0a5c174, 0x0, 0x0)
/home/stephan/go/src/github.com/janelia-flyem/dvid/dvid/serialize.go:318 +0xcaf fp=0xc820ee2e78 sp=0xc820ee2c60
`

I've been meaning to get my CI environment back up. The previous one, drone.io, appears to have been sunset. That should make it easier to catch these kind of issues.

The lz4 support is provided via cloudflare's golz4 package, which is at r126 of the main lz4 lib. The main lz4 lib is now at r131 (https://github.com/Cyan4973/lz4/releases) so it's possible that an issue was resolved between the versions. I'll look into it.

[DocSavage]

In the meantime, you might want to try the precompiled executable in the release:
https://github.com/janelia-flyem/dvid/releases/tag/v0.8.2

This might circumvent an issue with the new toolchain in your OS release.

[unidesigner]

Thanks for your quick help. I did not know about the precompiled executables. I just tested them and they seem to work as expected. I'll report here if I run into issues with those.

[DocSavage]

I can verify that "make test" fails on the lz4 routines when using Go 1.7 + gcc 6.1.1, however it's ok if using Go 1.7 + gcc 4.8.4. Unfortunately the go library I use (http://github.com/cloudflare/golz4) is lagging behind its embedded C code, and that C code (https://github.com/Cyan4973/lz4) is also lagging behind the cutting edge lz4. I'll consider moving to a Go implementation of lz4 that's more up-to-date (https://github.com/pierrec/lz4), but am worried about incompatibility between that and already stored lz4 serializations. I'll continue to use older gcc for binary builds for now.

[DocSavage]

Fixed in commit 7380190 by moving to pure Go lz4.

[DocSavage]

Master branch has reverted to older CGo lz4 due to issues. See issue #251.

[DocSavage]

This issue has been solved by updating the go lz4 library with new lz4 source code. See closed issue #251