Build error on Ubuntu Xenial with yara3.4

[gwillem]

Any ideas? Thanks for making this available!

$ go install -tags yara3.4
# github.com/hillu/go-yara
./rules_callback.go:89:7: could not determine kind of name for C.CALLBACK_MSG_MODULE_IMPORTED

$ dpkg -l | grep yara
ii  libyara-dev:amd64                           3.4.0+dfsg-2build1                           amd64        YARA development libraries and headers
ii  libyara3:amd64                              3.4.0+dfsg-2build1                           amd64        YARA shared library
ii  yara                                        3.4.0+dfsg-2build1                           amd64        Pattern matching swiss knife for malware researchers

$ go version
go version go1.10.1 linux/amd64

[hillu]

Uh-oh. I must have introduced that incompatibility when I rewrote the callback code in Go. Please check if commit ffb2dee fixes the problem for you.

[gwillem]

Thanks for the quick followup! But, doesn't seem to work yet:

$ cd $GOPATH/src/github.com/hillu/go-yara
$ git pull
$ git checkout ffb2dee29c87d40f4ce67d8c36dc90c69a7eafdb
$ export | grep CGO
$ go install -tags yara3.4
# github.com/hillu/go-yara
./rule.go:193: m.cptr.data undefined (type *_Ctype_struct__YR_MATCH has no field or method data)
./rule.go:193: m.cptr.data_length undefined (type *_Ctype_struct__YR_MATCH has no field or method data_length)
$ export CGO_CFLAGS="-I/usr/include/yara"
$ export CGO_LDFLAGS="-L/usr/lib/x86_64-linux-gnu -lyara"
$ go install -tags no_pkg_config -tags yara3.4
# github.com/hillu/go-yara
./rule.go:193: m.cptr.data undefined (type *_Ctype_struct__YR_MATCH has no field or method data)
./rule.go:193: m.cptr.data_length undefined (type *_Ctype_struct__YR_MATCH has no field or method data_length)

[gwillem]

With some trial and error I produced a static Linux GO build based on the latest Yara (3.7.1). If anyone else is interested:

Build Yara:

cd yara-3.7.1
./bootstrap.sh
./configure --disable-shared --enable-static --without-crypto
make

Build go-yara:

go get github.com/hillu/go-yara
cd $GOPATH/src/github.com/hillu/go-yara
export YARA_SRC=/home/willem/src/yara-3.7.1
export CGO_CFLAGS="-I${YARA_SRC}/libyara/include"
export CGO_LDFLAGS="-L${YARA_SRC}/libyara/.libs -lyara -lm"
go install -tags yara_static -tags no_pkg_config

Build your client:

cd $GOPATH/src/github.com/gwillem/yara-test
go build --ldflags '-extldflags "-static -lm"'

My resulting executable is ~4MB and can be run on systems without libyara. PS I don't use the crypto/hash features of Yara.

[hillu]

Ah well. Now I remember that the length had been renamed to data_length in YARA 3.5. And in trying to fix this, I re-discovered #5. Please check if commit c671230 works for you with yara 3.4.

[gwillem]

Thanks for your continued effort! Much appreciated.

Building go-yara now succeeds (using go install -tags yara3.4) , but when building my test app I get:

$ go build
# github.com/hillu/go-yara
../../code/golang/src/github.com/hillu/go-yara/compiler_addfile_yara36.go:37:19: could not determine kind of name for C.yr_compiler_add_fd

Your README.md suggests that it is looking at the wrong libyara files, but I have purged any manually installed version and ensured with strace that no other libyara files are accessed during the build process than the Ubuntu install:

$ strace -fe file go install -tags yara3.4 |& grep libyara
<nothing>

# my test app
$ strace -fe open go build |& grep libyara | grep -v ENOENT
[pid 28181] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 3
[pid 28181] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 3
[pid 28183] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 4
[pid 28183] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 4
[pid 28185] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 4
[pid 28185] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 4
[pid 28188] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 3
[pid 28188] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 3
[pid 28190] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 4
[pid 28190] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 4
[pid 28192] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 4
[pid 28192] open("/usr/include/yara/libyara.h", O_RDONLY|O_NOCTTY) = 4

The referenced header file contains:

#define YR_MAJOR_VERSION   3
#define YR_MINOR_VERSION   4
#define YR_MICRO_VERSION   0

[hillu]

@gwillem Instead of go build for your own application, you need to use go build -tags yara3.4 also. You don't want compiler_addfile_yara36.go to be built.

(The go toolchain hides the fact that behind the scenes, go-yara gets rebuilt on demand pretty well...)

[gwillem]

Success! Thanks so much!

I've rewritten parts of my text search app to use Yara. The performance has improved somewhat 😄:

BenchmarkScanNative-4   	       2	 818671777 ns/op
BenchmarkScanYara-4     	     300	   5751997 ns/op