Compilation issue due to bls dependency

[pventuzelo]

Hi,

I tried to compile zrnt with go-fuzz but when i got this issue:

$ go-fuzz-build -tags "preset_mainnet" . 
failed to execute go build: exit status 2
# github.com/herumi/bls-eth-go-binary/bls
/tmp/go-fuzz-build338426294/gopath/src/github.com/herumi/bls-eth-go-binary/bls/bls.go:16:10: fatal error: mcl/bn_c384_256.h: No such file or directory
 #include <mcl/bn_c384_256.h>
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.

Is it possible to provide a way (like build flags) to switch bls library from herumi to another one?
Or do you know which command line can fix this issue?

thx,

[protolambda]

Hmm, looks like a file in herumi is missing. How are you including dependencies? Herumi has a Go repo that provides prebuilt binaries that get linked in. Something may be going wrong there. Can you build zrnt on your system with the regular git clone and go get? Let's confirm if it's build setup, or a broken dependency.

[pventuzelo]

I just create a fuzz.go file:

package fuzz

import (
    "bytes"
)

import (
	"github.com/protolambda/zrnt/eth2/phase0"
	"github.com/protolambda/zssz"
)

func Fuzz(data []byte) int {

	var state phase0.BeaconState
    reader := bytes.NewReader(data)
	if err := zssz.Decode(reader, uint64(len(data)), &state, phase0.BeaconStateSSZ); err != nil {
		return 0
	}

	ffstate := phase0.NewFullFeaturedState(&state)
	ffstate.LoadPrecomputedData()
	ffstate.CurrentProposer()
	return 1
}

Nothing fancy here.

Regarding command line:

git clone --recursive https://github.com/protolambda/zrnt
cd zrnt
go get -tags preset_mainnet ./...
cp ../../fuzz.go .
go-fuzz-build -tags "preset_mainnet" .

[gnattishness]

I've run into similar while upgrading beacon-fuzz to v0.11.1.
From what I understand, go mod vendor won't copy the herumi lib and include directories because they contain no go files.
Apparently cgo is only officially supported (at least for go build to detect that a binary needs rebuilding) when the all the c files are in the same directory as the go ones.

I have yet to confirm whether go-fuzz uses go mod vendor when using "go modules" mode (GO111MODULE=on), but the error is the same.

[protolambda]

Yes, with -work you can see the temporary build files, and there's no CGO file at all in there anywhere, so logically this fails. CGO doesn't seem supported by go-fuzz currently: dvyukov/go-fuzz#101

[gnattishness]

A dodgy workaround can apparently involve adding empty .go files to the relevant directories.
Ah here's the issue I was trying to find:
golang/go#26366

And relevant comment:
golang/go#26366 (comment)
https://groups.google.com/forum/#!topic/golang-nuts/_GWRF1KIvFw

And perhaps an alternative tooling solution:
https://github.com/go-modules-by-example/index/blob/master/012_modvendor/README.md
golang/go#26366 (comment)

[protolambda]

By all means, go ahead. Herumi was pretty welcoming to my earlier PRs to herumi-BLS to improve the build system. If adding dummy go files helps vendoring, then you can make it work.

[protolambda]

Is this still a problem, I am not sure if I missed some information in other channels?

[gnattishness]

+1 to close
It can build fine with the go1.14 go build instrumentation.
Some modifications could make it usable for go-fuzz-build -libfuzzer but, in any case, would be asssociated with the herumi dependency, not ZRNT.

[pventuzelo]

All good for me ;)
as mention by @gnattishness, we found a workaround to be able to make the fuzzing working using go1.14.

Thanks @protolambda