Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compilation issue due to bls dependency #20

Closed
pventuzelo opened this issue Apr 8, 2020 · 9 comments
Closed

Compilation issue due to bls dependency #20

pventuzelo opened this issue Apr 8, 2020 · 9 comments

Comments

@pventuzelo
Copy link

pventuzelo commented Apr 8, 2020

Hi,

I tried to compile zrnt with go-fuzz but when i got this issue:

$ go-fuzz-build -tags "preset_mainnet" . 
failed to execute go build: exit status 2
# github.com/herumi/bls-eth-go-binary/bls
/tmp/go-fuzz-build338426294/gopath/src/github.com/herumi/bls-eth-go-binary/bls/bls.go:16:10: fatal error: mcl/bn_c384_256.h: No such file or directory
 #include <mcl/bn_c384_256.h>
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.

Is it possible to provide a way (like build flags) to switch bls library from herumi to another one?
Or do you know which command line can fix this issue?

thx,

@protolambda
Copy link
Owner

protolambda commented Apr 8, 2020

Hmm, looks like a file in herumi is missing. How are you including dependencies? Herumi has a Go repo that provides prebuilt binaries that get linked in. Something may be going wrong there. Can you build zrnt on your system with the regular git clone and go get? Let's confirm if it's build setup, or a broken dependency.

@pventuzelo
Copy link
Author

pventuzelo commented Apr 8, 2020

I just create a fuzz.go file:

package fuzz

import (
    "bytes"
)

import (
	"github.com/protolambda/zrnt/eth2/phase0"
	"github.com/protolambda/zssz"
)

func Fuzz(data []byte) int {

	var state phase0.BeaconState
    reader := bytes.NewReader(data)
	if err := zssz.Decode(reader, uint64(len(data)), &state, phase0.BeaconStateSSZ); err != nil {
		return 0
	}

	ffstate := phase0.NewFullFeaturedState(&state)
	ffstate.LoadPrecomputedData()
	ffstate.CurrentProposer()
	return 1
}

Nothing fancy here.

Regarding command line:

git clone --recursive https://github.com/protolambda/zrnt
cd zrnt
go get -tags preset_mainnet ./...
cp ../../fuzz.go .
go-fuzz-build -tags "preset_mainnet" .

@gnattishness
Copy link

gnattishness commented Apr 8, 2020

I've run into similar while upgrading beacon-fuzz to v0.11.1.
From what I understand, go mod vendor won't copy the herumi lib and include directories because they contain no go files.
Apparently cgo is only officially supported (at least for go build to detect that a binary needs rebuilding) when the all the c files are in the same directory as the go ones.

I have yet to confirm whether go-fuzz uses go mod vendor when using "go modules" mode (GO111MODULE=on), but the error is the same.

@protolambda
Copy link
Owner

protolambda commented Apr 8, 2020

Yes, with -work you can see the temporary build files, and there's no CGO file at all in there anywhere, so logically this fails. CGO doesn't seem supported by go-fuzz currently: dvyukov/go-fuzz#101

@gnattishness
Copy link

gnattishness commented Apr 8, 2020

A dodgy workaround can apparently involve adding empty .go files to the relevant directories.
Ah here's the issue I was trying to find:
golang/go#26366

And relevant comment:
golang/go#26366 (comment)
https://groups.google.com/forum/#!topic/golang-nuts/_GWRF1KIvFw

And perhaps an alternative tooling solution:
https://github.com/go-modules-by-example/index/blob/master/012_modvendor/README.md
golang/go#26366 (comment)

@protolambda
Copy link
Owner

protolambda commented Apr 8, 2020

By all means, go ahead. Herumi was pretty welcoming to my earlier PRs to herumi-BLS to improve the build system. If adding dummy go files helps vendoring, then you can make it work.

@protolambda
Copy link
Owner

protolambda commented Jun 6, 2020

Is this still a problem, I am not sure if I missed some information in other channels?

@gnattishness
Copy link

gnattishness commented Jun 7, 2020

+1 to close
It can build fine with the go1.14 go build instrumentation.
Some modifications could make it usable for go-fuzz-build -libfuzzer but, in any case, would be asssociated with the herumi dependency, not ZRNT.

@pventuzelo
Copy link
Author

pventuzelo commented Jun 8, 2020

All good for me ;)
as mention by @gnattishness, we found a workaround to be able to make the fuzzing working using go1.14.

Thanks @protolambda

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants