Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch OSS projects to use native go-fuzz. #3638

Merged
merged 6 commits into from
Apr 14, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions docs/getting-started/new-project-guide/go_lang.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,18 @@ process. The key specifics of integrating a Go project are outlined below.
## Go-fuzz support

OSS-Fuzz supports **go-fuzz** in the
[libFuzzer compatible mode](https://github.com/dvyukov/go-fuzz#libfuzzer-support)
[libFuzzer compatible mode](https://github.com/mdempsky/go114-fuzz-build)
only. In that mode, fuzz targets for Go use the libFuzzer engine with native Go
coverage instrumentation. Binaries compiled in this mode provide the same
libFuzzer command line interface as non-Go fuzz targets.

## Project files

First, you need to write a Go fuzz target that accepts a stream of bytes and
calls the program API with that. This fuzz target should reside in your project
repository
([example](https://github.com/golang/go/blob/4ad13555184eb0697c2e92c64c1b0bdb287ccc10/src/html/fuzz.go#L13)).

The structure of the project directory in OSS-Fuzz repository doesn't differ for
projects written in Go. The project files have the following Go specific
aspects.
Expand Down Expand Up @@ -66,7 +71,7 @@ RUN go get github.com/ianlancetaylor/demangle

### build.sh

In order to build a Go fuzz target, you need to call `go-fuzz-build -libfuzzer`
In order to build a Go fuzz target, you need to call `go-fuzz`
command first, and then link the resulting `.a` file against
`$LIB_FUZZING_ENGINE` using the `$CXX $CXXFLAGS ...` command.
[Example](https://github.com/google/oss-fuzz/blob/356f2b947670b7eb33a1f535c71bc5c87a60b0d1/projects/syzkaller/build.sh#L19):
Expand All @@ -78,7 +83,7 @@ function compile_fuzzer {
fuzzer=$3

# Instrument all Go files relevant to this fuzzer
go-fuzz-build -libfuzzer -func $function -o $fuzzer.a $path
go-fuzz -func $function -o $fuzzer.a $path

# Instrumented, compiled Go ($fuzzer.a) + fuzzing engine = fuzzer binary
$CXX $CXXFLAGS $LIB_FUZZING_ENGINE $fuzzer.a -lpthread -o $OUT/$fuzzer
Expand Down
2 changes: 1 addition & 1 deletion projects/go-attestation/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ function compile_fuzzer {
fuzzer=$3

# Instrument all Go files relevant to this fuzzer
go-fuzz-build -libfuzzer -func $function -o $fuzzer.a $package
go-fuzz -func $function -o $fuzzer.a $package

# Instrumented, compiled Go ($fuzzer.a) + fuzzing engine = fuzzer binary
$CXX $CXXFLAGS $LIB_FUZZING_ENGINE $fuzzer.a -lpthread -o $OUT/$fuzzer
Expand Down
2 changes: 1 addition & 1 deletion projects/go-dns/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ function compile_fuzzer {
fuzzer=$3

# Instrument all Go files relevant to this fuzzer
go-fuzz-build -tags fuzz -libfuzzer -func $function -o $fuzzer.a $path
go-fuzz -tags fuzz -func $function -o $fuzzer.a $path

# Instrumented, compiled Go ($fuzzer.a) + fuzzing engine = fuzzer binary
$CXX $CXXFLAGS $LIB_FUZZING_ENGINE $fuzzer.a -lpthread -o $OUT/$fuzzer
Expand Down
6 changes: 3 additions & 3 deletions projects/go-json-iterator/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ FROM gcr.io/oss-fuzz-base/base-builder
MAINTAINER taowen@gmail.com
RUN go get github.com/json-iterator/go

RUN mkdir fuzz
COPY fuzz_json.go fuzz/
RUN mkdir $GOPATH/src/fuzz
COPY fuzz_json.go $GOPATH/src/fuzz
COPY build.sh $SRC/
WORKDIR fuzz
WORKDIR $GOPATH/src/fuzz
4 changes: 2 additions & 2 deletions projects/go-json-iterator/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ function compile_fuzzer {
fuzzer=$3

# Instrument all Go files relevant to this fuzzer
go-fuzz-build -libfuzzer -func $function -o $fuzzer.a $path
go-fuzz -func $function -o $fuzzer.a $path

# Instrumented, compiled Go ($fuzzer.a) + fuzzing engine = fuzzer binary
$CXX $CXXFLAGS $LIB_FUZZING_ENGINE $fuzzer.a -lpthread -o $OUT/$fuzzer
}

compile_fuzzer . Fuzz fuzz_json
compile_fuzzer fuzz Fuzz fuzz_json
16 changes: 15 additions & 1 deletion projects/golang/build.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,22 @@
# Copyright 2020 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

function compile_fuzzer {
fuzzer=$(basename $1)

# Instrument all Go files relevant to this fuzzer, compile and store in $fuzzer.a
go-fuzz-build -libfuzzer -o $fuzzer.a github.com/dvyukov/go-fuzz-corpus/$fuzzer
go-fuzz -o $fuzzer.a github.com/dvyukov/go-fuzz-corpus/$fuzzer

# Instrumented, compiled Go ($fuzzer.a) + libFuzzer = fuzzer binary
$CXX $CXXFLAGS $LIB_FUZZING_ENGINE $fuzzer.a -lpthread -o fuzzer-$fuzzer
Expand Down
2 changes: 1 addition & 1 deletion projects/gonids/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ function compile_fuzzer {
fuzzer=$3

# Instrument all Go files relevant to this fuzzer
go-fuzz-build -libfuzzer -func $function -o $fuzzer.a $path
go-fuzz -func $function -o $fuzzer.a $path

# Instrumented, compiled Go ($fuzzer.a) + fuzzing engine = fuzzer binary
$CXX $CXXFLAGS $LIB_FUZZING_ENGINE $fuzzer.a -lpthread -o $OUT/$fuzzer
Expand Down
18 changes: 16 additions & 2 deletions projects/kubernetes/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,24 @@
# Copyright 2020 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
################################################################################

FROM gcr.io/oss-fuzz-base/base-builder

ENV GOPATH /gopath

RUN go get -u -d github.com/dvyukov/go-fuzz/...
RUN go get github.com/ianlancetaylor/demangle

RUN git clone --depth 1 https://github.com/kubernetes/kubernetes.git /gopath/src/k8s.io/kubernetes

WORKDIR /gopath/src/k8s.io/kubernetes
Expand Down
2 changes: 1 addition & 1 deletion projects/kubernetes/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ function compile_fuzzer {
local fuzzer="${pkg}_${function}"

# Instrument all Go files relevant to this fuzzer
go-fuzz-build -libfuzzer -func "${function}" -o "${fuzzer}.a" "k8s.io/kubernetes/test/fuzz/${pkg}"
go-fuzz -func "${function}" -o "${fuzzer}.a" "k8s.io/kubernetes/test/fuzz/${pkg}"

# Instrumented, compiled Go ($fuzzer.a) + fuzzing engine = fuzzer binary
$CXX $CXXFLAGS $LIB_FUZZING_ENGINE "${fuzzer}.a" -lpthread -o "${OUT}/${fuzzer}"
Expand Down
4 changes: 2 additions & 2 deletions projects/syzkaller/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ RUN go get -u -d github.com/google/syzkaller/...
# Dependency for one of the fuzz targets.
RUN go get github.com/ianlancetaylor/demangle

RUN git clone --depth 1 https://github.com/google/syzkaller.git syzkaller
WORKDIR syzkaller
RUN git clone --depth 1 https://github.com/google/syzkaller.git $GOPATH/src/syzkaller
WORKDIR $GOPATH/src/syzkaller
COPY build.sh $SRC/
10 changes: 5 additions & 5 deletions projects/syzkaller/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,16 @@ function compile_fuzzer {
fuzzer=$3

# Instrument all Go files relevant to this fuzzer
go-fuzz-build -libfuzzer -func $function -o $fuzzer.a $path
go-fuzz-build -libfuzzer -func $function -o $fuzzer.a $path

# Instrumented, compiled Go ($fuzzer.a) + fuzzing engine = fuzzer binary
$CXX $CXXFLAGS $LIB_FUZZING_ENGINE $fuzzer.a -lpthread -o $OUT/$fuzzer
}

compile_fuzzer ./pkg/compiler Fuzz compiler_fuzzer
compile_fuzzer ./prog/test FuzzDeserialize prog_deserialize_fuzzer
compile_fuzzer ./prog/test FuzzParseLog prog_parselog_fuzzer
compile_fuzzer ./pkg/report Fuzz report_fuzzer
compile_fuzzer syzkaller/pkg/compiler Fuzz compiler_fuzzer
compile_fuzzer syzkaller/prog/test FuzzDeserialize prog_deserialize_fuzzer
compile_fuzzer syzkaller/prog/test FuzzParseLog prog_parselog_fuzzer
compile_fuzzer syzkaller/pkg/report Fuzz report_fuzzer

# This target is way too spammy and OOMs very quickly.
# compile_fuzzer ./tools/syz-trace2syz/proggen Fuzz trace2syz_fuzzer