Make CMake sanitizer agnostic #82

bshastry · 2019-08-20T07:21:42Z

Fixes a minor issue in #78 (see #78 (comment)) and another one (see google/oss-fuzz#2834 (comment))

bshastry · 2019-08-20T07:23:00Z

pwnall · 2019-08-21T00:39:22Z

The CI errors might be due to problems on the master branch. These problems were fixed on the latest master, so rebasing might help.

Dor1s · 2019-08-21T03:34:55Z

To make sure we're on the same page: when producing a build for OSS-Fuzz, we should use CFLAGS or CXXFLAGS for compilation , then link with clang++ and CXXFLAGS. None additional sanitizer / fuzzing flags are needed.

For CI, -fsanitize=address -fsanitize-coverage=trace-pc-guard is fine, though I'd rather suggest the following:

compile using -fsanitize=address,fuzzer-no-link
link using -fsanitize=address,fuzzer

bshastry · 2019-08-21T08:06:00Z

Update: Rebased and incorporated @Dor1s suggestion of separate compile/link flags.

bshastry · 2019-08-24T07:46:09Z

Once this PR is merged,I will create an ossfuzz integration

bshastry · 2019-08-25T08:06:09Z

Dear all, I created a branch locally to add snappy to ossfuzz and verified that all sanitizer builds (asan, ubsan, msan) are successful. Looking forward to the review of this PR. Thank you!

bshastry · 2019-08-30T04:23:44Z

Dear all, A gentle reminder that this PR is awaiting a review :-)

Thank you!

bshastry · 2019-09-06T06:36:33Z

Dear all, I'm waiting on a review to get this merged :-)

Thank you
Bhargava

bshastry · 2019-09-06T10:20:28Z

Rebased on current master

inferno-chromium · 2019-09-10T04:56:03Z

@pwnall - can you please help to review this latest rebased change. Thanks!

pwnall · 2019-09-17T17:12:34Z

Sorry for the radio silence here. I won't be able to tackle this for another week or so.

bshastry · 2019-09-20T08:22:57Z

Oops, spotted another mistake in this PR in the meantime :-)

Now, snappy builds should build transparently against choice of sanitizer-fuzzer :-)

pwnall · 2019-10-01T06:36:18Z

@bshastry @inferno-chromium @Dor1s I don't like the branching in this PR. I'd prefer that both the CMake config and the Travis config stay as simple as possible, and that it's easy to get a reasonable fuzzing build up in development.

I've prototyped an alternative at pwnall@a4b8446 in https://github.com/pwnall/snappy/tree/fix_fuzz -- TL;DR: the fuzzing flags get extracted to SNAPPY_FUZZING_FLAGS. ossfuzz can set SNAPPY_FUZZING_FLAGS="" and define its own flags with CXXFLAGS and LDFLAGS.

bshastry · 2019-10-01T07:09:46Z

@pwnall I had a quick look at your prototype. ~~Sadly, it is going to break afl build within ossfuzz because it is not linked against $LIB_FUZZING_ENGINE~~

I was mistaken. This can work provided we don't hard code the linking option to "-fsanitize=fuzzer" and write the build script correctly.

Also, I think ossfuzz expects one sanitizer per build. So one shouldn't set cxxflags to enable two sanitizers (e.g., address and undefined) at a given time. Moreover, memory and address sanitizer cannot be used together. For these reasons, the MSan CI on the prototype will likely fail as well

bshastry · 2019-10-01T07:56:05Z

@pwnall Rebased and tried to incorporate your feedback. Hope I got it right.

bshastry · 2019-10-11T12:54:01Z

Dear @pwnall

I readied the PR for a re-review. Looking forward.

Thank you,
Bhargava

bshastry · 2019-11-10T16:37:49Z

Dear all,

I was wondering if you had the chance to review this PR. Would be nice to see snappy fuzzed continuously :-)

Best
Bhargava

inferno-chromium · 2019-11-12T03:12:01Z

@pwnall - can you please take another look.

bshastry · 2019-11-20T18:58:34Z

Ping :-)

bshastry · 2019-12-06T09:17:14Z

Rebased.

Dor1s · 2019-12-06T16:24:39Z

@pwnall could you please take a look?

bshastry

Annotated PR with my notes.

bshastry · 2019-12-20T09:31:41Z

CMakeLists.txt

+option(SNAPPY_OSSFUZZ_BUILD "Build Snappy for oss-fuzz." OFF)
+
+set(SNAPPY_FUZZING_FLAGS "-fsanitize=address" CACHE STRING
+    "Compiler flags for fuzzing (e.g., -fsanitize=address).")


SNAPPY_FUZZING_FLAGS are used for the CI fuzzer build.

bshastry · 2019-12-20T09:32:32Z

CMakeLists.txt

@@ -45,6 +45,11 @@ option(SNAPPY_BUILD_TESTS "Build Snappy's own tests." ON)

 option(SNAPPY_FUZZING_BUILD "Build Snappy for fuzzing." OFF)

+option(SNAPPY_OSSFUZZ_BUILD "Build Snappy for oss-fuzz." OFF)


SNAPPY_OSSFUZZ_BUILD is set by the oss-fuzz builder. See https://github.com/google/oss-fuzz/pull/2874/files#diff-92672b1afbd2ebe31528dff0bd201f41R22-R23

bshastry · 2019-12-20T09:33:27Z

CMakeLists.txt

    message(WARNING "Fuzzing builds are only supported with Clang")
-  endif (NOT "${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
+  endif(NOT "${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")


Warn if either the CI fuzzing build or the ossfuzz build is not being compiled via clang

bshastry · 2019-12-20T09:34:28Z

CMakeLists.txt

+  if(NOT DEFINED ENV{LIB_FUZZING_ENGINE})
+    message(SEND_ERROR "OSS-Fuzz builds require LIB_FUZZING_ENGINE to be set")
+  endif(NOT DEFINED ENV{LIB_FUZZING_ENGINE})
+endif(SNAPPY_OSSFUZZ_BUILD)


Error if the ossfuzz build has not defined LIB_FUZZING_ENGINE environment variable.

bshastry · 2019-12-20T09:34:59Z

CMakeLists.txt

+  if(NOT CMAKE_CXX_FLAGS MATCHES "${SNAPPY_FUZZING_FLAGS}")
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${SNAPPY_FUZZING_FLAGS}")
+  endif(NOT CMAKE_CXX_FLAGS MATCHES "${SNAPPY_FUZZING_FLAGS}")
+endif(SNAPPY_FUZZING_BUILD)


Append fuzzing related compiler flags for the CI fuzzing build

bshastry · 2019-12-20T09:35:31Z

CMakeLists.txt

+  if(DEFINED ENV{LIB_FUZZING_ENGINE})
+    set_target_properties(snappy_compress_fuzzer
+      PROPERTIES LINK_FLAGS $ENV{LIB_FUZZING_ENGINE}
+    )


Link against LIB_FUZZING_ENGINE for the ossfuzz build

bshastry · 2019-12-20T09:35:45Z

CMakeLists.txt

+  else()
+    set_target_properties(snappy_compress_fuzzer
+      PROPERTIES LINK_FLAGS "-fsanitize=fuzzer"
+    )


Link against libfuzzer for the CI fuzzing build

bshastry · 2019-12-20T09:36:05Z

CMakeLists.txt

+  if(DEFINED ENV{LIB_FUZZING_ENGINE})
+    set_target_properties(snappy_uncompress_fuzzer
+            PROPERTIES LINK_FLAGS $ENV{LIB_FUZZING_ENGINE}
+    )


Same here and below

bshastry · 2020-01-17T07:11:27Z

Rebased.

bshastry · 2020-02-18T11:26:26Z

Friendly reminder that this PR is pending another round of reviews :-)

bshastry · 2020-04-03T16:02:07Z

Friendly reminder that this PR is pending review CC @inferno-chromium :-)

This PR blocks google/oss-fuzz#2874

bshastry · 2021-01-04T12:16:53Z

I think this PR may be closed now since integration is complete?

bshastry · 2021-01-23T11:24:07Z

Closing due to complete integration

googlebot added the cla: yes label Aug 20, 2019

bshastry force-pushed the libfuzzer-harness branch from 332bd43 to 254bdbb Compare August 21, 2019 08:04

bshastry force-pushed the libfuzzer-harness branch 5 times, most recently from 5947a54 to 6c7dc2b Compare August 21, 2019 14:02

bshastry force-pushed the libfuzzer-harness branch from 6c7dc2b to dd70a3c Compare September 6, 2019 10:20

bshastry force-pushed the libfuzzer-harness branch from dd70a3c to 1a77adf Compare September 20, 2019 08:20

bshastry force-pushed the libfuzzer-harness branch from 1a77adf to bb89c2c Compare September 21, 2019 13:41

bshastry mentioned this pull request Sep 21, 2019

[Snappy] Add to oss-fuzz google/oss-fuzz#2874

Closed

bshastry force-pushed the libfuzzer-harness branch from bb89c2c to 0a4f692 Compare October 1, 2019 07:55

bshastry force-pushed the libfuzzer-harness branch 3 times, most recently from 76a560b to f270149 Compare October 1, 2019 16:34

bshastry force-pushed the libfuzzer-harness branch from f270149 to 999ab17 Compare October 8, 2019 06:51

bshastry force-pushed the libfuzzer-harness branch from 999ab17 to c808648 Compare December 6, 2019 09:17

bshastry force-pushed the libfuzzer-harness branch from c808648 to 64eaea0 Compare December 6, 2019 09:21

bshastry commented Dec 20, 2019

View reviewed changes

Make CMake sanitizer and fuzzer agnostic

8d6d17a

bshastry force-pushed the libfuzzer-harness branch from 64eaea0 to 8d6d17a Compare January 17, 2020 07:11

bshastry closed this Jan 23, 2021

		@@ -45,6 +45,11 @@ option(SNAPPY_BUILD_TESTS "Build Snappy's own tests." ON)

		option(SNAPPY_FUZZING_BUILD "Build Snappy for fuzzing." OFF)

		option(SNAPPY_OSSFUZZ_BUILD "Build Snappy for oss-fuzz." OFF)

Make CMake sanitizer agnostic #82

Make CMake sanitizer agnostic #82

Conversation

bshastry commented Aug 20, 2019 • edited Loading

bshastry commented Aug 20, 2019

pwnall commented Aug 21, 2019

Dor1s commented Aug 21, 2019

bshastry commented Aug 21, 2019

bshastry commented Aug 24, 2019

bshastry commented Aug 25, 2019

bshastry commented Aug 30, 2019

bshastry commented Sep 6, 2019

bshastry commented Sep 6, 2019

inferno-chromium commented Sep 10, 2019

pwnall commented Sep 17, 2019

bshastry commented Sep 20, 2019

pwnall commented Oct 1, 2019

bshastry commented Oct 1, 2019 • edited Loading

bshastry commented Oct 1, 2019

bshastry commented Oct 11, 2019

bshastry commented Nov 10, 2019

inferno-chromium commented Nov 12, 2019

bshastry commented Nov 20, 2019

bshastry commented Dec 6, 2019

Dor1s commented Dec 6, 2019

bshastry left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bshastry commented Jan 17, 2020

bshastry commented Feb 18, 2020

bshastry commented Apr 3, 2020

bshastry commented Jan 4, 2021

bshastry commented Jan 23, 2021

bshastry commented Aug 20, 2019 •

edited

Loading

bshastry commented Oct 1, 2019 •

edited

Loading