fuzzing support #508

mattklein123 · 2017-02-25T18:11:40Z

We should start adding fuzz testing (probably using llvm). Some thinking needs to be put into how to get the most value out of this. (Which granularity to fuzz, etc.). At minimum it would be nice to fuzz not only the HTTP codecs, but also all the way through the downstream HTTP connection manager processing. Doing this in a sane way which also yields good coverage is not going to be trivial. Also need to figure out how/when to run the tests.

@PiotrSikora would you like to own this? (At least in terms of getting some stuff started and then educating the rest of us on best practices so that we can all contribute).

PiotrSikora · 2017-02-26T02:36:32Z

Yes, please assign this to me.

htuch · 2017-08-29T14:17:03Z

@amb67 suggested we take a look at http://lcamtuf.coredump.cx/afl/.

dnoe · 2017-08-29T15:37:12Z

Would this be something that runs during CI or is it too heavyweight?

htuch · 2017-08-29T15:46:24Z

We could definitely do some light fuzzing in CI (time boxed fuzzing of X mins), and then a heavier duty fuzz in nightly runs or elsewhere.

…

On Tue, Aug 29, 2017 at 11:37 AM Dan Noé ***@***.***> wrote: Would this be something that runs during CI or is it too heavyweight? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#508 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKaLv6t3PEHAnephIsDCWJ1eqKNarZ_Pks5sdDA0gaJpZM4MMFpg> .

mattklein123 · 2017-08-29T16:06:15Z

Assuming we get accepted to CNCF we will have have more resources to apply to CI for things like fuzzing/perf/etc. We should still plan for what we ultimately want to do though.

htuch · 2017-08-29T19:32:50Z

@PiotrSikora is this still on your radar, or should we tag "help wanted"?

htuch · 2018-01-24T22:11:16Z

I've started work on the new project setup for https://github.com/google/oss-fuzz. I'm working through a bunch of build issues, as we need to build Envoy in a Docker image with an oss-fuzz supplied compiler (looks to be clang-6.0) and its own set of CFLAGS/CXXFLAGS (including -stdlib=libc++). Will track progress here.

htuch · 2018-01-25T20:38:04Z

FTR the following pertinent restrictions exist in oss-fuzz today:

Single process model (https://github.com/google/oss-fuzz/blob/master/docs/faq.md#can-i-launch-an-additional-process-eg-a-daemon-from-my-fuzz-target)
We are restricted to /tmp for filesystem access (https://github.com/google/oss-fuzz/blob/master/docs/fuzzer_environment.md#file-system)
No network, including loopback (https://github.com/google/oss-fuzz/blob/master/docs/fuzzer_environment.md#network-access)

This implies that we can't make use of any of our existing integration tests with a fuzz frontend, it probably makes sense to focus on some unit tests for inspiration as well as new fuzz areas.

I'm going to start with just a simple Envoy::Base64 utility fuzzer to get the infrastructure in place (the encode/decode function are a direct match for the fuzz inputs https://llvm.org/docs/LibFuzzer.html#fuzz-target).

Following this, we can focus on configuration (protobuf) ingest and protocol handling on the data plane as the low hanging targets of interest. Any other suggestions welcome as well.

mattklein123 · 2018-01-25T21:55:47Z

@htuch the no network restriction is pretty unfortunate. I wonder if we could somehow fuzz/inject a fake client directly into the normal Envoy process maybe via a custom transport socket implementation?

…xample. This is the groundwork for envoyproxy#508. See test/fuzz/README.md for the spiel on what we're integrating with, the fuzz test model and how to create new fuzz tests. There were a bunch of changes needed to the build infrastructure to get it to work in the oss-fuzz Docker build environment. In particular, we needed to allow clang + libc++ static linking on Linux and to supply extra flags that some of the external deps were not handling correctly. Signed-off-by: Harvey Tuch <htuch@google.com>

#2476) This is the groundwork for #508. See test/fuzz/README.md for the spiel on what we're integrating with, the fuzz test model and how to create new fuzz tests. There were a bunch of changes needed to the build infrastructure to get it to work in the oss-fuzz Docker build environment. In particular, we needed to allow clang + libc++ static linking on Linux and to supply extra flags that some of the external deps were not handling correctly. Signed-off-by: Harvey Tuch <htuch@google.com>

htuch · 2018-04-23T16:29:45Z

Status here is we have server config fuzzing (server_fuzz_test) happening and are busy crunching through a bunch of crash reports. In addition, we have H1 wire fuzzing (h1_capture_fuzz_test) working, most of the crash reports are test infra related at this point. Some additional ideas that have come up in discussion for fuzzers:

Config validation mode fuzzing (similar to existing server_fuzz_test in structure).
One-way H1 fuzzing; skip the upstream aspect of h1_capture_fuzz_test and use direct replies.
Fuzzing H1 with router/HCM fuzzing combined, to support corpuses that have special configuration needs and explore the impact of configurations beyond h1_capture_fuzz_test.
Dumb H2 fuzzing (basically a h2_capture_fuzz_test).
Stateful and protocol aware (e.g. HPACK) H2 fuzzing
xDS (starting with EDS, since a lot of the xDS crashes would show up on the boostrap config in the existing server config fuzz test).
Header processing libraries
String libraries
Examine coverage reports & look at high value low hanging fruit.

This PR introduces a transport socket extension that wraps a given transport socket, interposes on its plain text traffic and records it into a proto trace file on the filesystem. This can be used for a number of purposes: 1. As a corpus for fuzzing the data plane. 2. Converted to PCAP using a soon-to-be-written utility, allowing existing tools such as Wireshark to be used to decode L4/L7 protocol history in the trace. Essentially this lets us take advantage of the PCAP ecosystem. Relates to envoyproxy#1413 and envoyproxy#508. Risk Level: Low (opt-in). Testing: New SSL integration tests, demonstrating plain text intercept. Signed-off-by: Harvey Tuch <htuch@google.com>

* tap/fuzz: transport socket extension for traffic capture. This PR introduces a transport socket extension that wraps a given transport socket, interposes on its plain text traffic and records it into a proto trace file on the filesystem. This can be used for a number of purposes: 1. As a corpus for fuzzing the data plane. 2. Converted to PCAP using a soon-to-be-written utility, allowing existing tools such as Wireshark to be used to decode L4/L7 protocol history in the trace. Essentially this lets us take advantage of the PCAP ecosystem. Relates to #1413 and #508. Risk Level: Low (opt-in). Testing: New SSL integration tests, demonstrating plain text intercept. Signed-off-by: Harvey Tuch <htuch@google.com>

htuch · 2018-06-07T16:57:17Z

#3493 just merged with the string lib fuzzing.

Fuzzer for the H2 codec. This is similar in structure to //test/common/http/http2:codec_impl_test, where a client H2 codec is wired via shared memory to a server H2 codec and stream actions are applied. We fuzz the various client/server H2 codec API operations and in addition apply fuzzing at the wire level by modeling explicit mutation, reordering and drain operations on the connection buffers between client and server. Part of envoyproxy#508. Risk Level: Low Testing: Tested with corpus under bazel test and under oss-fuzz Docker image. ~500 cases per second with python infra/helper.py build_fuzzers --sanitizer=address envoy <envoy path> && python infra/helper.py run_fuzzer envoy codec_impl_fuzz_test. Test corpus has 87.9% coverage of http2/codec_impl.cc. Signed-off-by: Harvey Tuch <htuch@google.com>

mattklein123 · 2018-08-01T20:34:11Z

Calling this complete. We can open individual issues on fuzzing tasks.

Fuzzer for the H2 codec. This is similar in structure to //test/common/http/http2:codec_impl_test, where a client H2 codec is wired via shared memory to a server H2 codec and stream actions are applied. We fuzz the various client/server H2 codec API operations and in addition apply fuzzing at the wire level by modeling explicit mutation, reordering and drain operations on the connection buffers between client and server. Part of #508. Risk Level: Low Testing: Tested with corpus under bazel test and under oss-fuzz Docker image. ~500 cases per second with python infra/helper.py build_fuzzers --sanitizer=address envoy <envoy path> && python infra/helper.py run_fuzzer envoy codec_impl_fuzz_test. Test corpus has 87.9% coverage of http2/codec_impl.cc. Signed-off-by: Harvey Tuch <htuch@google.com>

envoyproxy#508)

Signed-off-by: Alan Chiu <achiu@lyft.com> Description: ci: add master aar release dist workflow job Risk Level: low Testing: on master Docs Changes: n/a Release Notes: n/a [Optional Fixes #Issue] [Optional Deprecated:] Signed-off-by: JP Simard <jp@jpsim.com>

mattklein123 added the area/security label Feb 25, 2017

mattklein123 assigned PiotrSikora Apr 5, 2017

mattklein123 mentioned this issue Jul 5, 2017

Security Assessment #1204

Closed

htuch added the api/v2 label Aug 29, 2017

mattklein123 added the help wanted Needs help! label Oct 28, 2017

mattklein123 unassigned PiotrSikora Dec 25, 2017

htuch self-assigned this Jan 9, 2018

mattklein123 removed the api/v2 label Jan 15, 2018

htuch removed the help wanted Needs help! label Jan 24, 2018

This was referenced Jan 25, 2018

libc++ clarification google/oss-fuzz#1114

Closed

libc++ support for Clang + Linux bazelbuild/bazel#4524

Closed

htuch mentioned this issue Jan 30, 2018

fuzz: initial fuzzing infrastructure, docs and and Base64::encode() e… #2476

Merged

mattklein123 added this to the 1.6.0 milestone Feb 1, 2018

mattklein123 modified the milestones: 1.6.0, 1.7.0 Mar 5, 2018

mattklein123 added the help wanted Needs help! label Mar 16, 2018

mattklein123 removed this from the 1.7.0 milestone Mar 25, 2018

htuch mentioned this issue Apr 27, 2018

tap/fuzz: transport socket extension for traffic capture. #3244

Merged

mattklein123 added this to the 1.7.0 milestone May 12, 2018

htuch assigned anirudhmurali May 18, 2018

mattklein123 modified the milestones: 1.7.0, 1.8.0 May 28, 2018

htuch mentioned this issue Aug 1, 2018

fuzz: H2 codec fuzzer. #4017

Merged

mattklein123 closed this as completed Aug 1, 2018

wolfguoliang pushed a commit to wolfguoliang/envoy that referenced this issue Jan 23, 2021

zh-translation: docs/root/configuration/http/http_filters/ext_authz_f… (

1ba624e

envoyproxy#508)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fuzzing support #508

fuzzing support #508

mattklein123 commented Feb 25, 2017

PiotrSikora commented Feb 26, 2017

htuch commented Aug 29, 2017 •

edited

Loading

dnoe commented Aug 29, 2017

htuch commented Aug 29, 2017 via email

mattklein123 commented Aug 29, 2017

htuch commented Aug 29, 2017

htuch commented Jan 24, 2018

htuch commented Jan 25, 2018

mattklein123 commented Jan 25, 2018

htuch commented Apr 23, 2018 •

edited

Loading

htuch commented Jun 7, 2018

mattklein123 commented Aug 1, 2018

fuzzing support #508

fuzzing support #508

Comments

mattklein123 commented Feb 25, 2017

PiotrSikora commented Feb 26, 2017

htuch commented Aug 29, 2017 • edited Loading

dnoe commented Aug 29, 2017

htuch commented Aug 29, 2017 via email

mattklein123 commented Aug 29, 2017

htuch commented Aug 29, 2017

htuch commented Jan 24, 2018

htuch commented Jan 25, 2018

mattklein123 commented Jan 25, 2018

htuch commented Apr 23, 2018 • edited Loading

htuch commented Jun 7, 2018

mattklein123 commented Aug 1, 2018

htuch commented Aug 29, 2017 •

edited

Loading

htuch commented Apr 23, 2018 •

edited

Loading