From 7a41b5575dc948ae334138b7334437cf6025df17 Mon Sep 17 00:00:00 2001 From: hulkoba Date: Mon, 11 Dec 2023 10:26:12 +0100 Subject: [PATCH] docs(fuzz): add readme --- .gitignore | 1 + test/fuzz/readme.md | 71 +++++++++++++++++++ ...b4b0d3255bfef95601890afd80709 => .gitkeep} | 0 3 files changed, 72 insertions(+) create mode 100644 test/fuzz/readme.md rename test/fuzz/reports/{crash-da39a3ee5e6b4b0d3255bfef95601890afd80709 => .gitkeep} (100%) diff --git a/.gitignore b/.gitignore index 2085d61f0..65730d87d 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,7 @@ node_modules/ npm* test/lib/ +test/typescript/definitions.js dist/ openpgp.store/ coverage diff --git a/test/fuzz/readme.md b/test/fuzz/readme.md new file mode 100644 index 000000000..9d28be04c --- /dev/null +++ b/test/fuzz/readme.md @@ -0,0 +1,71 @@ +# Fuzz Testing + +Fuzz testing is + +> An automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a program. + +This `fuzz/` directory contains the fuzz tests for openpgp.js. +To generate and run fuzz tests, we use the [Jazzer.js](https://github.com/CodeIntelligenceTesting/jazzer.js/) library. + +## Running a fuzzer + +This directory contains fuzz targets like for example `createMessageBinary`. + +You can run this fuzz target without options: +```sh +npx jazzer test/fuzz/createMessageBinary.cjs +``` +(You will notice the `.cjs` file ending. This is because jazzer does not support esm, yet) + +or with the given settings at your package.json: + +```sh +TARGET=createMessageBinary npm run fuzz +``` + +Notice, that `TARGET` is the path to your fuzz target module. + +You should see the fuzzer that looks similar to this: + +``` +#128 pulse corp: 1/1b lim: 4 exec/s: 64 rss: 173Mb +#256 pulse corp: 1/1b lim: 6 exec/s: 51 rss: 173Mb +#512 pulse corp: 1/1b lim: 8 exec/s: 46 rss: 174Mb +#1024 pulse corp: 1/1b lim: 14 exec/s: 40 rss: 178Mb +#2048 pulse corp: 1/1b lim: 21 exec/s: 40 rss: 179Mb +``` + +It will continue to generate random inputs forever, until it finds a bug or is terminated. +The testcases for bugs it finds can be seen in the form of `crash-*`, `timeout-*` or `oom-*` at `test/fuzz/reports`. + +## Running with coverage +To generate a coverage report, run the `fuzz-coverage` script from the package.json: +```sh +TARGET=createMessageBinary npm run fuzz-coverage +``` + +The coverage reports can be found in `test/fuzz/coverage`. + + +## The fuzz target module +All functions that need to be fuzz-tested are here, at the `test/fuzz/` directory. + +A fuzz target module needs to export a function called fuzz, +which takes a Buffer parameter and executes the actual code under test. + +Jazzer.js provides the wrapper class FuzzedDataProvider, which allows reading primitive types from the Buffer. + +See further details in [Fuzzing using fuzz targets and the CLI](https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/docs/fuzz-targets.md) or [Advanced Fuzzing Settings](https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/docs/fuzz-settings.md#advanced-fuzzing-settings) + + +### Run limitations + +You can pass the `-max_total_time` flag to the internal fuzzing engine to stop the fuzzing run after 10 seconds. +```sh +npx jazzer test/fuzz/createMessageBinary.cjs -- -max_total_time=10 +``` + +Or you can limit the number of runs: +```sh +npx jazzer test/fuzz/createMessageBinary.cjs -- -runs=4000000 +``` \ No newline at end of file diff --git a/test/fuzz/reports/crash-da39a3ee5e6b4b0d3255bfef95601890afd80709 b/test/fuzz/reports/.gitkeep similarity index 100% rename from test/fuzz/reports/crash-da39a3ee5e6b4b0d3255bfef95601890afd80709 rename to test/fuzz/reports/.gitkeep