Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AFL in zcutil (with all-in-one script) #4171

Merged
merged 7 commits into from Nov 6, 2019

Conversation

@defuse
Copy link
Contributor

defuse commented Oct 23, 2019

Supersedes #4156 and #4167.

Fuzzing targets and input sets are defined by the contents of directories in ./src/fuzzing/. Inside the directory, there's a fuzz.cpp and fuzz.h with a main() function that will replace zcashd's actual main() as well as an input subdirectory containing the inputs, one per file. To just run a fuzzer, you can, for example...

make clean # if you've previously build zcashd without AFL instrumentation
./zcutil/afl/afl-getbuildrun.sh DecodeHexTx

Alternatively you can...

./zcutil/afl/afl-get.sh /tmp/afl   # (or wherever you want to build AFL)
./zcutil/afl/afl-build.sh /tmp/afl DecodeHexTx -j$(nproc)
./zcutil/afl/afl-run.sh /tmp/afl DecodeHexTx

Run make clean whenever you switch between a normal build and an AFL-instrumented build.

@defuse defuse referenced this pull request Oct 23, 2019
@zebambam zebambam self-requested a review Oct 23, 2019
Copy link
Contributor

zebambam left a comment

This is really amazing work, I think it's going to set a whole heap of awesome in motion. There are a couple of things that I think need changing before it'll be useful though:

a. AFL-get, AFL-build, AFL-fuzz seem to be the stages, so let's keep those separate and then have an AFL-getbuildfuzz if someone wants to do them all at once.
b. AFL-fuzz will need to have a pass-through for arguments. For instance, on my machine the default 50 Megs wasn't big enough to load the monolith, so I'd need to pass -m 100M to AFL, but there's no way to do that currently. I think $@ in the right place is one way to do that, and a way that you've used elsewhere so it makes sense to me to reuse that approach for consistency.
c. There are 'on screen instructions' on how to go from AFL-get -> AFL-build, and that's great! Can you please add some for how to go AFL-build -> AFL-run? That way the process will be even easier, which I think is something we should shoot for so that devs need only to know the absolute minimum should they want to run this themselves.

Thanks again for doing this, I think it's great work and really needed.

@defuse

This comment has been minimized.

Copy link
Contributor Author

defuse commented Oct 23, 2019

@zebambam thanks for the suggestions, I updated the PR.

@zebambam

This comment has been minimized.

Copy link
Contributor

zebambam commented Oct 23, 2019

Legend!

Copy link
Contributor

zebambam left a comment

tested ACK

@zebambam

This comment has been minimized.

Copy link
Contributor

zebambam commented Oct 23, 2019

@zkbot try

@zkbot

This comment has been minimized.

Copy link
Collaborator

zkbot commented Oct 23, 2019

⌛️ Trying commit 0175475 with merge 6cddc2a...

zkbot added a commit that referenced this pull request Oct 23, 2019
Add AFL in zcutil (with all-in-one script)

Supersedes #4156 and #4167.

Fuzzing targets and input sets are defined by the contents of directories in `./src/fuzzing/`. Inside the directory, there's a `fuzz.cpp` and `fuzz.h` with a `main()` function that will replace `zcashd`'s actual `main()` as well as an `input` subdirectory containing the inputs, one per file. To just run a fuzzer, you can, for example...

```
make clean # if you've previously build zcashd without AFL instrumentation
./zcutil/afl/afl-fuzz.sh DecodeHexTx
```

Alternatively you can...

```
./zcutil/afl/afl-get.sh /tmp/afl   # (or wherever you want to build AFL)
./zcutil/afl/afl-build.sh /tmp/afl DecodeHexTx -j$(nproc)
./zcutil/afl/afl-run.sh /tmp/afl DecodeHexTx
```

Run `make clean` whenever you switch between a normal build and an AFL-instrumented build.
@defuse defuse referenced this pull request Oct 23, 2019
@zebambam

This comment has been minimized.

Copy link
Contributor

zebambam commented Oct 23, 2019

Small point because this is a plumbing PR, but the one that I tested isn't finding new paths..

` american fuzzy lop 2.52b (zcashd)

┌─ process timing ─────────────────────────────────────┬─ overall results ─────┐
│ run time : 0 days, 0 hrs, 42 min, 52 sec │ cycles done : 1502 │
│ last new path : none yet (odd, check syntax!) │ total paths : 4 │
│ last uniq crash : none seen yet │ uniq crashes : 0 │
│ last uniq hang : none seen yet │ uniq hangs : 0 │
├─ cycle progress ────────────────────┬─ map coverage ─┴───────────────────────┤
│ now processing : 3* (75.00%) │ map density : 6.82% / 6.82% │
│ paths timed out : 0 (0.00%) │ count coverage : 1.00 bits/tuple │
├─ stage progress ────────────────────┼─ findings in depth ────────────────────┤
│ now trying : havoc │ favored paths : 1 (25.00%) │
│ stage execs : 66/256 (25.78%) │ new edges on : 1 (25.00%) │
│ total execs : 1.54M │ total crashes : 0 (0 unique) │
│ exec speed : 597.1/sec │ total tmouts : 1 (1 unique) │
├─ fuzzing strategy yields ───────────┴───────────────┬─ path geometry ────────┤
│ bit flips : 0/128, 0/124, 0/116 │ levels : 1 │
│ byte flips : 0/16, 0/12, 0/4 │ pending : 0 │
│ arithmetics : 0/896, 0/0, 0/0 │ pend fav : 0 │
│ known ints : 0/88, 0/336, 0/176 │ own finds : 0 │
│ dictionary : 0/0, 0/0, 0/0 │ imported : n/a │
│ havoc : 0/1.54M, 0/0 │ stability : 100.00% │
│ trim : 99.89%/67, 0.00% ├────────────────────────┘
└─────────────────────────────────────────────────────┘ [cpu:200%]

`

@str4d

This comment has been minimized.

Copy link
Contributor

str4d commented Oct 23, 2019

Concept ACK, and the changes look sensible. I'll review more closely when I'm at my laptop and can actually try compiling it.

@zkbot

This comment has been minimized.

Copy link
Collaborator

zkbot commented Oct 23, 2019

☀️ Test successful - pr-try
State: approved= try=True

@str4d
str4d approved these changes Oct 24, 2019
Copy link
Contributor

str4d left a comment

Tested ACK. I tested the zcutil/afl/afl-getbuildrun.sh script, and it successfully fetched AFL and built the fuzzer. It failed during the run phase with:

[*] Checking core_pattern...

[-] Hmm, your system is configured to send core dump notifications to an
    external utility. This will cause issues: there will be an extended delay
    between stumbling upon a crash and having this information relayed to the
    fuzzer via the standard waitpid() API.

but I am assuming that from this point forward everything should be fine (my laptop isn't exactly geared up to fuzzing 😛).

.gitignore Outdated

src/fuzzing/*/output
src/fuzz.cpp
src/fuzz.h

This comment has been minimized.

Copy link
@str4d

str4d Oct 24, 2019

Contributor

It would be good to add these two copied files to make clean, so they aren't around during normal development.

@defuse defuse force-pushed the defuse:fuzzer-packaging branch from 0175475 to bfa8da3 Oct 28, 2019
@defuse

This comment has been minimized.

Copy link
Contributor Author

defuse commented Oct 28, 2019

Added a commit to have make clean delete fuzz.cpp.

@zebambam

This comment has been minimized.

Copy link
Contributor

zebambam commented on bfa8da3 Oct 28, 2019

utACK

@defuse defuse referenced this pull request Oct 28, 2019
@defuse

This comment has been minimized.

Copy link
Contributor Author

defuse commented Oct 29, 2019

There's a bug in this, AFL is trying to pass the data to stdin but the fuzzing stubs are trying to read from a file, that's why there's no new paths.

@defuse

This comment has been minimized.

Copy link
Contributor Author

defuse commented Oct 29, 2019

Fixed the bug in the latest commit.

@zebambam

This comment has been minimized.

Copy link
Contributor

zebambam commented Oct 29, 2019

utACK

@str4d str4d added this to the v2.1.1 milestone Nov 6, 2019
@str4d
str4d approved these changes Nov 6, 2019
Copy link
Contributor

str4d left a comment

Tested ACK

@str4d

This comment has been minimized.

Copy link
Contributor

str4d commented Nov 6, 2019

@zkbot r+

@zkbot

This comment has been minimized.

Copy link
Collaborator

zkbot commented Nov 6, 2019

📌 Commit f189a5f has been approved by str4d

@zkbot

This comment has been minimized.

Copy link
Collaborator

zkbot commented Nov 6, 2019

⌛️ Testing commit f189a5f with merge bc9285b...

zkbot added a commit that referenced this pull request Nov 6, 2019
Add AFL in zcutil (with all-in-one script)

Supersedes #4156 and #4167.

Fuzzing targets and input sets are defined by the contents of directories in `./src/fuzzing/`. Inside the directory, there's a `fuzz.cpp` and `fuzz.h` with a `main()` function that will replace `zcashd`'s actual `main()` as well as an `input` subdirectory containing the inputs, one per file. To just run a fuzzer, you can, for example...

```
make clean # if you've previously build zcashd without AFL instrumentation
./zcutil/afl/afl-getbuildrun.sh DecodeHexTx
```

Alternatively you can...

```
./zcutil/afl/afl-get.sh /tmp/afl   # (or wherever you want to build AFL)
./zcutil/afl/afl-build.sh /tmp/afl DecodeHexTx -j$(nproc)
./zcutil/afl/afl-run.sh /tmp/afl DecodeHexTx
```

Run `make clean` whenever you switch between a normal build and an AFL-instrumented build.
@zkbot

This comment has been minimized.

Copy link
Collaborator

zkbot commented Nov 6, 2019

💔 Test failed - pr-merge

@str4d

This comment has been minimized.

Copy link
Contributor

str4d commented Nov 6, 2019

(What looks like a) transient gtest failure in one of the builders, so I manually cancelled. I'll retry shortly.

@str4d

This comment has been minimized.

Copy link
Contributor

str4d commented Nov 6, 2019

@zkbot retry

zkbot added a commit that referenced this pull request Nov 6, 2019
Add AFL in zcutil (with all-in-one script)

Supersedes #4156 and #4167.

Fuzzing targets and input sets are defined by the contents of directories in `./src/fuzzing/`. Inside the directory, there's a `fuzz.cpp` and `fuzz.h` with a `main()` function that will replace `zcashd`'s actual `main()` as well as an `input` subdirectory containing the inputs, one per file. To just run a fuzzer, you can, for example...

```
make clean # if you've previously build zcashd without AFL instrumentation
./zcutil/afl/afl-getbuildrun.sh DecodeHexTx
```

Alternatively you can...

```
./zcutil/afl/afl-get.sh /tmp/afl   # (or wherever you want to build AFL)
./zcutil/afl/afl-build.sh /tmp/afl DecodeHexTx -j$(nproc)
./zcutil/afl/afl-run.sh /tmp/afl DecodeHexTx
```

Run `make clean` whenever you switch between a normal build and an AFL-instrumented build.
@zkbot

This comment has been minimized.

Copy link
Collaborator

zkbot commented Nov 6, 2019

⌛️ Testing commit f189a5f with merge 5ec69e8...

@zkbot

This comment has been minimized.

Copy link
Collaborator

zkbot commented Nov 6, 2019

☀️ Test successful - pr-merge
Approved by: str4d
Pushing 5ec69e8 to master...

@zkbot zkbot merged commit f189a5f into zcash:master Nov 6, 2019
1 check passed
1 check passed
homu Test successful
Details
@str4d str4d removed the review needed label Nov 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.