Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

afl-fuzz reports 16.00% stability for a simple constant object expression #7612

Closed
vicuna opened this issue Aug 25, 2017 · 2 comments
Closed

afl-fuzz reports 16.00% stability for a simple constant object expression #7612

vicuna opened this issue Aug 25, 2017 · 2 comments
Assignees
Labels
Milestone

Comments

@vicuna
Copy link

@vicuna vicuna commented Aug 25, 2017

Original bug ID: 7612
Reporter: talex
Assigned to: @stedolan
Status: resolved (set by @xavierleroy on 2017-10-11T09:21:16Z)
Resolution: fixed
Priority: normal
Severity: minor
Version: 4.05.0
Target version: 4.06.0 +dev/beta1/beta2/rc1
Category: back end (clambda to assembly)
Monitored by: @gasche @yallop @hcarty @yakobowski

Bug description

The program:

let () = AflPersistent.run (fun () -> ignore (object end))

should behave the same every time it is run. Instead, AFL reports: "stability : 16.00%"

The AFL docs say:

If a program always behaves the same for the same input data, it will earn a score of 100%.

I believe this means that AFL thinks it is finding new paths when it isn't, making the fuzzing inefficient.

Stephen Dolan said (https://ocamllabs.slack.com/archives/C0QBB5457/p1497434875205918):

Hmm. Looking at the Cmm, that program does indeed seem to have some state (!)
There's a lazily initialised table per class, and a branch to see whether it has already been initialised.
[...]
The bug is indeed the lazy initialisation of classes being picked up by afl-fuzz, and the fix is to ensure that this code is not instrumented. However, class initialisation is pretty hairy, with a bunch of different paths depending on whether the class closes over values, etc.

Steps to reproduce

docker run --rm -it ocaml/opam:debian-9_ocaml-4.05.0
opam sw 4.05.0+afl
eval opam config env
opam install afl-persistent
ocamlfind ocamlopt -package afl-persistent -linkpkg test.ml
sudo apt-get install afl --no-install-recommends
mkdir in; echo > in/empty
afl-fuzz -i in -o out ./a.out

AFL reports "stability : 16.00%"

@vicuna
Copy link
Author

@vicuna vicuna commented Sep 15, 2017

Comment author: @stedolan

Thanks for the report. Github PR here: #1345

@vicuna
Copy link
Author

@vicuna vicuna commented Oct 11, 2017

Comment author: @xavierleroy

GPR merged in 4.06

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.