afl-fuzz reports 16.00% stability for a simple constant object expression

[vicuna]

Original bug ID: 7612
Reporter: talex
Assigned to: @stedolan
Status: resolved (set by @xavierleroy on 2017-10-11T09:21:16Z)
Resolution: fixed
Priority: normal
Severity: minor
Version: 4.05.0
Target version: 4.06.0 +dev/beta1/beta2/rc1
Category: back end (clambda to assembly)
Monitored by: @gasche @yallop @hcarty @yakobowski

Bug description

The program:

let () = AflPersistent.run (fun () -> ignore (object end))

should behave the same every time it is run. Instead, AFL reports: "stability : 16.00%"

The AFL docs say:

If a program always behaves the same for the same input data, it will earn a score of 100%.

I believe this means that AFL thinks it is finding new paths when it isn't, making the fuzzing inefficient.

Stephen Dolan said (https://ocamllabs.slack.com/archives/C0QBB5457/p1497434875205918):

Hmm. Looking at the Cmm, that program does indeed seem to have some state (!)
There's a lazily initialised table per class, and a branch to see whether it has already been initialised.
[...]
The bug is indeed the lazy initialisation of classes being picked up by afl-fuzz, and the fix is to ensure that this code is not instrumented. However, class initialisation is pretty hairy, with a bunch of different paths depending on whether the class closes over values, etc.

Steps to reproduce

docker run --rm -it ocaml/opam:debian-9_ocaml-4.05.0
opam sw 4.05.0+afl
eval opam config env
opam install afl-persistent
ocamlfind ocamlopt -package afl-persistent -linkpkg test.ml
sudo apt-get install afl --no-install-recommends
mkdir in; echo > in/empty
afl-fuzz -i in -o out ./a.out

AFL reports "stability : 16.00%"

[vicuna]

Comment author: @stedolan

Thanks for the report. Github PR here: #1345

[vicuna]

Comment author: @xavierleroy

GPR merged in 4.06