AFL++ communication error - Pipe read returns 0

[v-p-b]

I ran into an annoying non-deterministic bug when using KF/x with AFL++. I traced back the issue to the fact that sometimes the pipe read by AFL++ here returns 0 (but not -1, that would indicate an error):

https://github.com/AFLplusplus/AFLplusplus/blob/stable/src/afl-forkserver.c#L172

Retrying here instead of returning doesn't improve the situation, because all subsequent reads also return 0.

This results in read_s32_timed returning 0, that ultimately ends up killing the setup phase with the well known "Unable to communicate with fork server" message:

https://github.com/AFLplusplus/AFLplusplus/blob/stable/src/afl-forkserver.c#L191

https://github.com/AFLplusplus/AFLplusplus/blob/stable/src/afl-forkserver.c#L1299

I'm not entirely sure which write this should be on KF/x's side, but I could confirm, that afl_wait successfully writes the 4 expected bytes to the pipe. As far as I understand, the next communication should be by afl_report, so I suspect that something must go wrong between these two lines:

https://github.com/intel/kernel-fuzzer-for-xen-project/blob/master/src/main.c#L93

https://github.com/intel/kernel-fuzzer-for-xen-project/blob/master/src/main.c#L123

Interestingly, it feels that running KF/x improves the success rate, but I don't have the data to support this.

Tagging in @domenukk, hoping he can shed some light on the expected behavior at AFL++ side.

[v-p-b]

It seems that KF/x doesn't complete the PT decoding before AFL++ quits:

https://github.com/intel/kernel-fuzzer-for-xen-project/blob/master/src/ptcov.c#L166

I thought that this must be a timing issue, so I tried moving afl_report before pt_decode. Interestingly in this case AFL++ dies earlier, again by reading 0 length from the same pipe:

https://github.com/AFLplusplus/AFLplusplus/blob/bd0a23de73011a390714b9f3836a46443054fdd5/src/afl-forkserver.c#L1109

A modified the code further with military grade printf's to see whether afl_wait is called at all. The answer turns out to be: sometimes o.O

Scratch that, afl_wait runs as expected, New Years hangover reduced my vision...

[tklengyel]

Moving pt_decode before afl_report is not a good idea - the coverage map only gets updated after pt_decode is done, so if you signal to afl you are done then afl will take the empty/half-baked coverage map.

[v-p-b]

Yeah I know, I just wanted to test if it's the decoding delay thas is causing this.

[domenukk]

I don't think I can help you much here, the read should never return 0 and as far as I can see, this should only happen when the pipe gets closed /EOF - so not timing related.
If you retry the write a few times, does it work for obscure reasons, or will the read it always return 0?

[tklengyel]

Would be good to understand where the communication gets stuck. If afl_setup worked and you get data from afl through the read in afl_wait then the pipe is working. Does afl get the PID back that's sent in afl_wait? Then afl_report will write 4 bytes (either 0 or SIGABRT) to signal its done with the data and the coverage map is ready. So is afl returning from their read before that happens? Is there some timeout in afl that bails too early?

[tklengyel]

Also, btw, usually when I see "Unable to communicate with fork server" that just means KF/x exited early. Always verify that kfx runs fine first standalone and that it doesn't report a crash. Just replace @@ with the path to one of your seeds and keep all options the same.

[v-p-b]

It is a segfault in libxdc :P As I understand writes don't block the sender, so KF/x can send whatever, then by the time AFL++ tries to read the data the fork either crashed or not. It's interesting why this happens non-deterministically (sometimes libxdc can decode all traces, but during most startups it can't).

Closing this for now, thanks for your help!