tests: Avoid accumulating allocated memory in a global state if LogPrintf is called when fuzzing

[practicalswift]

Avoid accumulating allocated memory in a global state if LogPrintf is called when fuzzing.

The accumulation takes place via the m_msgs_before_open buffering.

Resolved by enabling logging and writing log messages to standard output.

This has the added benefit of making the fuzzing operator aware of any log printing caused by fuzzing which is likely to be an anomaly in itself (in the general case).

The only fuzzing harness in master that I've seen calling LogPrintf is messageheader_deserialize via the call to CMessageHeader::IsValid() which somewhat surprisingly does a LogPrintf in the case of nMessageSize > MAX_SIZE :)

Before:

$ src/test/fuzz/messageheader_deserialize corpus/
…
INFO: libFuzzer disabled leak detection after every mutation.
      Most likely the target function accumulates allocated
      memory in a global state w/o actually leaking it.
      You may try running this binary with -trace_malloc=[12]
      to get a trace of mallocs and frees.
      If LeakSanitizer is enabled in this process it will still
      run on the process shutdown.
…

After:

$ src/test/fuzz/messageheader_deserialize corpus/
…
2020-01-07T23:20:34Z CMessageHeader::IsValid(): (@, 4278190080 bytes) nMessageSize > MAX_SIZE
…

How to test this PR

$ make distclean
$ ./autogen.sh
$ CC=clang CXX=clang++ ./configure --enable-fuzz \
      --with-sanitizers=address,fuzzer,undefined
$ make
$ src/test/fuzz/messageheader_deserialize
…

[Crypt-iQ]

Concept ACK.

I'm not too familiar with libFuzzer so I can't speak to that, but won't this buildup only occur if AFL is in persistent mode since then there's a loop rather than a new binary being executed?

[fanquake]

Travis failure:

Traceback (most recent call last):
  File "test/fuzz/test_runner.py", line 175, in <module>
    main()
  File "test/fuzz/test_runner.py", line 115, in main
    universal_newlines=True,
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/addr_info_deserialize', '-help=1']' 
died with <Signals.SIGABRT: 6>.

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

[practicalswift]

@Crypt-iQ Yes, the build-up will only occur if in-process fuzzing is used: libFuzzer is always in-process and AFL only when in persistent mode.

[maflcko]

Concept ACK. Approach NACK.

[practicalswift]

@MarcoFalke Approach ACK now with the updated version? :)

[practicalswift]

@MarcoFalke How can we proceed with this one? Would be really nice to not have deal with this memory leak when fuzzing the deserialisation code :)

[practicalswift]

Closing due to lack of interest :)