Distance calculation failure

[blbi]

Hi,

I tried to run AFLGo with openjpeg according to README.
I succeeded to instrument but when I run gen_distance_fast.py script, the error messages are printed repeatedly as follows.

~/aflgo$ $AFLGO/scripts/gen_distance_fast.py $SUBJECT/build/bin $TMP_DIR opj_dump
(0) Constructing CG for /home/user/aflgo/openjpeg-2.1.1/build/bin/opj_dump.0.0.preopt.bc..
(1) Computing distance for callgraph
(1) Computing distance for control-flow graphs (this might take a while)
cfg distance calculator failed while calculating distance for /home/user/aflgo/temp/BBtargets.txt.
cfg distance calculator failed while calculating distance for /home/user/aflgo/temp/BBtargets.txt.
cfg distance calculator failed while calculating distance for /home/user/aflgo/temp/BBtargets.txt.
...

~/aflgo/temp$ cat step0.log 
Writing 'callgraph.dot'...

Can you tell me what's the problem?

Additionally, I also tried to fuzz gif2png. However the configuration step failed with the following error.

$ ./configure
...
checking whether make sets $(MAKE)... yes
checking for gcc... /home/user/aflgo/gif2png-2.5.8/afl-clang-fast
checking whether the C compiler works... no
configure: error: in `/home/user/aflgo/gif2png-2.5.8':
configure: error: C compiler cannot create executables
See `config.log' for more details

I followed all 1~6 step on README file, Is there something I missed?

[mboehme]

Does it generate the distance.cfg.txt file? If not, can you try gen_distance.sh instead of gen_distance_fast.py?
//cc @Liblor

[blbi]

Thank you! It works!
However I met another error..

Working in CG mode..
Loading targets..
No targets available
-- Problem in Step 2 of generating !
-- You can resume by executing:
$ ./scripts/genDistance.sh /home/user/aflgo/openjpeg-2.1.1/build/bin /home/user/aflgo/temp opj_dump /home/user/aflgo/temp /home/user/aflgo/temp

When I run the command, It returns the same error.

I wrote the BBtargets.txt file manually instead of executing showlinenum.awk because I have no diff file.
Is it related to the error?? My BBtargets.txt file looks like this.

circuit@circuit:~/aflgo/temp$ cat BBtargets.txt 
src/lib/openjp2/cio.c:87

[Liblor]

I think the issue is the same. Probably, gen_distance_fast.py doesn't catch the error nicely (if this is the case I'll try to address that to get more helpful bugreports). I'll see if I can reproduce the problem

[blbi]

Thanks. The targeted openjpeg is version 2.1.1.
Is there any way to proceed the fuzz test right away or should I wait for the problem to be fixed?

[Liblor]

The issue is that you probably don't compile opj_dump statically, as this isn't the default! In that case, gen_distance_fast.py / genDistance.sh can't find the targeted function in the callgraph, because they are in some shared library (that's why genDistance.sh complains that no targets are available).

Compile opj_dump as follows and it should work:

cmake -DBUILD_SHARED_LIBS=off ..
make opj_dump

(also remove the previous build directory and temp dir, to be sure to have no conflicting files)

Let me know, if you got it running.

[blbi]

I succeeded with the cmake option! However my distance.cfg.txt file is empty..😢

~/aflgo/temp$ cat distance.cfg.txt 
~/aflgo/temp$ cat step0.log 
Writing 'callgraph.dot'...

Both scripts genDistance.sh and gen_distance_fast.py show the same result.
How can i fix it? This is my step2.log step2.log

[Liblor]

Hi,
unfortunately I can't reproduce your issue. Maybe you had an old state file in the $TMP_DIR folder?

Here are the steps I performed:

# clear tmp dir and set the target
rm -r $TMP_DIR/*
echo "cio.c:87" > $TMP_DIR/BBtargets.txt

# Set aflgo-instrumenter
export CC=$AFLGO/afl-clang-fast
export CXX=$AFLGO/afl-clang-fast++

# Set aflgo-instrumentation flags
export ADDITIONAL="-targets=$TMP_DIR/BBtargets.txt -outdir=$TMP_DIR -flto -fuse-ld=gold -Wl,-plugin-opt=save-temps"
export CFLAGS="$ADDITIONAL"
export CXXFLAGS="$ADDITIONAL"

# Build opj_dump
cd $SUBJECT
mkdir build && cd build
cmake -DBUILD_SHARED_LIBS=off ..
make opj_dump

# Clean up
cat $TMP_DIR/BBnames.txt | rev | cut -d: -f2- | rev | sort | uniq > $TMP_DIR/BBnames2.txt && mv $TMP_DIR/BBnames2.txt $TMP_DIR/BBnames.txt
cat $TMP_DIR/BBcalls.txt | sort | uniq > $TMP_DIR/BBcalls2.txt && mv $TMP_DIR/BBcalls2.txt $TMP_DIR/BBcalls.txt

# Distance computation
$AFLGO/scripts/gen_distance_fast.py $SUBJECT/build/bin $TMP_DIR opj_dump

and the distance file isn't empty:

cat $TMP_DIR/distance.cfg.txt | sort | head
cio.c:78,6
cio.c:83,4
cio.c:85,3
cio.c:86,2
cio.c:87,1
j2k.c:1369,13
j2k.c:1376,12
j2k.c:1377,11
j2k.c:1385,13
j2k.c:1392,12

Edit: if it still doesn't work, could you also share your BBcalls.txt, Ftargets.txt, Fnames.txt and callgraph.distance.txt.

[blbi]

Thanks for your help!! I succeeded to run the fuzzer!😊😊