0.1 Introduction
0.2 What Is Fuzzing?
0.3 Why Fuzz?
0.4 Terms Used Throughout the Module
1.6 How to Download VirtualBox
1.8 How to Download SourceTrail
2.1 How to Create an AFL++ Docker Container
2.2 How to Create Target Docker Container
2.3 How to Run AFL++ on Exercise 1
2.5 Challenge: Fuzz Exercise 2
4.2 How to Create a Slice for Exercise 3
4.4 Solution: How to Fuzz Exercise 3
4.5 Conclusion
Estimated Time: 2 minutes
This repository contains a README which explains the concept of fuzzing and why one would want to fuzz. The README goes on to teach anyone familiar with Computer Science how they can start fuzzing their own projects using the three examples ("exercise1", etc.) in the repository. We have designed this module to be completed on a Mac - Windows users can still complete the module but will need to use the virtual machines.
Quick note: Fuzzing is an intermediate level topic. Although we include basic instructions, like how to install the tools needed to fuzz a target (installing Docker, using Sourcetrail, etc.), to be successful while fuzzing requires some experience. Knowing how to build different projects is essential to fuzz a target, and that is not something that is covered in this module.
From Wikipedia, fuzzing is automated input testing for a codebase or executable. While fuzzing, invalid or unexpected (sometimes random) data is provided to an executable in hopes of finding some undefined behavior, or vulnerability. For more information about what fuzzing is and how it can be beneficial, you can look at Google's Fuzzing repo.
Fuzzing is used to find vulnerabilities in software. You can fuzz your own code or open-source projects to find and patch edge cases of bugs or for other academic purposes. Again, a more detailed description of why one would choose to fuzz can be found in Google's Fuzzing repo.
- Container: An isolated environment on a computer that allows code to run freely without interacting with the rest of your computer or other hardware. You can think of this like a Virtual Machine which only uses a command line interface.
- Wrapper: A program that allows a fuzzer to interact with the slice being fuzzed. It is the code which glues together the fuzzer and the target code.
- Mutations: New inputs which the fuzzer generates by modifying previous inputs with the goal of reaching more areas of the target code.
- Slice: A section of a codebase that is isolated from the rest of it. Isolation of slices makes finding vulnerabilities and bugs easier and more efficient.
Estimated Time: 15 minutes
We use a number of tools throughout this learning module (with download links as they come up), so we will outline them here.
- Docker: We will create a container using Docker to create an isolated environment for fuzzing the target code.
- AFL++: AFL++ is the fuzzer we will use in this learning module. AFL++ will attach to the target code using a wrapper, and it will handle all of the inputs and mutations. When running code with AFL++, there is a nice output that updates automatically to show how long the code has been running and some information about the number of crashes/hangs that have been detected.
- Kali Linux: A virtual machine that we used during our testing process for Windows users. Any other virtual machine will work for our purposes. This is only necessary for windows users.
- Sourcetrail: A tool that can be used to get familiar with a codebase. There are two main windows within Sourcetrail - the code being looked at and a graphical interpretation of it. By clicking on functions within the code, you can see where those functions are called in other spots throughout the codebase. We found Sourcetrail to be quite useful for exploring large projects because we could more easily determine how various parts of the codebases were connected.
- To clone the target code for this module, simply use either the https or ssh clone process for this repository.
- Clone the target code into wherever you want on your os.
- To download Docker, go to this link on the Docker website and choose the correct operating system and chip. The website should automatically propose the correct software for your system, but download links for other versions will also be available on the page.
- After the download is complete, run through the steps in the setup and open the desktop app on your computer.
- Disclaimer; this may take up a lot of space on your computer
- The next step is to go to the AFLplusplus Github Repo and hit the green "Code" button, and copy the HTTPS link to clone the repository.
- In your terminal - Terminal for Mac and PowerShell for Windows - write
git clone https://github.com/AFLplusplus/AFLplusplus
to clone AFLpluslplus onto your computer.
- For our purposes, we chose Kali Linux as our virtual machine, but any other virtual machine should suffice. We used a VM to run Sourcetrail because, while running Sourcetrail on Windows, Sourcetrail was routinely unable to locate the correct path for the code.
- To do this properly on a Windows machine, simply download your VM of choice - we used VirtualBox or VMWare with a Kali Linux image, all which can be found on the Kali website with the VM download on the page just below.
- To download the correct Kali image, make sure you are matching your computer architecture with the image you download. Under the Bare Metal header, you can choose 64 or 32-bit images. The recommended image to download is the "Installer" image, which is the first one you see and is 2.8GB in size. You can read more about which image to download here.
- After downloading the correct image, scroll down on the page under the "Virtual Machines" header, and choose either the VMWare or the VirtualBox download (make sure to choose either 64 or 32 bit here as well).
- After this is done - run through the setup of the VM and open it up, and run the image. After you run, you should get a login screen - the default credentials to enter are a username and password of
kali
. Then, follow the next set of steps to download Sourcetrail in your VM.
Through our experience on this project, we would recommend using VirtualBox rather than VMware.
To open your image of Kali Linux on VirtualBox, all you have to do is follow the steps on this website.
The image of Kali Linux described earlier in the module is one of the "ready-to-use" virtual images, which means all of the correct
settings should import as well when you import the image to VirtualBox. Make sure you are using the file with the .ova
extension.
After you import the image, you can hit "Import" and then after a few minutes you should see the image in the sidebar of the
VirtualBox application. Select it, and then press the green "Start" arrow to start up the VM.
Once you are logged in, you should see a Desktop with the Kali logo in the background. In the top left corner, there is an
icon resembling a terminal. Open this, and follow the download steps for Sourcetrail as well as doing git clone
for the chosen
target(s). After the target(s) are cloned in the VM, you can build them and move forward with the steps outlined in the Indexing
in Sourcetrail section
VMware is a free-to-use, at the personal level, virtual machine software for many potential operating systems. The VMware code can be downloaded for free here. Once VMWare is installed, it may be used with any potential linux OS which provides a VMWare configuration. In the context of this project, we decided to use Kali, however, any linux version would work insofar as there exists a VMware file for it. Once the virtual machine software has finished installing, one can simply open the Kali, or any other, VM file within the program to run it.
When going through the process of building our targets on Windows, we had compatibility issues with Sourcetrail that prevented the files in our targets from being indexed properly by Sourcetrail. We remedied the issue by using a virtual machine. As mentioned previously, we used Kali Linux with VMWare/VirtualBox, however any VM should work as a solution. If you are not familiar with VMs, we will walk through using Kali Linux with VMWare and VirtualBox in this guide.
- To download Sourcetrail, which we can use to walk through and analyze code, go to the Sourcetrail github and download the release that is compatible with your operating system. Then run through the setup on your computer and open up the app.
Estimated Time: 45 Minutes
This is completed in the main OS terminal:
- First, open the Docker desktop app on your computer, otherwise the next step will throw an error.
- In your terminal, run the following command
docker pull aflplusplus/aflplusplus
- After this command has run, you should be able to open the Docker app and see the AFL++ container under the Container/Apps tab. Next to the container, it should say
aflplusplus/aflplusplus
in blue.- If you do not see a new Container, go to the Images tab (also on the left) and find the most recent aflplusplus/aflplusplus image. Hover over it, click run on the far right, and then create a new container. Once you have created the container, return to the Container / Apps tab.
- Hover over that container, and click the play button to start the container (if there is a stop button instead of a play button, then the container is already running).
- Click on the CLI button, which has ">_" in a circle. This will open a new command line window that is already within your AFL++ docker container.
The following step is completed in the Docker CLI terminal:
- You should be in the
AFLplusplus
directory (you can confirm this withpwd
). Once there,make
the AFL++ executables. This will take several minutes. Once it is donemake
ing, you can close the CLI.
This is completed in the main OS terminal:
- Start the docker container for AFL++ using the Docker app on your desktop
- Within the command line, type:
docker ps
- This will output the running docker containers on your machine at the moment. Copy the
CONTAINER ID
for the running AFL++ container
- This will output the running docker containers on your machine at the moment. Copy the
- Next, with your copied container ID, type:
docker commit [container id]
- This will output a SHA256 hash of the committed container. Copy the first 7-10 characters of the commit hash.
- To start the AFL++ container with the target code, first navigate to the top directory of your clone of this github repository. Then type:
docker run --rm -it -v $(pwd):/[name of the directory you are adding to the container] [the commit hash that you copied in the previous step]
- For example, the complete command looks something like the following for one of the authors of this repository:
docker run --rm -it -v "/Users/george/Desktop/CS Capstone/capstone/medium":"/Users/george/Desktop/CS Capstone/capstone/medium" f9a71912b4
- If you don't want to navigate to the directory of the code you want to fuzz, you can replace $(pwd) with the full path to the directory you want to fuzz, starting at your home directory
- If the terminal prints the following error: _docker:
invalid reference format: repository name must be lowercase
, add "quotation marks" around the$(pwd):/\[directory\]
- This error arises when directory names contain space characters
- If you are on a Windows machine (and your directory names do not have space characters), you may have to use the following syntax when running this command, since your system recognizes a different filepath format:
docker run --rm -it -v C:\Users\george\Desktop\Capstone\capstone\medium:/Users/george/Desktop/Capstone/capstone/medium f9a71912b4
- For example, the complete command looks something like the following for one of the authors of this repository:
This is completed in the target container Docker CLI:
- After starting the docker container, you should be within the
/AFLplusplus
directory. Navigate to the folder of the code you want to fuzz. You will need to go up a directory from/AFLplusplus
(cd ..
), and thencd
into [name of the directory you are adding to the container] from step 4 of "Creating a Container to Fuzz Code". - Create a build directory (standard practice to name it build)
mkdir build
- Change directory into build
cd build
- Add AFL++ tooling to the compiler for your executable:
CC=/AFLplusplus/afl-clang-fast CXX=/AFLplusplus/afl-clang-fast++ cmake ..
- Informational Note:
afl-clang-fast/++
is just one example of compilers you can use with AFL++ - different compilers have different advantages. You can use any of the compilers within the/AFLplusplus
directory, and theCXX
variable name is always the same as theCC
variable, with++
appended to the end. You can read more about the different compilers and their advantages within the AFL++ docs.
- Make the files in build
make
- If you do not already have a seed directory, follow this process to create and populate one using the
dd
command. If you do have such a directory, skip to step 7.cd ..
mkdir seeds
for i in {0..4}; do dd if=/dev/urandom of=seed_$i bs=64 count=10; done
cd ..
cd build
You can read more about the dd
command at this Stack Exchange post.
- Once you have a seed directory, enter the following command:
/AFLplusplus/afl-fuzz -i [full path to your seeds directory] -o out -m none -d -- [full path to the executable]
Congratulations, you are now running AFL++ on your target code! There should be a UI in terminal which shows you various statistics about the fuzzing process - look for the number of crashes detected.
- Once there is at least 1 crash shown in the UI of AFL++, hit
Ctrl + C
to exit AFL++. You can find the inputs that caused the program to crash by traversing to theout/default/crashes
directory. - You can use a bugger, such as
gdb
orllvm
, to figure out what part of the input actually caused the program to crash. There are also other directories inout/default
that show you some information gathered during the fuzzing process - feel free to explore them.
Now that you have run AFL++ on Exercise 1, we would like you to try to run AFL++ on Exercise 2 without instruction from the module. If you get stuck, instructions on how to run AFL++ on Exercise 2 will be in the exercise2-instructions
directory.
- After the project is built, open Sourcetrail and click New Project. Name the project something relevant and in the "Sourcetrail Project Location" text box, click the three dots on the right and navigate to the directory where your project lives. After the path is listed in the text box, click "Add Source Group" at the bottom of the window.
- In the new pop up, select the "CDB" option if it is a C or C++ project. A new window will pop up. Click the three dots in the "Compilation Database" textbox, and navigate to the "compile_commands.json" file within the build folder you created in the target. This will allow the files to be indexed in Sourcetrail.
- On a Windows machine, this process is the same, but must be done within the virtual machine.
- Once everything is indexed in Sourcetrail, open the project in another program for viewing code (like VS code, sublime, etc.) so you can see the files listed out and can navigate the file tree. This will allow you to glance over code and find things that might be interesting to investigate further using Sourcetrail (Sourcetrail has a search bar, so you can find any function or chunk of code very quickly).
- The best way to find potential entry points in your targets is to look for files dealing with inputs from an outside source
- This could look like sensors, GPS systems, user inputs, communication modules, etc.
- We had success looking at "MAVLink"-related functions and files.
- After finding a file that could be of interest, navigate to it in Sourcetrail by looking it up. You will see the functions in the file and the inputs to those functions. You can click through those things to learn more about them.
- In general, its good practice to bookmark things in Soucetrail that look interesting so you can come back to them later.
When creating a slice, you want to narrow down the code that will be running as much as possible. For example, if you were looking to fuzz a drone system, you may just want to fuzz the communication module or the gps module to see if there are flaws in the logic that the fuzzer you are using can detect. Each fuzzing project is different, based on the layout of the codebase and the build system that is used to compile the executable. Looking at the documentation for the codebase can be very helpful to learn how all the different modules interact with each other, which can then help you figure out where to start looking at the code in Sourcetrail.
For the slice, you can comment out some of the code in the target to ignore features that you are not interested in fuzzing. The essential pieces to run the code should be the only code left uncommented. For example, if there is a section that sleeps
a certain period of time to wait for another module to be up and running, but you are not interested in that module of the target, then comment it out.
The goal of making a slice is to streamline the running process of the program so AFL++ can more efficiently run it internally.
When using AFL++, something that needs to be included in the wrapper are the following lines:
#ifdef __AFL_HAVE_MANUAL_CONTROL
__AFL_INIT();
#endif
These lines need to be included in your main
before you read inputs from STDIN
. They allow AFL++ to have control over the inputs by feeding its inputs through STDIN
.
In the wrapper, you should include all files that are necessary for the module you are fuzzing to run. Excess files are unnecessary. For example, let's fuzz exercise3
that we have created for this module...
The first step in understanding a target is figuring out how all the functions interact. Let's look at this in Sourcetrail (if you haven't downloaded Sourcetrail, follow the instructions here).
- In a terminal window, navigate to the
Fuzzing-Module/exercise3
directory.- Note: this terminal window should not be one that is running a Docker container - we will be making a new container for this project. You can close any terminal windows with a Docker container currently running.
- Create a
build
directorymkdir build
- Go into that
build
directorycd build
- Type the following line into the command line:
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..
- This line uses CMake to create a JSON file that will tell Sourcetrail how the files interact with each other.
- Open up Sourcetrail. On the popup that opens with the app, click "New Project"
- Create a name for the project. It doesn't matter what the name of it is, but a good idea of a name would be the name of the code you are analyzing (e.g., "exercise3")
- Select the location this Sourcetrail project is going to "live". This will be the top directory of the code you are running, thus for our purposes it is
Fuzzing-Module/exercise3
. Select this folder wherever you cloned this directory to by clicking on the circle with the three dots in it on the right side of the "Sourcetrail Project Location" bar. - Click the "Add Source Group" button at the bottom of the window.
- We want to add a source group for "C/C++ from Compilation Database" - click that option, and then "Next"
- In the "Compilation Database" entry box, click the circle with the three dots in it. This should open up the
exercise3
folder. The file we are looking for iscompile_commands.json
within thebuild
folder. Select it, and then click "Open". - Click "Create" at the bottom of the popup. This will close the current popup, and cause another popup to appear that is blue. The last step is to index all the files - click "Start"
Once the indexing is done, you will be able to look at a graphical representation of the code! A good place to start is by looking at the files that exist in this project.
Use Sourcetrail to analyze the Sys_Status
class. We have included a sample slice and wrapper for the specs class in the file specs-slice.cpp
.
With a basic understanding of how creating a slice works, we can re-review how to fuzz Exercise 3, which is slightly different than the other pieces of code that we worked with in this module. There are many different slices that can be created using the code in Exercise 3, so you have to tailor the commands to whatever slice you are using. Aside from this, the majority of the steps will be the same. We can walk through these steps more briefly (these should be done in your regular computer terminal):
- Make sure that AFLplusplus and docker are running by opening up the docker application and clicking the start button next to the AFLplusplus docker container.
- Run the following commands:
-
docker ps
-docker commit [id of afl++ container]
- go to the /Fuzzing-Module/exercise3 directory and run the following command:
-
docker run –rm -it -v $(pwd):/exercise3 [hash from commit two steps before]
. If this syntax does not work, back to the section describing how to create a target docker container to refresh yourself on other syntax you may need to use. - Following this, you should be in the AFLplusplus terminal. Within this terminal, do the following:
-
make
the AFLplusplus executables if needed. - Navigate to the code you added to the AFLplusplus container by doingcd ..
and thencd [added directory]
- Within this directory, create a
seeds
directory,cd
into it and create at least 5 seeds, the same way that we did in step 6 of "How to Run AFL++ on Exercise 1". - Once you have your 5 seeds,
cd ..
out of the seeds directory and create thebuild
directory. Go into this directory and run the following commands: -CC=/AFLplusplus/afl-clang-fast CXX=/AFLplusplus/afl-clang-fast++ cmake ..
-make
-/AFLplusplus/afl-fuzz -i ../seeds/ -o out -m none -d -- ./specs-slice
- At this point, AFLplusplus should be running and you should be seeing crashes happen. An important thing to notice is that the executable we are using is
./specs-slice
, which is a small section of our code (the slice). When writing your own slice, you should change this parameter to be the name of whatever file you created the slice in. - From running AFLplusplus, there should be ~4 unique crashes. From looking at the code, it is clear that this comes from the
choose_color()
function, which will crash if it takes in any non-numerical inputs. These are the kinds of this to look for in other areas of the program when creating another slice.
Based on what you have learned about creating a slice from these past sections, take a look at the other files in Exercise 3, and try to create another slice from other areas of the code which seem vulnerable using your new knowledge of fuzzing.
Through this module, you have learned the basics of fuzzing. We walked through a basic introduction to fuzzing, and important software to download. We also discussed using AFL++ and Docker, and some example exercises to fuzz. One of the most important parts of this module is understanding the errors that you ran into while going through our tutorial. Although many potential errors are accounted for in our explanations, knowing where these exercises come from is half the battle of understanding how to fuzz because it will provide a foundation for fuzzing much more complex programs, such as flight controller software. If you want to spend more time learning about this, go back through the errors you ran into and things that were missed while walking through the practice exercises.
Fuzzing is used consistently in industries like aviation, finance, healthcare, energy, automotive, and more, as security regulations increase across all industries. If you want to learn more about the practical applications of fuzzing in industry, check out these articles:
- Making fuzzing smarter: This paper discusses fuzzing and how it can be made more efficient, smarter, and more state-of-the-art.
- Ethereum Network Vulnerability: how advanced fuzz testing discovered a serious DOS vulnerability in the Ethereum network.
- Different types of fuzzing: This article discussed a few different types of fuzzing - blackbox, whitebox, and grammar-based - and the effectiveness of each.
- In general...a list of bugs you can find by fuzzing