Weekly Status

11/9/18

Fixed several bugs in the pass and runtime library, including how floating point numbers are handled. This required adding a new runtime function to handle floating types, because casting a double into a void* changed the bit layout and I was losing information. The python script (id-parser.py) that reads in the JSON output from the test runs is capable of generating C++ classes for functions that only use primitive types. This was an attempt to get some initial results. Using the libc-test repository which musl uses, I can compile and run all the tests, but I am focusing on the src/math/pow.c test for initial results.

The pow function is called over 11,000 times in the course of the test, so I had id-parser.py cap the tests at 10 for now. As of this writing, the pow identifier does not find pow in the statically linked pow unit test, and I am investigating why.

On the evaluation front, I found in GitHub the source of many different malware, including Mirai. Mirai looks to be ideal for a case study; it is statically linked, runs on embedded devices, well-known, and recent. Additionally, Mathias contacted a researcher who is compiling a large corpus of malware, and I have been granted access to the repository. I have not looked through it yet, but I am sure that there will be other malware case studies we can include.

Something to address in the paper from my presentation at the weekly meeting: Function call sequences that must happen in order (e.g. open-write-close) are not supported yet.

4/9/18

Created LLVM pass and runtime library to capture the input and return values for function calls. The pass is simple right now. It simply finds all CallInst, inserts a new function call to the runtime library lof_precall, and 0 or more calls to another runtime function lof_record_arg. Finally, a function call to lof_postcall is added immediately after the CallInst to record the return value. All function calls and function arguments are stored in a simple stack implementation using a linked list implementation I wrote. In other words, each function is represented as a stack of arguments, and each function argument stack is pushed onto a global stack. All data is written to a file called lof-output.json and is obviously in an easily parseable JSON format.

The library functions have the following function signatures:

lof_precall(void* funcaddr)

lof_record_arg(Data value, TypeID typeId, size_t size)

lof_postcall(Data value, TypeID typeId, size_t size)

Data is a union of various primitive types, and TypeID is an enum provided by LLVM that indicates if the value is a float, int, void*, etc.

Questions for discussion:

1. How to handle void*? Currently I am just reading the first 4 bytes.
2. How to handle structs? I'd like to punt this down the road for now until primitive types are handled.

28/8/18

Evaluated memcpy, memmove, strcpy, strncpy against busybox and nginx. 100% accuracy. Go me. This shows that we can identify functions post-execution by controlling their inputs, at least for some functions. Those functions that require some initial state probably won't work, but maybe we can find a lot of other functions with more function identifiers. We need an automated way to generate them.

Evaluated approaches for automating the generation of function identifiers. Problems I am trying to solve:

1. A priori, I don't know how a valid function execution changes program state
2. I don't know how to get valid input to a function to measure program state change

Unit tests seem to solve these problems, because the programmer writing the test would know best good inputs to a function (theoretically), and any relevant state change would be tested for in the unit test.

Two approaches were discussed:

1. Modify the various existing testing frameworks libraries currently use to memory map the binary into the address space first, and then replace function unit test is testing with a function in the binary. If test passes with binary function, then binary function == unit test function.
2. Instrument testing framework to capture the function parameters and return values all unit tests use when executing a function we want to describe. We save all input and return values (including struct subfields and following pointers recursively), and automatically create a fbf::FunctionIdentifier subclass containing the input. We then use the currently existing framework to compare functions in the BUA. A successful identification would be if the return value of the BUA function (or the underlying data if a pointer is returned) is the same as the unit test function return value, and any change in the input data pre- and post-execution should be the same in the unit test function and the binary function.