This is Quest, a tool that generates C code for testing C compilers. A test is run by compiling it with the compiler under test and subsequently executing the binary that the compiler produced. A compiler bug manifests itself by an assertion failure during the execution of the binary.
The code generated tests one specific part of a C compiler: how it implements passing values to a function by function call and returning a value from it using the return statement. Stated simpler, the generated code by Quests checks that a value passed to a function is indeed received unaltered from the caller and likewise when a valuereturned to the caller. Conceptually, the code generated by Quest looks like this:
static int i = 1;
static char c = 'c';
static float r = 3.14;
static float f(int ii, char cc)
{
assert(ii == i);
assert(cc == c);
return r;
}
static void test(void)
{
float rr;
rr = f(i ,c);
assert(rr == r);
}
int main(int argc, char **argv)
{
test();
}
By compiling and runnung the code with a C compiler we can test that values are passed and returned to and from a function as expected:
$ quest > testcase.c
$ gcc -o testcase testcase.c
$ ./testcase
If the compiler fails to generate correct code, executing the code will lead to an assertion failure that would highlight the problem. Quest generates tests in a random yet controlled way. Generating a test case, compiling and running it can be done in a loop until a bug is found.
Why is compiling function calls even an issue? What looks simple on the surface of C source code is quite complicated inside a compiler. How values are supposed to be passed between functions is governed by a calling convention. Such a convention specifies how values are passed in registers and on the stack using a specific alignment. C calling conventions in particular are complicted when structured values are passed, like structures, or var args. As a consequence, even mature compilers exhibit bugs and Quest helps to find them. The beauty is that we don't have to know the details of a calling convention in order to test it: generating test cases and executing them is enough.
Directory doc/ contains a paper whith more details:
Christian Lindig. Random Testing of C Calling Conventions. In
Jong Deok Choi and Raimondas Lencevicius, editors, Sixth
Internation Symposium on Automated and Analysis-Driven
Debugging", ACM Press, Monterey, CA, USA, September 2005.
Directory bugs contains a few compiler bugs that were uncovered with Quest in the past. This provides also examples for code generated by Quest.
Quest is available from OCaml's package manager Opam:
$ opam install quest
This will install the quest binary.
Quest is implemened in OCaml as a literate program. It should build on Unix systems without a problem and does not depend on libraries outside the OCaml standard libraries and its tools.
$ opam install lipsum
$ make PREFIX=$HOME # see the Makefile
To be written. See doc/quest.pod for now.
-
The manual page quest.pod in doc/ explains Quest in detail.
-
If you have a Quest binary you can view the manual page also with
quest -man
which emits it to stdout. A short summary of options is available with:
quest -help
-
The Lua 2.5 reference manual in doc/ introduces the Lua scripting language. It is only required to define new generators beyond the existing ones.
Recently testing calling concentions was discussed in a thread on Hacker News:
-
Quest pretty prints C code to keep it readable.
-
Quest can split test cases into two files such that they may be compiled by different compilers to check their interoperability.
-
Neither Quest itself, nor the generated code requires any special run-time libraries or configuration files.
-
Quest is developed on Mach OS X but contains no system-specific source code. However, the build process needs to be adapted for other systems.
-
Quest is controlled by an embedded Lua interpreter. This makes it easy to tailor the generation of test cases.
Christian Lindig lindig@gmail.com. Please report bugs on GitHub https://github.com/lindig/quest
quest - generates C code for testing a C compiler's calling convention
Quest generates C code for testing C compilers. A test is run by compiling the generated code it with the compiler under test and subsequently executing the binary that the compiler produced. A compiler bug manifests itself by an assertion failure during the execution of the binary. This leads to automatic testing: generate and execute tests until an assertion failure is found.
The code generated by Quest tests one specific part of a C compiler: how it implements passing values to a function by function call and returning a value from it using the return statement. Stated simpler, the generated code by Quests checks that a value passed to a function is indeed received unaltered from the caller and likewise when a valuereturned to the caller.