tests

This directory contains test cases for the repc crate.

Each directory within ./testfiles is a test case. Each such directory contains a file called input.txt and a directory output. input.txt is the input to repc in form of a cly file. For each target, output contains a file called {TARGET_NAME}.expected.txt. This file is in the form of a cly output file.

During testing the matching output to input.txt is calculated for each target and compared to the expected output. If they are not the same, a file {TARGET_NAME}.actual.txt is generated next to the expected.txt file and the test fails.

Configuration

Testing can be configured at the test case level and globally.

Test case configuration

Each test case can contain a file called config.toml that takes the following form:

include_compilers = ["clang"]
exclude_compilers = ["msvc"]
include_targets = ["x86_64-pc-windows-msvc"]
exclude_targets = ["x86_64-unknown-linux-gnu"]
use_clang_for_msvc_targets = true

Each key is optional. If an exclude/include key is set, it is used to filter the set of targets tested.

The use_clang_for_msvc_targets key is only used during test generation. See below.

Global Configuration

Next to this readme is a file called config.toml that has the following form:

include_compilers = ["clang"]
exclude_compilers = ["msvc"]
include_targets = ["x86_64-pc-windows-msvc"]
exclude_targets = ["x86_64-unknown-linux-gnu"]
include_tests = ["0013"]
exclude_tests = ["0014"]

compiler = "./compiler.sh"

The include/exclude keys work like in the test case configuration except that they affect all test cases. These keys should only be used during development.

The compiler key is used for test generation. See below.

Test Generation

1. For each test case and target (except as excluded via config.toml) we check if the expected.txt does not exist or is outdated.
2. If so, we generate C code that represents the types in input.txt.
3. We compile the C code with the target's compiler, generating debug info in the process.
4. We extract the layouts of the types from the debug info.

The compilation happens via the compiler specified in the global configuration. This compiler is invoked once for each test case/target combination. The following environment variables are set during the invocation:

• COMPILER: The compiler to use: gcc, clang, or msvc.
• TARGET: The target to compile for.
• INPUT: The path of the input C file.
• OUTPUT: The path to which to write the output.

The COMPILER is usually the system compiler of the TARGET except that some test cases opt into using clang for the MSVC targets by setting use_clang_for_msvc_targets = true.

INPUT and OUTPUT are absolute paths within a temporary directory. The compiler can use this directory to store additional temporary files.

The thing to store at OUTPUT depends on the COMPILER. For gcc and clang, it should be an object file. For msvc it should be a pdb file generated by the /Zi flag.

In the case of gcc and clang, the object file must contain debug information in the DWARF5 format. Furthermore, bitfields must be described in DWARF4 format. For gcc this is achieved by passing the -gdwarf-5 flag. For clang this is achieved by passing the -gdwarf-5 -glldb flags.

This repository contains a compiler.sh that works on my machine. It will not work on your machine but can serve as a reference.

We use the following compiler versions:

• msvc: 19.28.29337. Other versions will probably also work.
• gcc: 9.3. Newer versions will probably also work.
• clang: https://github.com/mahkoh/llvm-project/tree/fixed-dwarf We use a custom version because LLVM generates incorrect debug information due to a bug.

You can acquire msvc by using https://github.com/mstorsjo/msvc-wine. Follow mstorsjo/msvc-wine#23 to avoid some problems.

You can compile and install clang with the following commands:

cd /path/to/llvm/root
mkdir build
cd build
cmake -G Ninja -DCMAKE_INSTALL_PREFIX=$HOME/bin/custom-clang -DLLVM_ENABLE_PROJECTS=clang -DCMAKE_BUILD_TYPE=Release ../llvm
ninja
ninja install

Replace CMAKE_INSTALL_PREFIX by the desired install path. Do not attempt to build clang in debug mode. It will take longer and consume unholy amounts of ram and disk space.

You have to build a copy of binutils and gcc for each target. Luckily they build relatively fast so that this is only a matter of a few hours. To do this, first check out the source code of GCC and binutils. Then modify the environment variables at the top of ./gcc/build to fit your system. Then create a separate directory and execute ./gcc/build-all from within that directory. This will compile the necessary tools and install them below this directory.

Once this setup is complete, you can run the generator binary within this directory to generate the test cases.