Using FIDL to create a Custom Software Fault Injector and a Custom Instruction Selector

Overview

FIDL (Fault Injection Description Language) is a systematic way of specifying a Software Fault Model for the LLFI system. The FIDL Algorithm takes a FIDL yaml and automatically generates a custom instruction selector and a custom fault injector.

This tool is located under <LLFI_SRC_ROOT>/tools/FIDL/ directory as FIDL-Algorithm.py.

FIDL-Algorithm.py takes a FIDL (Fault Injection Description Language) yaml and 
generates an instruction/register selector C++ code, and a fault injection 
run-time C++ code.

Usage: FIDL-Algorithm.py [OPTIONS]

List of options:
-a <FIDL yaml> : add a FI run-time and selector from a FIDL yaml
-r <name/type> : removes the specified injector by '<FMode>(<FClass>)'
                 or remove all 'custom' or 'default' injector
-l <type>      : lists all active injectors/selectors by 'custom' or 'default'
-h             : shows help

Every time the content of a FIDL yaml is changed, this script should be executed
(-a <FIDL yaml>) to reflect the change(s) in the generated C++ code.

Failure Class and Failure Mode pair should be unique, otherwise the previous 
Failure Class and Failure Mode pair is overwritten.

FIDL yaml can be provided to the tool file by file, or multiple FIDL yaml can be provided as a single file by storing each configuration as items in a list.

To use the tool, create a FIDL yaml file (described in the section below) and execute FIDL-Algorithm.py -a <fidl file>.yaml and rebuild LLFI. For injection using the GUI, the new fault should automatically appear if it is applicable. For injection using the command line, simply include the fault in the input.yaml as you would any other software faults.

LLFI comes with 39 default software failures, with 36/37 failures written as a FIDL yaml:

A) API Failure Class:

1. BufferOverflow(API) - causes a buffer overflow in the fread or fwrite function (size_t count is incremented by 45).
2. BufferUnderflow(API) - causes a buffer underflow in the fread or fwrite function (size_t count is decremented by 40).
3. InappropriateClose(API) - fclose is called right after a call to fopen.
4. IncorrectOutput(API) - corrupts a randomly selected return value.
5. NoClose(API) - the file remains open after a call to fclose (a fake file is opened and closed instead).
6. NoOpen(API) - corrupts const char * filename of fopen.
7. NoOutput(API) - while (1); is injected before a randomly selected return instruction.
8. WrongAPI(API) - corrupts FILE * stream of fread, fwrite, fgetc, or fopen.
9. WrongMode(API) - corrupts const char * mode of fopen (leading to the file being opened in the wrong mode).

B) Data Failure Class:

1. BufferOverflowMalloc(Data) - under allocates malloc or calloc by 40 bytes (size_t).
2. BufferOverflowMemmove(Data) - increases the number of bytes to move (size_t num) in memcpy or memmove by 45.
3. DataCorruption(Data) - corrupts a randomly selected call's source register 0 (similar to hardware fault injection).
4. WrongDestination(Data) - corrupts void * destination of memcpy, memmove, or memcmp.
5. WrongPointer(Data) - corrupts void * ptr (pointer to be read/written) of fread, or fwrite.
6. WrongSource(Data) - corrupts void * source of memcpy, memmove, or memcmp.

C) Input/Output Failure Class:

1. WrongRetrievedAddress(I/O) - corrupts FILE * stream of fread, or const void * ptr of fwrite.
2. WrongRetrievedFormat(I/O) - changes size_t size in fread (changing the size of each element to be read).
3. WrongSavedAddress(I/O) - corrupts void * ptr of fread, or FILE * stream of fwrite.
4. WrongSavedFormat(I/O) - changes size_t size in fwrite (changing the size of each element to be written).

D) Message Passing Failure Class:

1. Deadlock(MPI) - corrupts int socket of recv, or send (system attempts to send/receive from the wrong socket, resulting in a deadlock).
2. InvalidMessage(MPI) - increments the pointer void * buffer of recv, or send by 1024.
3. InvalidSender(MPI) - corrupt struct sockaddr *address of connect, or accept.
4. NoAck(MPI) - while (1); is injected before returning from recv().
5. NoDrain(MPI) - sets int flags of recv to 5000.
6. NoMessage(MPI) - while (1); is injected before calling recv, or send.
7. PacketStorm(MPI) - decrements size_t length (specified buffer length) of recv by 40

E) Resource Failure Class:

1. CPUHog(Res) - delays a randomly selected return instruction by 3s.
2. Deadlock(Res) - injects a deadlock before a call to pthread_join but only on the calling thread.
3. InvalidPointer(Res) - corrupts a randomly selected return value from malloc, or calloc.
4. LowMemory(Res) - allocates some memory for malloc or calloc (may be more or less than the amount requested), but such that no more memory can be allocated after this call.
5. MemoryExhaustion(Res) - malloc or calloc returns NULL, and no more memory can be allocated after this call.
6. MemoryLeak(Res) - free frees newly allocated memory (the intended location (void * ptr) remains un-freed).
7. StalePointer(Res) - frees an allocation right after a call to malloc, or calloc.
8. ThreadKiller(Res) - pthread_cancels() pthread_t thread of pthread_join.
9. UnderAccumulator(Res) - over allocates malloc or calloc by 40 bytes (size_t).

F) Timing Failure Class:

1. RaceCondition(Timing) - causes pthread_mutex_lock() or pthread_mutex_trylock() to lock a fake mutex (inducing the race condition).
2. HighFrequentEvent(Timing) (not specified as a FIDL yaml) - delays a randomly selected return instruction by 3s (this failure has a higher chance (2x) of selecting returns from fread, fopen, and fwrite).

These failures are automatically built with the setup script. You can also build these yourself by executing ./FIDL-Algorithm.py -a default. 36 of the failures are specified in the file <LLFI_SRC_ROOT>/tools/FIDL/config/default_failures.yaml. To modify the behaviour of these failures, modify this file and rebuild LLFI.

FIDL yaml parameters

This is an example of a FIDL yaml:

Failure_Class: Class3
Failure_Mode: FMode3

New_Failure_Mode:
    Trigger: 
        call: [fread, fwrite]
    Target:
        src:  
          fread: [2]
          fwrite: [0]
    Action:
        Perturb: InappropriateCloseInjector
        option: False

Failure_Class: and Failure_Mode: will be the name (<F_Mode>(<F_Class>) of the fault model. (required)

Under New_Failure_Mode: you can specify the following options:

Trigger: (required)

Under Trigger, you can specify call: [<selected instructions>] (as a list) to select these instructions for fault injection, call*: [<keywords>] (as a list) to select all instructions containing these keywords, call: all to select all instructions, or use return to target all return and return values. If selecting all instructions, only 1 src register or dst register can be specified.

Trigger*: (not required)

To narrow down the range of targeted instructions, you can specify LLFI indices here (as a list) and the LLFI system will only target these instructions for injection. For example, adding Trigger*: [3, 4, 7, 9] will restrict the LLFI indices which can be injected to 3, 4, 7, and 9.

Target: (required)

If return is selected in Trigger:, this does not need to be specified (the return value is automatically selected), otherwise, you can specify dst or src. You cannot simultaneously inject both a src and a dst register. If specifying src, selected from 1 or more register(s) for each instruction (as a list), like so:

- Failure_Class: API
  Failure_Mode: WrongAPI

  New_Failure_Mode:
    Trigger: 
      call: [fread, fwrite, fgetc, fopen]
    Target:
      src: 
        fread: [3]
        fwrite: [3]
        fgetc: [2]
        fopen: [0, 1]
    Action:
      Corrupt

Action: (required)

The following basic injection can be specified:

1. Corrupt - Uses the BitCorruptionInjector to to flip a random bit in the selected register
2. Freeze - A while (1); is injected.
3. Delay - Delays computation in the injected instruction by 3s.

More complicated injections can be specified under Action: Perturb::

MemoryLeakInjector - allocates a block which is not freed afterward
WrongFormatInjector - changes the format of fread or fwrite
PthreadDeadLockInjector - injects a deadlock via a mutex
PthreadThreadKillerInjector - cancels a thread
PthreadRaceConditionInjector - unlocks a mutex
StalePointerInjector - frees memory 
ChangeValueInjector
InappropriateCloseInjector
MemoryExhaustionInjector

For ChangeValueInjector you need to specify a boolean option: and an integer value:, like so:

Action:
  Perturb: ChangeValueInjector
  option: False
  value: -45

True indicates that the value will be replaced by value:, while False indicates that the value will increased by value:.

For MemoryExhaustionInjector, you just need to specify a boolean option:, like so:

Action:
  Perturb: MemoryExhaustionInjector
  option: False

True indicates that no memory should be allocated (NULL is returned), while False indicates that some memory should be allocated for malloc or calloc. Note that in both of these options, no more memory can be allocated after the injection.

For InappropriateCloseInjector you also need to specify a boolean option:, like so:

Action:
  Perturb: InappropriateCloseInjector
  option: False

True indicates the FILE should be close, while False indicates that a fake FILE will be open and the pointer to it passed back in *buf.

For further details regarding what each action does, refer to the file <LLFI_SRC_ROOT>/runtime_lib/_SoftwareFaultInjectors.cpp.

If the above injectors do not meet your need, you can also specify a Custom_Injector:, like so:

Failure_Class: Class2
Failure_Mode: FMode2

New_Failure_Mode:
    Trigger: 
        call: [fread, fwrite]
    Trigger*: [1, 50, 55, 60]
    Target:
        src: 
            fread: [2]
            fwrite: [0]
    Action:
        Perturb: Custom_Injector

Custom_Injector: | 
    int *Target = (int *) buf;
    *Target = *Target + 1000;

The code provided under the Custom_Injector: field will be inserted into this function

virtual void injectFault(long llfi_index, unsigned size, unsigned fi_bit, char *buf);

which will perform the fault injection at a randomly selected instruction.

For some more examples of FIDL yaml files, visit <LLFI_SRC_ROOT>/tools/FIDL/config/default_failures.yaml or the <LLFI_SRC_ROOT>/tools/FIDL/sample_scripts directory.