Skip to content

Get Started with LLFI Using Command Line

Jump to bottom
Behroosh edited this page May 29, 2017 · 7 revisions

Overview

This page describes the procedure of performing fault injections on the target program. The first part of the page is a description of the procedure. However, since the whole procedure relies on a configuration script named input.yaml, we will describe the parameters in input.yaml in the second part of the page.

Fault Injection Procedure

LLFI accepts a single intermediate representation (IR) file as input and performs the fault injections. The input IR code can be either in a readable (.ll) or a non-readable format (.bc). To do the fault injection, you first need an input.yaml file in the same directory as the input IR file. The input.yaml should contain appropriate configurations that will be further explained in the second part of the page.

LLFI also provides a tool named GenerateMakeFile under <LLFI_DST_ROOT>/tools/ to compile your source files to a single IR file. Run GenerateMakeFile --help to see more details. However, there might be cases that GenerateMakeFile will not work, and in that case, you need to generate the single IR file by yourself. Note that the IR file generated by GenerateMakeFile do not contain any optimizations, so you may want to do some optimizations (e.g. -mem2reg) before passing it to LLFI. Finally, if you want to include debugging symbols in the generated .ll file (e.g., for debugging with LLDB), invoke the GenerateMakeFile script with the --debug option. To compile all C/C++ sources in a directory, execute tools/GenerateMakefile --readable --all -o <output>.ll.

The procedure contains 3 main steps, and each of which is coded in a separate file. Here is a brief description of each step/file:

1) instrument:

The script exists under <LLFI_DST_ROOT>/bin/. The goal of the script is to instrument the input IR file with function calls to profiling and fault injection libraries and get the transformed IR files (*-profiling.ll and *-faultinjection.ll). After that, the script can also generate profiling executable (*-profiling.exe) and fault injection executable (*-faultinjection.exe). The script requires the input.yaml under the same directory as input IR file, and generates the instrumented IR files (and executables) under a sub-directory (default: llfi/). More details can be found by running instrument --help.

This should be run whenever changes are made to compileOption in input.yaml.

Example:

<LLFI_DST_ROOT>/bin/instrument -l<shared_library> --readable factorial.ll

The command above will create a folder llfi/ under the directory of factorial to store the generated instrumented IR code (factorial-profiling.ll, factorial-faultinjection.ll) and executables (factorial-profiling.exe, factorial-faultinjection.exe).

Another generated file under llfi/ is factorial-llfi_index.ll. This is same to the original IR code factorial.ll except llfi_index is added to each instruction to identify the instructions. This file can be used in helping fault analysis and specifying a set of instructions as injection targets.

Under the same directory of factorial.ll (the parent directory of llfi/), the following files are generated:

llfi.config.compiletime.txt is created as an output file of LLFI's LLVM passes. It contains the description of the current failure class, failure mode and other compile-time information may be used in GUI.

llfi.stat.totalindex.txt stores the total number of static instructions labeled as candidates of fault injection.

llfi.stat.graph.dot will be generated if tracingPropagation: is set to True in the input.yaml. It is used in tracing analysis of the fault propagation.

Option --readable means the generated IR code will be in readable format, namely .ll files. The input IR code can be either in readable (.ll) or non-readable format (.bc). Option -l<shared_library> specifies the shared libraries required for the program to link/run. It follows the same usage manner of common compilers. -L option is also provided to specify the search directories of shared libraries.

2) profile:

The script exists under <LLFI_DST_ROOT>/bin/. The script executes the profiling executable that is generated in step 1), and produces llfi.prof.stat.txt which will be used by the fault injection executable. (DEPRECATE: To run this, you need to sit at the parent directory of the profiling executable (*-profiling.exe), which is the same directory as input IR file if you haven't moved files after step 1)). More details can be found by running profile --help.

Example:

<LLFI_DST_ROOT>/bin/profile ./llfi/factorial-profiling.exe 6

The command above will profile the executable: factorial-profiling.exe with 6 as the argument. After you profile the program, you should find following files under the same directory of factorial.ll:

llfi.stat.prof.txt: stores the total number of dynamic counts of potential injection points. If the total_cycle in this file is equal to zero, it means no potential injections points are found so that no injection can be done upon the targeting IR code with current compile-time options.

Under llfi/, following files should be generated:

baseline/golden_std_output: stores the output of the targeting program which is printed to the stander output of the terminal. baseline/<output file name>.prof.<file extension> (dependent): This file will be generated if the targeting program has output file saved to disk. LLFI monitors the directory of the targeting IR (factorial.ll not llfi/) and treat all generated files during profiling phase as output files of the targeting program and stores them under llfi/baseline/. baseline/llfi.stat.trace.prof.txt: this file will be generated if tracingPropagation: is set to True in the input.yaml. It stores the records the trace all the dynamic instructions. It is used in trace analysis. prog_input/<program input files>: LLFI will examine the arguments of the program and if they are files, LLFI will stored them under llfi/prog_input in case the target program removing the input file after running.

###3) injectfault: The script exists under <LLFI_DST_ROOT>/bin/. The script executes the fault injection executable that is generated in step 1). (DEPRECATE: To run this, you need to sit at the parent directory of the fault injection executable (*-faultinjection.exe), which is the same directory as input IR file if you haven't moved files after step 1)). More details can be found by running injectfault --help. To run this, input.yaml must contain parameters under runOption: Please see below for the input.yaml guide.

These scripts should be run in order initially for each target IR file. After compilation of the target program, and the creation of the profiling and fault injection executables, you may choose to only run injectfault to test different fault injections. However, if you need to make changes on the compileOption in input.yaml (e.g. Inject into different sets of instructions.), you need to restart the procedure from step 1)

Example:

<LLFI_DST_ROOT>/bin/injectfault ./llfi/factorial-faultinjection.exe 6

The command above will initiate fault injection experiment with specified times (defined as numOfRuns: in the runOption: of input.yaml) of faulty runs.

After it finishes, following files will generated:

llfi.config.runtime.txt: It stores the description of the run-time option of LLFI. It is referred by the run-time library of LLFI to determine when/where/how to inject a fault. It also can be used in GUI and later analysis, but not recommended as it may be rewritten several times during faulty runs and only the information of the last one will remain in this file.

llfi/error_output/: The files under this directory are numbered according to the order of faulty runs and stores the error messages from the corresponding runs. For example, errorfile-run-m-n stores the error message returned from the n th run of the m th runOption: defined in the input.yaml. Common error messages are Program crashed with a return code or Program hang.

llfi/std_output/: The files under this directory stores the standard output of the faulty runs. The names of the files follow the same manner as the files in llfi/error_output.

llfi/prog_output: The files under this directory stores the output saved to the disk of the faulty runs. The names of the files follow the same manner as the files in llfi/error_output.

llfi/llfi_stat_output/llfi.stat.fi.injectedfaults._m_-_n_.txt: These files store important information of the injected fault for each faulty run. The names of these files also follow the same manner as the files in llfi/error_output. Each file contains the detailed low-level information of the injected fault of its corresponding faulty run. Here is an example of its content:

FI stat: fi_type=bitflip, fi_max_multiple=3, fi_index=1998, fi_cycle=1523447, fi_reg_index=0, fi_reg_pos=2 fi_reg_width=64, fi_bit=47, opcode=getelementptr

The message above shows that this injection is done with bitflip type fault (fi_type), the maximum number of bit-flips that could occur in one run of the program under test is equal to 3 (fi_max_multiple), the llfi_index of the static instruction being corrupted is 1998 (fi_index), the injection is done at dynamic counts: 1523447 (fi_cycle), the corrupted register is the first potential fault injection register of the selected instruction (fi_reg_index) and is the 2nd source register (fi_reg_pos -> 0: destination register, 1: first source register, 2: second source register ...). The target register is 64-bit wide (fi_reg_width), the flipped bit is 47th bit (fi_bit). The opcode of the selected instruction is getelementptr, an LLVM IR instruction type, (opcode).

llfi/llfi_stat_output/llfi.stat.trace._m_-_n_.txt: These files are tracing records of the corresponding faulty runs. These are generated if tracingPropagation: is set to True and used later in trace analysis.

##input.yaml parameters The input.yaml file is used to configure the compilation and fault injection parameters and options. This file must be provided in same directory as your single IR file before going through the whole procedure. If you have moved the directory created in step 1), you also need to provide the file in the parent directory of your profiling/faultinjection executable in order to run the step 2)/3).

A few things to note about yaml formating is that you must use spaces instead of tabs, and that you must leave one space after using a : or - sign. Incorrect formatting will cause the script to crash or throw an error.

input_masterlist.yaml under <LLFI_SRC_ROOT>/bin/ is provided as a guide that contains all possible input keys and values that will be described below. It will be a useful reference for constructing your own input.yaml. There is a sample input.yaml is provided as sample_input.yaml also.

The input.yaml file can be thought of as having 2 mandatory sections. compileOption:, and runOption:. An optional kernelOption:, and an optional timeOut: can be selected also. Here is the top hierarchy of an input.yaml file, we will go through each block in the following paragraphs:

compileOption:
	<compile-time options>
runOption:
	- run:
	    <run-time options>
	[- run:
	    <run-time options>]
[defaultTimeOut: <default time out value>]
[kernelOption:
	- forceRun]

###compileOption hierarchy

compileOption hierarchy defines the compile-time options of LLFI, including instruction selector, register selector, tracing options, etc. instruction selector and register selector are used to specify the scope of fault injection. Through configuring these two selectors, user can constrain the potential locations where a fault may occur.

compileOption:
	### Required field ###
    instSelMethod:
        <content of inst selector>...
    regSelMethod: <content of reg selector>...

	### Optional field ###
    tracingPropagation: True/False
    tracingPropagationOption:
		<content of tracing propagation option>
	includeInjectionTrace:
	    - forward
	    - backward

####instSelMethod: Define the instruction selector instSelMethod is the block to define the instruction selector. It is located within the dictionary of compileOption. Below is the structure of this block:

    instSelMethod:
      - <insttype/funcname/customInstselector>:
          include:
            - <inst types/func names/name of custom inst selector>
          exclude:
            - <inst types/func names>
          options:
            - <command line options for the inst selector>

#####Built-in instruction selectors (for Hardware Injection):

  • insttype: to include/exclude certain types of instructions as fault injection targets. Lists are specified as described in input_masterlist.yaml. An example of instSelMethod block with insttype instruction selector applied:
    instSelMethod:
      - insttype:
          include:
            - all
          exclude:
            - ret
  • funcname: to include/exclude certain list of functions as fault injection targets. Lists are specified as described in input_masterlist.yaml. Notice that funcname is currently not supported in the GUI. An example of instSelMethod block with funcname instruction selector applied:
    instSelMethod:
      - funcname:
          include:
            - refresh_potential
            - flow_cost
            - flow_org_cost
            - primal_feasible
            - dual_feasible
            - main
            - getfree
          exclude:
            - dual_feasible
            - main

#####Custom instruction selectors: LLFI also enables the user to specify a custom instruction selector to meet a specific scope of fault injection. Custom instruction selectors can be programmed and archived in LLFI llvm pass library so that user can specify it in input.yaml to use it. LLFI also enables the user to add options to that specific instruction selector through LLVM command line interface.

Custom instruction selectors are specified with option: customInstSelector, options passed to that selector are grouped in option: option under the name of customInstSelector.

  • llfiindex: Here is an example with llfiindex instruction selector, with options injecttoindex to specify the static instructions with index=2293 and index=568 to the potential injection set. Notice that llfiindex is currently not supported in the GUI
    instSelMethod:
      - customInstselector:
          include:
            - llfiindex
          options:
            - -injecttoindex=2293
            - -injecttoindex=568

LLFI also ships with many ready-to-go custom instruction selectors, many examples can be found under test_suite/.

For Software Failure injection, specify a failure from this list under the include: of customInstselector:. LLFI does not support multiple software failures injection, however, to speed up the injection process, multiple failures can be specified, and this script can be used to individually inject each failure.

####regSelMethod: Define the instruction selector regSelMethod is the block to define the register selector. It is located within the dictionary of compileOption. regSelMethod: can be regloc or customregselector. Below is the structure of this block:

    regSelMethod: <regloc/customregselector>
    
    ### if regloc is selected as regSelMethod ###
    regloc: dstreg/allsrcreg/srcreg1/srcreg2/srcreg3/srcreg4
    
    ### if customregselector is selected as regSelMethod ###
    customRegSelector: <name of the custom register selector>
  • regloc: if regloc is selected as regSelMethod, this option will enabled to specify srcreg, dstreg etc... as described in input_masterlist.yaml, to include certain operands as fault injection register target. Example, select the destination register of the selected instructions as potential fault injection points:
    regSelMethod: regloc
    regloc: dstreg
  • customregselector: if customregselector is selected as regSelMethod, this option is enabled to specify the custom register selector to select registers as fault injection register target. Example, use custom register selector: BufferOverflow(API) to select potential fault injection points from the selected instructions.
    regSelMethod: customregselector
    customRegSelector: BufferOverflow(API)

For Software Failure injection, use customRegSelector: Automatic. Note: if customRegSelector is set to Automatic: customRegSelector: Automatic, LLFI will use the name of customInstselector as the name of customRegSelector. This requires custom instruction selector and custom register selector to have the same name.

####tracingPropagation: switch of tracing option

-tracingPropagation, when set to True, turns on instruction tracing, which traces the dynamic instruction values of all instructions. This can be used to study the fault propagation at the instruction level.

-tracingPropagationOption, works only when tracingPropagation is set to True, controls the configuration of the tracing propagation function:

  • maxTrace accepts an integer that specifies how many instructions to trace after fault is injected during fault injection run.
  • debugTrace can be selected as true to include debug information during the instrumentation, usually set to False.
  • generateCDFG, set to True when the tracing results will be used in generating Control/Data Flow Graph.

An example of tracingPropagation block:

    tracingPropagation: True # trace dynamic instruction values.

    tracingPropagationOption:
        maxTrace: 250
        debugTrace: True/False
        generateCDFG: True

####includeInjectionTrace: consider the backward/forward slice -includeInjectionTrace can be omitted or have -forward and/or -backward, to include forward/backward trace of the selected instructions above as fault injection targets.

###runOption hierarchy runOption defines the run-time options of LLFI. An runOption dictionary may contain multiple number of run blocks, each one will be considered as one run-time configuration and results from different blocks will be grouped separately (remember the errorfile-run-m-n in llfi/error_output/). Each run block has following options. You can specify up to 12 options, numOfRuns is mandatory.

  • numOfRuns: This is self-explanatory.
  • fi_type: type of faults to be injected. bitflip, stuck_at_0 , stuck_at_1 etc, default is bitflip. You can also develop custom fault types and specify them here. Use AutoInjection when injecting Software Failure. Note: if fi_type is set to AutoInjection: fi_type: AutoInjection, LLFI will use the name of customInstselector as the name of fault type. This requires a fault type share a same name with custom instruction selector exists.
  • fi_cycle: The runtime cycle to do the fault injection; if not specified, LLFI will randomly pick one of total runtime cycles collected in the profiling run. We recommend you not to specify the option unless you know what you are doing (e.g. you are reproducing a fault injection experiment).
  • fi_reg_index: The register index you intend to inject faults into. If not specified, LLFI will randomly pick one of the target registers of the selected instruction for fault injection. We recommend you not to specify the option unless you know what you are doing (e.g. you are reproducing a fault injection experiment). This option is not valid for Software Failure injection.
  • fi_index: The LLFI index you intend to inject faults into. NOTE here if you specify this option, LLFI will inject faults into every runtime instances of the specified instruction. Also NOTE that if you have specified fi_cyle, this option will not be used. If you want to inject randomly into ONLY ONE runtime instance of a specific instruction, use the custom instruction selector named llfiindex for compileOpt and fi_cycle for runtimeOpt. We recommend you not to specify the option unless you know what you are doing (e.g. you are reproducing a fault injection experiment).
  • fi_bit: The bit you want to inject faults into. If not specified, LLFI will randomly pick one of the target register bits for fault injection. We recommend you not to specify the option unless you know what you are doing (e.g. you are reproducing a fault injection experiment). This option is not valid for Software Failure injection.
  • fi_num_bits: specify the number of bits to flip for one faulty register. ONLY works when fi_type is set to bitflip.
  • window_len: specify the maximum number of dynamic instances between TWO consecutive faults. This option is not valid for Software Failure injection.
  • fi_max_multiple: Specify the number of bits to flip in one run of the program under test. ONLY works when fi_type is set to bitflip. NOTE that these number of bits are flipped in multiple words. The distance between each consecutive injection is controlled by window_len_multiple_startindex and window_len_multiple_endindex. Also NOTE that window_len and fi_max_multiple cannot be used with this option. Use fi_max_multiple along with window_len_multiple_startindex and window_len_multiple_endindex if you want to inject multiple bit-flip faults in MULTIPLE words. If you want to inject multiple faults into the SAME word, use fi_num_bits instead.
  • window_len_multiple_startindex: Specify the lower bound for the number of dynamic instructions executed between TWO consecutive faults. NOTE that the actual distance between each TWO consecutive injections is randomly selected between the window_len_multiple_startindex and window_len_multiple_endindex. Also NOTE that window_len and fi_max_multiple cannot be used with this option. This option is not valid for Software Failure injection.
  • window_len_multiple_endindex: Specify the upper bound for the number of dynamic instructions executed between TWO consecutive faults. NOTE that the actual distance between each TWO consecutive injections is randomly selected between the window_len_multiple_startindex and window_len_multiple_endindex. Also NOTE that window_len and fi_max_multiple cannot be used with this option. This option is not valid for Software Failure injection.
  • timeOut: use a different timeout value for this run block, instead of using a defaultTimeOut value (by default set to 500 seconds).

An example of several runOption block:

runOption:
    - run:
        numOfRuns: 5
        fi_type: bitflip

	### define a specific cycle, reg_index and bit to reproduce a fault, note a specific timeOut value is set for this configuration ###
    - run:
        numOfRuns: 1
        fi_type: bitflip
        fi_cycle: 15574729
        fi_index: 2293
        fi_reg_index: 0
        fi_bit: 49
        timeOut: 1000

	### define a specific cycle, reg_index but no bit position, a specific timeOut value is set ###
    - run:
        numOfRuns: 3
        fi_type: stuck_at_1
        fi_cycle: 2661577811
        fi_index: 664
        fi_reg_index: 0
        timeOut: 1000

	### define a specific maximum number of faults to occur in one run of a program in multiple words. The distance between each injection is a randomly selected value between the start and the end window length index###
    - run:
        numOfRuns: 5
        fi_type: bitflip
        fi_max_multiple: 4
        window_len_multiple_startindex: 2
        window_len_multiple_endindex: 10

	### defines a campaign with multiple bit-flip injections starting from injecting a fault in a specific location that is specified by the fi_cycle, fi_cycle, fi_index, fi_reg_index, fi_reg_pos and fi_bit. This configurations is useful to compare the result of a single bit-flip injection with the multiple bit-flip injections.###
    - run:
        numOfRuns: 5
        fi_type: bitflip
        fi_cycle: 684347
        fi_index: 417
        fi_reg_index: 0
        fi_reg_pos: 1
        fi_bit: 15
        fi_max_multiple: 3
        window_len_multiple_startindex: 10
        window_len_multiple_endindex: 10

###kernelOption If you wish to not be prompted by a confirmation screen whenever all runOptions are specified and numOfRuns is greater than 1, you may include the flag -forceRun. Remember that this will end up injecting exactly the same fault in each run. We recommend you not to specify the option unless you know what you are doing.

###defaultTimeOut To specify a defaultTimeOut value for all fault injection experiments EXCEPT another timeOut option is specified in a runOption dictionary. The defaultTimeOut/timeOut value is used to decide whether the program hangs or not. We recommend specifying this to be high enough to ensure you don't get false positives.

Dependable Systems Lab - University of British Columbia (UBC)

Clone this wiki locally