|
Warning
|
The contents of this repository and linked repositories are solely for research purposes and may contain software with vulnerabilities, such as outdated libraries. Do not use in production. |
This material is based upon work supported by the Naval Information Warfare Center Pacific and the Defense Advanced Research Project Agency under Prototype Other Transaction Agreement No. N66001-23-9-4004. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Naval Information Warfare Center Pacific or the Defense Advanced Research Project Agency.
This quick start is tested on Ubuntu 20.04. It assumes dependencies, including the Simics simulator, are installed, Simics is installed. To install Simics, see Install Simics. To install the remaining dependencies, see Install other dependencies.
To setup user-space C3 environment:
# Clone repository
mkdir -p [CC_INSTALL_DIRECTORY]
cd [CC_INSTALL_DIRECTORY]
git clone --recurse-submodules https://github.com/IntelLabs/c3-simulator.git .
# Configure Simics project with default SIMICS_BIN=/opt/simics/simics-6/simics-latest/bin
make -f config-user.mk simics_setup
# or, if needed, set SIMICS_BIN explicitly
make -f config-user.mk simics_setup SIMICS_BIN=/some/other/path/to/simics
# Generate checkpoint (built upon /opt/simics/checkpoints/glibc_latest.ckpt, if available)
make ckpt-cc_llvm
# Build C3 Simics modules
make -BTo run simple workloads (e.g., microbenchmarks/hello.cpp):
./simics scripts/runworkload_common.simics \
src_path=microbenchmarks \
src_file=hello.cpp \
workload_name=a.out \
model=c3 \
checkpoint=checkpoints/cc_llvm.ckpt \
gcc_flags="-g -gdwarf -Werror -ldl -lm -lpthread -pthread -Iinclude"clang++ microbenchmarks/hello.cpp -o a.out
./simics scripts/runworkload_common.simics \
src_path=. \
src_file=a.out \
workload_name=a.out \
model=c3 \
checkpoint=checkpoints/cc_llvm.ckpt \
nobuild=TRUE \
env_vars="LD_LIBRARY_PATH=/home/simics/glibc/glibc-2.30_install/lib"To run individual unit tests (.e.g, unit_tests/common/gtest_hello.cpp):
./simics unit_tests/runtest_common.simics \
src_path=unit_tests/common \
src_file=gtest_hello.cpp \
workload_name=a.out \
model=c3 \
checkpoint=checkpoints/cc_llvm.ckpt \
include_folders=unit_tests/include/unit_tests \
gcc_flags="-g -gdwarf -Werror -ldl -lm -lpthread -pthread -Iinclude -DC3_MODEL=c3"To run all unit tests:
# Run all unit tests locally on all available models
pytest -v python_tests/test_unit.py --checkpoint checkpoints/cc_llvm.ckpt
# Run all unit tests for specific models (can specify multiple --model args)
pytest -v python_tests/test_unit.py --checkpoint checkpoints/cc_llvm.ckpt --model c3The c3_model has initial functional support for integrity checking. By default, only writes are integrity checked, see Using C3 integrity verification for more details.
To enable integrity support, you can pass enable_integrity=1 to the simics
scripts. The pytest scripts also support two pseudo modules,
c3-integrity and c3-integrity-intra, which
will run on the c3, but will enable module and test
configuration to exercise C3 integrity checking or C3 integrity checking along
with compiler-instrumented intra-object tripwires.
# For single workloads with integrity:
./simics scripts/runworkload_common.simics \
src_path=microbenchmarks \
src_file=hello.cpp \
workload_name=a.out \
model=c3 \
enable_integrity=1 \
checkpoint=checkpoints/cc_llvm.ckpt \
gcc_flags="-g -gdwarf -Werror -ldl -lm -lpthread -pthread -Iinclude"
# For running pre-compiled binaries with integrity:
clang++ microbenchmarks/hello.cpp -o a.out
./simics scripts/runworkload_common.simics \
src_path=. \
src_file=a.out \
workload_name=a.out \
model=c3 \
checkpoint=checkpoints/cc_llvm.ckpt \
nobuild=TRUE \
env_vars="LD_LIBRARY_PATH=/home/simics/glibc/glibc-2.30_install/lib:/home/simics/llvm/llvm_install/lib" \
enable_integrity=1
# For single unit tests with integrity:
./simics unit_tests/runtest_common.simics \
src_path=unit_tests/common \
src_file=gtest_hello.cpp \
workload_name=a.out \
model=c3 \
enable_integrity=1 \
checkpoint=checkpoints/cc_llvm.ckpt \
include_folders=unit_tests/include/unit_tests \
gcc_flags="-g -gdwarf -Werror -ldl -lm -lpthread -pthread -Iinclude -DC3_MODEL=c3"
# For unit tests via pytest:
pytest -v python_tests/test_unit.py --checkpoint checkpoints/cc_llvm.ckpt --model c3-integrityTo enable ICV-based intra-object tripwires, you need to use the C3-enabled LLVM/Clang. This can be done by using the custom clang installed on a C3-kernel enabled checkpoint with (see Custom kernel checkpoint):
# For stand-alone workloads with intra-object integrity:
./simics scripts/runworkload_common.simics \
src_path=microbenchmarks \
src_file=hello.cpp \
workload_name=a.out \
model=c3 \
enable_integrity=1 \
checkpoint=checkpoints/cc_kernel.ckpt \
compiler="/home/simics/llvm/llvm_install/bin/clang++" \
gcc_flags="-ldl -lm -lpthread -pthread -fuse-ld=lld -finsert-intraobject-tripwires=all"
# For single unit tests with intra-object integrity:
./simics unit_tests/runtest_common.simics \
src_path=unit_tests/common \
src_file=gtest_hello.cpp \
workload_name=a.out \
model=c3 \
enable_integrity=1 \
checkpoint=checkpoints/cc_kernel.ckpt \
compiler="/home/simics/llvm/llvm_install/bin/clang++" \
include_folders=unit_tests/include/unit_tests \
env_vars="LD_LIBRARY_PATH=/home/simics/glibc/glibc-2.30_install/lib:/home/simics/llvm/llvm_install/lib" \
gcc_flags="-ldl -lm -lpthread -pthread -fuse-ld=lld -finsert-intraobject-tripwires=all -Iinclude -DC3_MODEL=c3"
# For unit tests via pytest:
pytest -v python_tests/test_unit.py --checkpoint checkpoints/cc_kernel.ckpt --have-kernel --model c3-integrity-intra|
Note
|
At present, the -finsert-intraobject-tripwires option does not support
multiple parallel compilation jobs. If compiling manually (e.g., not with the
runworkload_common.simics script), make sure to set -j1 to avoid parallel
builds.
|
The dependencies and installation is for Ubuntu 20.04. For other systems you may have to adapt the instruction here to fit your environment.
Download and install Simics v.2023.31 from https://software.intel.com/content/www/us/en/develop/articles/simics-simulator.html.
Create an /opt/simics directory owned by the current user.
The following commands can be executed in a directory where both the Simics package bundle and the package manager archive have been downloaded to install Simics:
tar xf intel-simics-package-manager-1.7.5-linux64.tar.gz
intel-simics-package-manager-1.7.5/ispm packages --install-bundle simics-6-packages-2023-31-linux64.ispm --install-dir /opt/simics/simics-6.0.169 --non-interactiveNext, launch the package manager GUI with intel-simics-package-manager-1.7.5/ispm-gui to associate
the needed addons with the Simics base package using the following steps:
-
When asked for the installation path for packages, enter
/opt/simics/simics-6.0.169and click "Save". -
Click the "Addons" tab.
-
Select "QSP-x86", "QSP-Clear-Linux", and "QSP-CPU".
-
Click "Save updates".
-
Close the package manager GUI.
If the addon tab is empty and you encountered a "Unable to load manifest" error during installation, you can navigate to the Platforms tab and manually import the corresponding manifest file from the installation path.
The Simics VMP kernel module significantly accelerate simulation. But as is, it may not be compatible with your system kernel or security requirements (e.g., module signing). If possible, it can be installed with:
/opt/simics/simics-6.0.169/bin/vmp-kernel-installOn Ubuntu 20.04, initial dependencies can be installed with:
apt install git curl make gccThe remaining dependencies can be installed with make -f config-user.mk
install_dependencies (use make -n to dry-run, as this will use sudo).
Alternatively, you may manually install the following dependencies: bison, curl, flex, git, g++-8, libatk1.0-dev, libatk-bridge2.0-dev, libgtk3-dev, python-3-pip, pytest, and pytest-xdist.
|
Note
|
This section, details various build options and configuration for the C3 simulator. Alternatively, you can follow the Quick Start to set up. |
Most of the configuration, build, and install commands use Makefiles. You can
use make -n <target> to dry-run and view commands make would execute.
To initialize the Simics project and build additional dependencies, you can run the following commands:
# To install Simics, download additional dependencies, and extract files
make -f config-user.mk simics_setup
# NOTE: If needed, set SIMICS_BIN (default: /opt/simics/simics-6/simics-latest/bin)
make -f config-user.mk simics_setup SIMICS_BIN=/some/other/path/bin
# Create or update CKPT_GLIBC checkpoint (default: checkpoints/cc_glibc.ckpt)
make ckpt-cc_glibc
# Create or update CKPT_LLVM checkpoint (default: checkpoints/cc_llvm.ckpt)
make ckpt-cc_llvm
# Build Cryptographic Computing Simics modules
make -BAlternatively, you can use the old ./setup_and_build.sh, or you can use the
-n dry-run flag when running make to inspect commands to run separately.
Linux dependencies are installed along with make install_dependencies.
Alternatively, install the following manually: bison, dwarves, flex, libelf-dev,
libssl-dev, and llvm.
To set up an Ubuntu checkpoint with a custom kernel, you first need to create a base Ubuntu checkpoint. You can generate such a checkpoint using:
./simics -batch-mode scripts/install_ubuntu.simics \
save_checkpoint=/opt/simics/checkpoints/ubuntu-20.4_latest.ckptThe scripts by default expect to find the checkpoint at /opt/simics/checkpoints/ubuntu-20.4_latest.ckpt,
override CKPT_KERNEL_BASE in config-local.mk to use different path.
|
Note
|
The script automation relies on external services and may fail in
different environments. In this case, you may need to manually install the
checkpoint by manually following the steps in scripts/install_ubuntu.simics.
|
For troubleshooting, it is recommended to run with graphical console enabled; the initial boot will be in the VGA view, after which GRUB will configure the serial console and continue installation via that.
This assumes you have an initial kernel checkpoint, if not, see Create initial Ubuntu checkpoint. Once you have an initial ubuntu checkpoint (default: CKPT_KERNEL_BASE=/opt/simics/checkpoints/ubuntu-20.4_latest.ckpt), you can generate a checkpoint with a custom kernel using:
# Set CKPT_KERNEL_BASE in config-local.mk if needed, (default: checkpoints/cc_kernel.ckpt)
make ckpt-cc_kernelThis will create a new checkpoint at checkpoints/cc_kernel.ckpt.GIT_SHA and create/update a symlink to it from checkpoints/cc_kernel.ckpt.
You can also manually update the kernel of an existing checkpoint with the command:
./simics scripts/update_ubuntu_kernel.simics \
checkpoint=/path/to/ubuntu_checkpoint_to_build_on.ckpt \
upload_llvm=TRUE \
upload_glibc=TRUE \
kernel=linux/linux.tar.gz \
save_checkpoint=checkpoints/new_checkpoint_name.ckptThis assumes linux.tar.gz contains a pre-made linux source within the src
directory of the package. Note, that this is not necessary if using the C3
packaged configuration and kernel, in which case you can just run make
ckpt—cc_kernel as instructed elsewhere.
# To clean only Simics modules:
make clean
# To clean most build artifacts (e.g., for glibc, llvm, and linux), run:
make mrproperNeither of the commands will remove checkpoints. To do so, delete the checkpoints folder(s) manually. Note that checkpoints by default are incremental and depend on the originating checkpoint.
To build LLDB with C3-coredump support run:
make make_llvm-lldbThis builds llvm/llvm_install/bin/lldb. It has C3 support for loading
variables (and pointers) in C3 cryptographic address format, and will
automatically read in C3 keys from a coredump file generated by a C3-enabled
process. This requires that the coredump was generate by the C3-enabled custom
kernel (e.g., using a cc_kernel checkpoint).
The following commands create doxygen documentation for malloc, crypto and
modules under doc/doxygen, you can browse the docs by starting from
doc/doxygen/html/index.html. The documentation is auto-generated from inline
annotations in comments in the source code files themselves.
make documentationThe c3_model has initial functional support for integrity checking based on ICV value bound to a virtual address and corresponding expected cryptographic address. By default, only writes are integrity checked, but exact behavior can be configured via Simics model attributes:
c30_0->integrity_break_on_write_mismatch = (default: FALSE)
c30_0->integrity_fault_on_write_mismatch = (default: TRUE)
c30_0->integrity_break_on_read_mismatch = (default: FALSE)
c30_0->integrity_fault_on_read_mismatch = (default: FALSE)
c30_0->integrity_warn_on_read_mismatch = (default: FALSE)Kernel support for ICV handling is not implemented, hence the virtual-address based ICVs of one C3-enabled application may pollute the virtual addresses of other C3-enabled applications running within the same simulator instance. At present, the ICVs can be programmatically reset in the Simics shell or script by:
c30_0->integrity_icv_reset = TRUEAlternatively, there is a helper function:, cc_trigger_icv_map_reset, that can
be used to trigger ICV reset from within the target (e.g., a program running on
the simulation). NOTE: this functionality is exposed without access-control
only for testing purposes.
The code listing below assume default paths, but these can be configured by
creating a config-local.mk file or setting the corresponding environment
variables. Some relevant variables and their default values are:
# The path to Simics installation bin directory
SIMICS_BIN=$/opt/simics/simics-6/simics-latest/bin
# Path for a base no-kernel checkpoint that is used as the starting point when
# generating checkpoints without custom kernel. If not set, new checkpoints will
# be created from scratch. Default value is ignored if path is not found.
CKPT_NOKERNEL_BASE=/opt/simics/checkpoints/glibc_latest.ckpt
# Path for checkpoint with glibc, this is generated with `make ckpt-cc_glibc`,
# and will be a symlink to tagged checkpoint folder.
CKPT_GLIBC=checkpoints/cc_glibc.ckpt
Similar to CKPT_GLIBC, but includes llvm.
CKPT_LLVM=checkpoints/cc_llvm.ckpt./simics [simics_args] [run_arg1=val1 run_arg2=val2 ...]Useful simics_args (optional):
-no-win |
run simics with GUI windows hidden (can be displayed on demand) |
-batch-mode |
run in batch mode (will exit with 0 on success or non-zero on error) |
Most run scripts are based on the generic template scripts/runworkload_common.simics It supports the following run-time arguments (see default values in the script):
checkpoint |
Specifies the checkpoint. |
system |
Sets the top level module. For QSP use "board" (default), for TGL: "tgl" |
compiler |
Overrides the compiler for the workload (unless using custom build command). Default: g++ |
gcc_flags |
Additional compiler flags |
model |
Selects the model to run the workload with. Default: cc, or c3. (Note: lim_disp configures the LIM model to perform data displacement instead of shifting.) |
enable_integrity=1 |
Enable C3 integrity checking |
run_args |
Specifies additional workload run arguments |
env_vars |
Overrides environment arguments for the workload run command |
build_cmd |
Overrides the default build command |
run_cmd |
Overrides the default run command |
pre_run_fixup |
Additional bash commands to execute inside Simics before running the workload |
debug |
Set to 1 to enable Simics module debug printfs |
download_bin_path |
If defined, the workload binary and the compiled libc will be downloaded to the specified host directory. |
disable_meta_check |
LIM-only setting. If set to 1, tags and bounds will not be evaluated |
break_on_exception |
LIM-only setting. If set to 1, will stop simulation on exceptions (excl. Page Fault) |
magic |
Set to 1 to enable magic breakpoint |
mem_profiler |
Set to 1 to enable memory profiler |
run_cycles=N |
If set, the workload will run for N billion cycles and pause. Default: and stop after completion |
cache |
Set to 1 to enable caching model |
exit |
Set to 1 to exit on completion (code 0) or error (non-zero code) |
Additional run-time arguments for specific scripts: spec/scripts/generic.simics:
spec |
Specifies the SPEC workload name. |
spec_size |
Specifies the SPEC experiment size (test/ref) |
Useful examples:
./simics scripts/runworkload_common.simics \
src_path=microbenchmarks \
src_file=hello.cpp \
workload_name=a.out \
model=cc \
checkpoint=checkpoints/cc_llvm.ckpt \
gcc_flags="-g -gdwarf -Werror -ldl -lm -lpthread -pthread -Iinclude"The tests are currently configured to use LLVM’s libunwind, consequently you must use an LLVM checkpoint to run unit tests (e.g., checkpoints/cc_llvm.ckpt as described above).
Run all tests (12 jobs in parallel):
pytest -n12 -v python_tests --checkpoint checkpoints/cc_llvm.ckpt [--model native|cc|lim]Run only spec tests:
pytest -n12 -v python_tests --checkpoint checkpoints/cc_llvm.ckpt[--model native|cc|lim]
# all spec workloads:
pytest -n12 -v python_tests/test_spec.py --checkpoint checkpoints/cc_llvm.ckpt
# specific workloads:
pytest -n12 -v python_tests/test_spec.py --checkpoint checkpoints/cc_llvm.ckpt --spec workload_name [--spec workload_name ...]Run only unit tests:
pytest -n12 -v python_tests/test_unit.py --checkpoint checkpoints/cc_llvm.ckptCommon options:
--checkpoint PATH |
Set the checkpoint to use |
--model |
Run tests only with the specified model. Can specify multiple models by appending '--model <model_name>' for each model. The 'c3-integrity' model will run on the c3 but configure it to use integrity |
-d |
Load-balance tests. Shortcut for '--dist=load' |
--have-kernel |
Run C3-kernel dependent tests |
You can also run individual unit tests:
./simics unit_tests/runtest_common.simics \
src_path=unit_tests/common \
src_file=gtest_hello.cpp \
workload_name=a.out \
model=cc \
checkpoint=checkpoints/cc_llvm.ckpt \
include_folders="unit_tests/include/unit_tests" \
gcc_flags="-g -gdwarf -Werror -ldl -lm -lpthread -pthread -Iinclude -Iunit_tests/include -DC3_MODEL=cc"To enforce coding guidelines locally, you can install pre-commit hooks that run tests on the staged changes before allowing a commit to pass. To enable default commit hooks, you can run
# To install, run:
make pre-commit-install
# To uninstall, run:
make pre-commit-uninstallThe pre-commit hook will apply whitespace fixes automatically to your working
tree, you can inspect those changes using git diff, and then add them to your
commit. The pre-commit hook also runs clang-format and cpplint checks. You
may need to manually address issues reported by cpplint. Cosmetic code style
changes can be automatically applied by running clang-format -i <filename>, or
without the -i flag to only inspect changes without applying them.
In some cases you may not be able to fix all changes, or you may need to commit
files that intentionally violate code style rules. To do so, you can always run
git commit --no-verify. However, when possible, avoid disregarding issues.