CCTLib is a library that ubiquitously collects calling contexts for every instruction executed by an application, and attributes costs to the specific data objects those instructions touch. It runs on top of Intel Pin as a runtime library and turns any Pin tool into a context-aware profiler with a few lines of glue code.
Doxygen API reference: https://cctlib.github.io/cctlib/
- Linux x86_64
- g++ 10 or newer
- Intel Pin 4.x (tested against pin-external-4.3-99850-gce5652921)
Install these before building:
g++>= 10 (Pin 4.x recommends this)autoconf,automake,libtoolunzip,tar,curl(for the bundledbuild.shdependency setup)
# 1. Download Pin 4.x and set PIN_ROOT.
curl -fSL -o pin.tar.gz \
https://software.intel.com/sites/landingpage/pintool/downloads/pin-external-4.3-99850-gce5652921-gcc-linux.tar.gz
tar xzf pin.tar.gz
export PIN_ROOT=$PWD/pin-external-4.3-99850-gce5652921-gcc-linux
# 2. Build CCTLib.
git clone https://github.com/CCTLib/cctlib.git
cd cctlib
./build.shbuild.sh unpacks externals/{libelf-0.8.9,sparsehash-2.0.3-95e5e93,json-v3.7.3},
configures, and runs make -j followed by make check. It produces:
src/obj-intel64/libcctlib.a— shadow-memory-based data-centric attribution.src/obj-intel64/libcctlib_tree_based.a— balanced-binary-tree-based attribution.src/obj-intel64/libcctlib_metric.a— per-IPNode custom metric slots.
Refer to Chabbi, Liu & Mellor-Crummey (CGO 2014) for the algorithmic details.
Actively maintained. Recent work covers a clean C++-exception unwind
implementation (Itanium ABI landing-pad re-anchor), a direct-self-
recursion collapse for the CCT, a shape-inspection pintool
(clients/cct_shape_check.cpp) plus 22 gtest-driven shape victims,
and continuous integration for clang-format + clang-tidy.
There is also a successor project, DrCCTProf, built on top of DynamoRIO with broader platform coverage. CCTLib remains the reference implementation on Pin and stays maintained for that use case.
Full API documentation is generated by Doxygen (make docs, or the
auto-published copy linked above). The load-bearing entry points:
// Initialize CCTLib. Call once from your Pin tool's main() after PIN_Init.
int PinCCTLibInit(IsInterestingInsFptr isInterestingIns,
FILE* logFile,
CCTLibInstrumentInsCallback userCallback,
VOID* userCallbackArg,
BOOL doDataCentric = false);
// From the client's InstrumentInsCallback, at run time, get the
// context handle for the current slot.
ContextHandle_t GetContextHandle(const THREADID id, const uint32_t slot);
// Get the data-object handle for an effective address (requires
// doDataCentric = true at init time).
DataHandle_t GetDataObjectHandle(VOID* address, THREADID threadId);
// Print the full calling context to CCTLib's log file.
// Caller must hold PIN_LockClient().
VOID PrintFullCallingContext(ContextHandle_t ctxtHandle);
// Materialize the full calling context as a vector<Context>.
// Caller must hold PIN_LockClient().
VOID GetFullCallingContext(const ContextHandle_t curCtxtHndle,
std::vector<Context>& contextVec);
// Canonicalize a handle to its enclosing TraceNode -- two handles that
// live in the same trace return the same value. Useful for shape
// inspectors that need TraceNode-identity comparisons.
ContextHandle_t GetTraceStartHandle(ContextHandle_t ctxtHandle);
// True iff two handles map to the same source line (possibly different
// instructions on that line).
bool IsSameSourceLine(ContextHandle_t ctxt1, ContextHandle_t ctxt2);
// Postmortem: rebuild CCTs from previously-serialized files.
// Never call together with PinCCTLibInit().
int PinCCTLibInitForPostmortemAnalysis(FILE* logFile,
std::string serializedFilesDirectory);
// Write CCT metadata to disk for later postmortem analysis.
void SerializeMetadata(std::string directoryForSerializationFiles = "");
// Emit all CCTs as GraphViz DOT files.
void DottifyAllCCTs();// Gathering calling context on every instruction:
#include "cctlib.H"
using namespace PinCCTLib;
VOID SimpleCCTQuery(THREADID id, uint32_t slot) {
ContextHandle_t h = GetContextHandle(id, slot);
// ... do something with h ...
}
VOID InstrumentInsCallback(INS ins, VOID* v, uint32_t slot) {
INS_InsertPredicatedCall(ins, IPOINT_BEFORE, (AFUNPTR)SimpleCCTQuery,
IARG_THREAD_ID, IARG_UINT32, slot, IARG_END);
}
int main(int argc, char* argv[]) {
PIN_Init(argc, argv);
PinCCTLibInit(INTERESTING_INS_ALL, gTraceFile, InstrumentInsCallback, 0);
PIN_StartProgram();
return 0;
}INTERESTING_INS_ALL is a predefined callback that returns true
for every instruction. Alternatives:
INTERESTING_INS_MEMORY_ACCESS— only load/store instructions.INTERESTING_INS_NONE— only function and call-site granularity.- A custom
BOOL (*)(INS)predicate for finer selection.
Full working code lives in tests/cct_client.cpp and the many
clients/*.cpp tools.
tests/ — minimal examples:
| File | What it does |
|---|---|
cct_client.cpp |
Gather calling context on every instruction. |
cct_client_mem_only.cpp |
Gather calling context on every memory access. |
cct_data_centric_client.cpp |
Data-centric attribution via shadow memory. |
cct_data_centric_client_tree_based.cpp |
Data-centric attribution via balanced BST. |
clients/ — production-scale tools:
| File | What it does |
|---|---|
deadspy_client.cpp |
DeadSpy: pinpoint dead writes with full context. |
redspy_client.cpp |
RedSpy: value-locality-based redundancy detection. |
loadspy_client.cpp |
LoadSpy: redundant loads. |
zerospy_client.cpp |
ZeroSpy: redundant zeros in loads. |
footprint_client.cpp |
Memory footprint by call context. |
cache_client.cpp |
Simulated cache miss attribution. |
ins_reuse_client.cpp |
Instruction reuse distance. |
omp_datarace_client.cpp |
OpenMP data-race detection. |
runtime_value_numbering_client.cpp |
Runtime value numbering for redundant computation. |
cct_metric_client.cpp |
Per-IPNode custom-metric skeleton. |
cctlib_reader.cpp |
Read serialized CCT for postmortem analysis. |
cct_shape_check.cpp |
CCT-shape assertion tool for the gtest suites. |
CCTLib carries an extensive gtest suite in tests/gtest/:
make -C tests/gtest testIncludes:
- Client integration tests (deadspy, redspy, loadspy, footprint, ins_reuse, rvn, omp_race, cache_client, cctlib_integration).
- Exception + signal integration — 39 tests across 13 victims covering all C++ exception patterns (simple throw, deep unwind, rethrow, catch-all, dtor cleanup, stress loop, polymorphic dispatch, uncaught path, ctor throw, catch-and-resume, setjmp/ longjmp, sigsegv recover).
- Exception shape suite — 14 victims +
cct_shape_check.cppassert on structural CCT invariants (marker attribution under landing-pad re-anchor, direct-self-recursion collapse under nested throw+rethrow, no throw-machinery ancestors under catch/try markers). - Recursion shape suite — 8 victims exercising direct vs indirect self-recursion, mixed direct+indirect chains, stripped binaries, and recursion terminating via throw / longjmp.
Set CCTLIB_EXPENSIVE_SHAPE=1 to include the four stress victims
(high-iteration throw loops).
The tree uses .clang-format (LLVM base, 4-space indent, attached
braces, no reflow) and .clang-tidy (bugprone-, performance-,
readability-, modernize- families with a documented exclude list
for invasive checks).
make format— reformat all C/C++ sources in place.make format-check— fail if any file would change (used by CI).make lint— runclang-tidyon the full source set; requires acompile_commands.jsonproduced bybear -- make.
Opt-in per clone:
make install-hooksThis sets core.hooksPath to .githooks/, so every git commit
runs clang-format-15 on the staged C/C++ files and auto-re-stages
what it changes. See .githooks/README.md.
Bypass with git commit --no-verify. Uninstall with
git config --unset core.hooksPath.
.github/workflows/lint.yml runs on
every push to master and every PR:
clang-format --dry-run --Werroron all C/C++ sources.clang-tidyon the full source set (with Pin installed via the cached Intel tarball).
.github/workflows/docs.yml builds the
Doxygen output on every push to master and publishes it to the
gh-pages branch.
MAX_IPNODES— maximum number of call-path handles (default1 << 32). Virtual address space is reserved eagerly; physical memory is committed on demand. Lower it with-DMAX_IPNODES=<num>if you run into VM limits.MAX_STRING_POOL_NODES— maximum number of distinct variable names in data-centric analysis (default1 << 30). Same eager-reserve / lazy-commit story; lower with-DMAX_STRING_POOL_NODES=<num>.
How do I build for CCTLib development?
Pass --enable-develop to ./configure (or edit build.sh to add
the flag before it runs ./configure).
How do I cite CCTLib? Chabbi, Liu, Mellor-Crummey. Call Paths for Pin Tools. CGO 2014. DOI: 10.1145/2544137.2544164.
- Milind Chabbi, Xu Liu, John Mellor-Crummey. Call Paths for Pin Tools. CGO 2014. doi:10.1145/2544137.2544164
- Milind Chabbi, Wim Lavrijsen, Wibe de Jong, Koushik Sen, John Mellor-Crummey, Costin Iancu. Barrier elision for production parallel programs. PPoPP 2015. doi:10.1145/2688500.2688502
- Milind Chabbi, John Mellor-Crummey. DeadSpy: a tool to pinpoint program inefficiencies. CGO 2012. doi:10.1145/2259016.2259033
- Shasha Wen, Xu Liu, Milind Chabbi. Runtime Value Numbering: A Profiling Technique to Pinpoint Redundant Computations. PACT 2015.
- Shasha Wen, Milind Chabbi, Xu Liu. REDSPY: Exploring Value Locality in Software. ASPLOS 2017. doi:10.1145/3037697.3037729
- Pengfei Su, Shasha Wen, Hailong Yang, Milind Chabbi, Xu Liu. Redundant Loads: A Software Inefficiency Indicator. ICSE 2019. doi:10.1109/ICSE.2019.00103