Getting started

Traces is a distribution containing trace readers and the trace dumper

Dependencies

• clang 3.1 (other versions of clang will not work) clang 3.1 binaries
• Python 2.7 or above
• python-ctypeslib (for the interactive reader)
• gccxml (for python-ctypeslib)
• python-urwid (for the interactive reader)
• tmpfs mounted at /dev/shm (enabled by default in most distributions)

Building traces

Traces has the following buildable components:

• The trace_user.a library. trace_user.a allocates and initializes the shared memory area region required for the traces to work.
• trace_instrumentor.so - A clang plugin responsible for instrumenting C/C++ code
• simple_trace_reader - A simple viewer for trace files written in C

Build instructions:

cd TRACES_INSTALL_DIR
make all

Compiling your code with traces enabled

Traces uses clang to instrument the code at function entry/leave as well as expand explicit trace points. The provided gcc wrapper ccwrap.py wraps gcc invocation with the trace instrumentation phase. The script instruments the code prior to passing it to gcc for final compilation. The .o file or executable produced by ccwrap.py will contain the instrumented code To use ccwrap.py instead of gcc, replace compiler references in your build system with 'ccwrap.py $REAL_CC', where, $REAL_CC is the name of the original compiler. ld invocations need to be wrapped as well. Wrap the ld invocations with ldwrap.py.

The provided demo application contains a Makefile showing how to use ccwrap.py and ldwrap.py

Quick start

1. After your application has been compiled with traces enabled, you can quickly start examining its traces: Run trace_dumper -o (to produce verbose output showing function entry/leave, use the '-d' parameter.
2. trace_dumper will show trace messages as they are generated by the traced application. Run the traced application. The trace_dumper should start printing traces immediately.

Normally, trace_dumper is used to write records directly to file rather than parse and display them online. To run trace_dumper in write mode, invoke it with the -w[filename] argument, optionally passing a destination filename. If no filename is specified, trace files are written to /mnt/logs and are rotated once the trace file reaches 1 GB.

Using explicit tracing in the code

Traces allows a developer to write explicit trace logs in addition to the automatic function traces. Traces supports the following function-like interfaces for writing logs: DEBUG(), INFO(), WARN(), ERROR(), FATAL() and REPR(). These functions are expanded by Traces to form a complete writing of a log entry. Each function accepts a list of parameters that will be presented in the log. Examples:

#include <trace_user.h>
int a = 5;
const char *some_str = "a_string";
DEBUG("A constant string and the value of a", a, "and now a variable string", some_str);

// Dump the raw byte-content of a
WARN("The raw value of a is", HEX_REPR(&a, sizeof(a)));

During runtime, only variable data is written to the trace record. The constant strings appear only in the static metadata. Traces uses a clang plugin to perform the expansion and resolve the types of each parameter passed to the record call.

Reading traces

There are two ways to view the contents of a trace file: simple_trace_reader and the interactive reader.

• non-interactive reader: trace_reader/simple_trace_reader -d $filename
• interactive-reader: python interactive_reader/ui.py $filename

The non-interactive displays only non-function and non-debug records. This behavior can be controlled through the command line.

The interactive reader supports searching and filtering according to the following criterias. See the help (press 'h' in the main display) for exact usage examples:

• severity
• Function name
• Function parameter value and name
• Function return value
• Struct field value and name
• Search by tid, pid, process name,
• Search by type name referenced in trace record
• Search by time and time range
• Boolean operations (not, or, and)