Skip to content

Trace Points

Eduard Mishkurov edited this page May 16, 2026 · 1 revision

Trace Points

Trace points are dormant logging points that can stay in production code and be activated later through runtime control. They are useful for code paths that are important to observe during an incident, but too noisy to log all the time.

A trace point always increments its hit counter when execution reaches it. It writes a normal log record only after that trace point has been enabled through the control server. This makes trace points different from ordinary disabled log statements: you can see whether a code path was reached even before you enable the log output.


When to use trace points

Use trace points for diagnostic locations that you may want to inspect in a live process:

  • rarely failing branches
  • request-processing stages
  • retry loops and state-machine transitions
  • suspicious paths where the exact frequency matters
  • temporary production diagnostics that should remain silent until enabled

For function entry/exit profiling, use Function tracing. Trace points solve a different problem: they mark arbitrary places in code and can be enabled, disabled, listed, and reset by location pattern.


Macros

The printf-style trace point macros mirror the normal logging levels:

LogmeD_TPt("debug trace: %s", ARGS1(value));
LogmeI_TPt("info trace: %s", ARGS1(value));
LogmeW_TPt("warning trace: %s", ARGS1(value));
LogmeE_TPt("error trace: %s", ARGS1(value));
LogmeC_TPt("critical trace: %s", ARGS1(value));

LogmeTPt(...) is the default info-level shortcut:

LogmeTPt("ProcessItem entered: %s", ARGS1(value));

Trace point macros accept the same leading channel/subsystem-style arguments as normal logme macros. For example, a trace point can write through an explicit channel when it is enabled:

LogmeTPt(channel, "loop iteration: %s", ARGS1(i));

When std::format support is enabled, the corresponding fLogme*_TPt macros are available:

fLogmeI_TPt("state {}", state);
fLogmeTPt("worker {} iteration {}", workerId, i);

What happens at runtime

Each trace point is represented by a static object created at the call site. The trace point records:

  • source file
  • function name
  • line number
  • hit counter
  • enabled/disabled state

The key used by runtime commands has this form:

file:function:line

The initial state is disabled. The counter still increases while disabled, but no log message is emitted until the trace point is enabled.

When a trace point is enabled and the logger condition allows output, logme emits a normal log record at the level selected by the macro. The emitted record uses the trace point call site as its location.


Runtime commands

Trace points are controlled through the control server. With logmectl, commands look like this:

logmectl -p 7791 trace list '*:*:*'
logmectl -p 7791 trace enable '*WorkerLoop*'
logmectl -p 7791 trace reset '*WorkerLoop*'
logmectl -p 7791 trace disable '*WorkerLoop*'

Supported command forms:

trace [list|stat|stats] [pattern]
trace enable <pattern>
trace disable <pattern>
trace reset [pattern]

list, stat, and stats currently return the same trace point table: state, location key, and hit count.

Example text output:

off C:\src\app\worker.cpp:WorkerLoop:642 hits=1179
on  C:\src\app\worker.cpp:HandleRequest:388 hits=42

Patterns

Patterns are matched against the lower-case file:function:line key. The matcher supports:

  • * for any sequence of characters
  • ? for one character

Useful examples:

*:*:*                     all trace points
*WorkerLoop*              any trace point whose key contains WorkerLoop
*worker.cpp:*:*           trace points in worker.cpp
*:HandleRequest:*         a specific function name
*:HandleRequest:388       a specific call site

Pattern matching is case-insensitive.


Counters and reset

Counters are useful even while trace points are disabled. A typical workflow is:

logmectl -p 7791 trace stat '*WorkerLoop*'
logmectl -p 7791 trace enable '*WorkerLoop*'
# reproduce the problem
logmectl -p 7791 trace stat '*WorkerLoop*'
logmectl -p 7791 trace reset '*WorkerLoop*'
logmectl -p 7791 trace disable '*WorkerLoop*'

trace reset without a pattern resets all registered trace point counters.


logmeweb

logmeweb provides a Trace points tab for the same control surface. It is more convenient when an application has many trace points because the web UI can show the points as a table, filter them, sort by hits or state, and apply enable/disable/reset actions to selected or filtered points.


Build behavior

Trace point macros are active only when logging macros are active. In inactive builds, the trace point macros expand away like the rest of the logging macro layer.

The fLogme*_TPt family requires std::format support in the same way as the normal fLogme* macros.


Example

The repository includes a TracePoints example. It runs a workload with disabled trace points, prints counters, enables all trace points, runs the workload again, and then disables them.

See also Runtime Control, Control Server, and logmectl.

Clone this wiki locally