Collapse

Collapse Logging

Collapse logging is a call-site feature for reducing repeated log noise without hiding the first occurrence of a problem.

It is implemented by the LogmeX_Collapse and LogmeX_CollapseIgnore macro families in Logme.h, CollapseContextCache in Context.h, and Context::ApplyCollapse() in Context.cpp.

Macro families

The printf-style macros are:

Level	Exact-message collapse	Regex-normalized collapse
Debug	LogmeD_Collapse(limit, ...)	LogmeD_CollapseIgnore(ignoreRegex, limit, ...)
Info	LogmeI_Collapse(limit, ...)	LogmeI_CollapseIgnore(ignoreRegex, limit, ...)
Warning	LogmeW_Collapse(limit, ...)	LogmeW_CollapseIgnore(ignoreRegex, limit, ...)
Error	LogmeE_Collapse(limit, ...)	LogmeE_CollapseIgnore(ignoreRegex, limit, ...)
Critical	LogmeC_Collapse(limit, ...)	LogmeC_CollapseIgnore(ignoreRegex, limit, ...)

X in the macro name is the normal log level letter: D, I, W, E, or C.

Collapse macros accept the same optional log arguments as normal LogmeX macros after the collapse-specific parameters: channel, subsystem, override, format string, and format arguments.

LogmeX_Collapse

LogmeX_Collapse(limit, ...) uses the final formatted message text as the repeat key.

LogmeW_Collapse(3, "network error: connection refused");

Behavior:

1. The first message is written normally.
2. Consecutive messages with the same formatted text are not written immediately.
3. When the repeat counter reaches limit, logme writes a collapsed summary using the original message text.
4. When a different message appears, the stored key is replaced and the new message is written normally.

The summary text is produced by the logger formatting path and has the form:

repeated N times: original message text

LogmeX_CollapseIgnore

LogmeX_CollapseIgnore(ignoreRegex, limit, ...) is intended for repeated messages that contain volatile fields such as request ids, correlation ids, timestamps, sequence numbers, counters, or addresses.

Before comparing the current message with the previous one, logme removes all substrings matched by ignoreRegex from the formatted message text. The regular expression is used only to build the comparison key. The original formatted message is still what gets printed.

LogmeE_CollapseIgnore(
  "request_id=[0-9]+"
  , 10
  , "request_id=%llu backend unavailable"
  , requestId
);

This can collapse messages such as:

request_id=101 backend unavailable
request_id=102 backend unavailable
request_id=103 backend unavailable

because the comparison key becomes the same after removing the request id.

Important details

• Collapse state is stored in a static CollapseContextCache at the macro call site. Different source lines have independent collapse state.
• The comparison key is based on formatted text, not on the format string and raw arguments.
• Only consecutive repeats are collapsed. A different message resets the call-site repeat state.
• limit == 0 disables collapse for that call site and the message is written normally.
• The collapse cache is protected by its own lock, so the repeat counter and last key are shared safely between threads using the same call site.
• LogmeX_CollapseIgnore compiles the regular expression in the static call-site cache. If the regular expression is invalid, ignore matching is disabled and the macro behaves like exact-message collapse.

When to use it

Use collapse when a message is important, but identical or nearly identical repeats would flood the log.

Good candidates:

• retry loops that report the same failure repeatedly
• temporary backend outages
• polling loops
• noisy network or protocol errors
• repeated validation failures where only ids or timestamps differ

Do not use collapse for messages where every individual occurrence must be preserved, such as audit records, security decisions, billing events, or one-record-per-request traces.

Difference from _Once and _Every

_Once, _Every, and _Collapse solve different noise problems:

Need	Recommended macro
Write only the first occurrence	LogmeX_Once(...)
Write at most once per time interval	LogmeX_Every(ms, ...)
Write the first occurrence and aggregate consecutive repeats	LogmeX_Collapse(limit, ...)
Aggregate repeats after ignoring volatile fields	LogmeX_CollapseIgnore(ignoreRegex, limit, ...)

Collapse is not a backend-side duplicate filter. It is an explicit call-site decision, so the repeated message can be suppressed before backend delivery.