Structured Logging

Structured Logging

logme writes normal text logs by default. That remains the primary mode because it is fast, readable and can still be parsed unambiguously by logme tools.

Structured logging adds another output representation for the same log events. It does not change how messages are produced by application code; it changes how accepted Context records are rendered.

---

Runtime structured output

Runtime structured output is generated by the logme formatter itself. In the current code path, JSON is produced by Context::ApplyJson() and XML by Context::ApplyXml().

Each log message is written as an independent structured record. logme does not keep a global JSON array or a global XML document open while the application is running.

For JSON, this means JSONL-style output: one object per record.

{"timestamp":"2026-05-04 12:00:00.000","level":"INFO","message":"server started"}
{"timestamp":"2026-05-04 12:00:01.000","level":"ERROR","message":"request failed"}

For XML, this means a stream of independent <event> elements.

<event><timestamp>2026-05-04 12:00:00.000</timestamp><level>INFO</level><message>server started</message></event>
<event><timestamp>2026-05-04 12:00:01.000</timestamp><level>ERROR</level><message>request failed</message></event>

This is intentional. Runtime logging must be able to append records independently. It should not have to rewrite closing brackets, hold a complete document in memory or coordinate a global document footer between backends.

---

Fields in structured records

The fields are controlled by OutputFlags and by the data available in the current Context.

The built-in structured field set includes:

• timestamp
• level
• process_id
• thread_id
• channel
• subsystem
• file
• line
• method
• message
• duration

A field appears only when the corresponding output flag and context data allow it. For example, subsystem is emitted only when subsystem output is enabled and the message has a non-empty SID. Source location fields depend on the location/detail flag. Duration is produced for procedure-style logging where duration text is available.

The field names can also be customized through the structured field name API and through JSON configuration using the structured_fields object.

---

Runtime output is not a finalized document

Runtime JSON output is a sequence of JSON objects, not a single JSON array. Runtime XML output is a sequence of <event> elements, not a complete XML document with one root element.

This distinction is important when opening logs in tools that expect one complete JSON or XML document. Such tools usually need a finalization step.

---

Post-processing with logmefmt

logmefmt converts existing log files into another representation. This is the right tool when you need a finalized JSON or XML document for analysis, export or integration with software that cannot consume streaming records.

For example:

logmefmt --input text --output json --finalize --in app.log --out app.json
logmefmt --input text --output xml --finalize --in app.log --out app.xml

With finalization enabled, the output can be wrapped into a complete document instead of remaining a stream of independent records.

---

When to use runtime structured output

Use runtime structured output when another process consumes records as they are written: a collector, monitoring agent, pipe, or custom backend pipeline. The consumer can process one JSON object or one XML <event> at a time.

Use logmefmt when the log already exists or when the next tool expects one complete JSON/XML document.

---

Why logme uses independent records

Independent records keep runtime logging simple and safe:

• no previous bytes need to be rewritten
• no complete log document has to stay in memory
• a crash does not leave a half-maintained global structure in the logging code
• each backend can append records independently

The trade-off is explicit: runtime structured output is stream-oriented, while finalized documents are produced as a post-processing step.