Skip to content

Comments

Add wire Debug Level for LLM Request/Response Logging#2020

Merged
ksylvan merged 3 commits intodanielmiessler:mainfrom
ksylvan:kayvan/add-llm-comms-logging
Feb 22, 2026
Merged

Add wire Debug Level for LLM Request/Response Logging#2020
ksylvan merged 3 commits intodanielmiessler:mainfrom
ksylvan:kayvan/add-llm-comms-logging

Conversation

@ksylvan
Copy link
Collaborator

@ksylvan ksylvan commented Feb 22, 2026

Add wire Debug Level for LLM Request/Response Logging

Closes #2018

Summary

This PR introduces a new debug level, wire (level 4), that logs the full HTTP-style request/response exchanges between Fabric and LLM backends. This is the most verbose logging tier and sits above the existing trace level. The change spans the logging package, the core chatter logic, the CLI flag definition, and all supported i18n locale files.


Files Changed

File Change Type Description
internal/log/log.go Modified Added Wire constant, updated LevelFromInt, added GetLevel() accessor
internal/log/log_test.go Added Unit tests for LevelFromInt covering all levels including the new Wire level
internal/core/chatter.go Modified Injected Wire-level debug logging at three key points in the LLM communication lifecycle
internal/cli/flags.go Modified Updated --debug flag description to document level 4
internal/i18n/locales/*.json Modified Updated set_debug_level description string across all 10 locale files

Code Changes

internal/log/log.go — New Wire Level and GetLevel() Accessor

// Wire logs full request/response exchanges with model backends.
Wire

The LevelFromInt function was updated to handle the new level precisely:

case i == 3:
    return Trace
case i >= 4:
    return Wire

Previously i >= 3 returned Trace, meaning there was no way to distinguish between 3 and any higher integer. This is now fixed with an exact match for 3 and a catch-all >= 4 for Wire.

A new thread-safe GetLevel() function was added:

func GetLevel() Level {
    mu.RLock()
    defer mu.RUnlock()
    return level
}

This avoids callers needing to reach into unexported state and ensures proper read-lock semantics.


internal/core/chatter.go — Wire-Level Logging at LLM Boundaries

Three logging points were added to capture the full conversation at the wire level:

  1. Outbound messages before sending to LLM:
if debuglog.GetLevel() >= debuglog.Wire {
    debuglog.Debug(debuglog.Wire, "FABRIC->LLM request messages (%d)\n", len(vendorMessages))
    for i, msg := range vendorMessages {
        debuglog.Debug(debuglog.Wire, "FABRIC->LLM [%d] role=%s content=%q\n", i, msg.Role, msg.Content)
        if len(msg.MultiContent) > 0 {
            debuglog.Debug(debuglog.Wire, "FABRIC->LLM [%d] parts=%d\n", i, len(msg.MultiContent))
        }
    }
}
  1. Each streaming response update:
if debuglog.GetLevel() >= debuglog.Wire {
    debuglog.Debug(debuglog.Wire, "LLM->FABRIC stream update type=%s content=%q\n", update.Type, update.Content)
    if update.Usage != nil {
        debuglog.Debug(debuglog.Wire, "LLM->FABRIC stream usage input=%d output=%d total=%d\n", ...)
    }
}
  1. Non-streaming (synchronous) response:
if debuglog.GetLevel() >= debuglog.Wire {
    debuglog.Debug(debuglog.Wire, "LLM->FABRIC response content=%q\n", message)
}

internal/log/log_test.go — New Test File

A table-driven unit test covers all levels including edge cases:

{in: -1, want: Off},
{in: 0, want: Off},
{in: 4, want: Wire},
{in: 9, want: Wire},  // any value >= 4 maps to Wire

Reason for Changes

Debugging LLM integration issues (e.g., malformed prompts, unexpected model responses, token usage discrepancies) previously required adding temporary ad-hoc logging or attaching a network proxy. The wire level formalises this as a first-class debug capability, making it straightforward to capture the exact data being exchanged with model backends without modifying application code.


Impact of Changes

  • No functional change for users running at debug levels 0–3. The >= 4 guard ensures zero overhead unless wire logging is explicitly enabled.
  • The LevelFromInt fix for the i >= 3 edge case is a subtle but correct behaviour change: previously, passing --debug 4 (or any value above 3) would have silently returned Trace. Now it correctly returns Wire. Any existing automation or scripts that passed --debug 4 expecting Trace behaviour will now receive Wire verbosity instead — this is the intended and more correct behaviour.
  • The GetLevel() accessor is additive and non-breaking.
  • Log output at wire level may contain sensitive data (full prompt content, model responses), which is appropriate for a local debug tool but worth noting for environments where logs are forwarded to external systems.

Test Plan

  • Run go test ./internal/log/... — the new TestLevelFromInt test should pass covering all level mappings.
  • Run Fabric with --debug 4 and confirm FABRIC->LLM and LLM->FABRIC lines appear on stderr.
  • Run Fabric with --debug 3 and confirm no wire-level output is produced.
  • Test both streaming and non-streaming model paths (streaming via UpdateChan, synchronous via direct Send).
  • Verify --help output and locale-specific help strings reflect 4=wire.

Additional Notes

  • The wire term is borrowed from network-layer logging conventions (e.g., "wire format"), making the intent self-documenting for engineers familiar with distributed systems debugging.
  • The 4=wire label in the i18n locale files is intentionally left untranslated (wire) across all languages, consistent with how technical debug terms are typically handled in developer tooling.
  • Multi-content (multimodal) messages are partially logged — only the part count is emitted, not the raw binary content of each part. This is a reasonable tradeoff to avoid flooding logs with base64-encoded image data.

…logging

## CHANGES

- Add `Wire` log level constant to debug level enum
- Expose `GetLevel()` function for safe concurrent level reads
- Log outbound message roles and content at wire debug level
- Log inbound stream updates and token usage at wire level
- Log non-streaming LLM responses at wire debug level
- Update `--debug` flag description to include new level 4
- Update `set_debug_level` locale strings across all 10 languages
## CHANGES

- Add debug level 4 (`wire`) to `--debug` flag options
- Update zsh completion to include new `4` debug level value
- Update bash completion to include `4` in `compgen -W` word list
- Update fish completion description and arguments with level 4
@ksylvan ksylvan merged commit 4147712 into danielmiessler:main Feb 22, 2026
1 check passed
@ksylvan ksylvan deleted the kayvan/add-llm-comms-logging branch February 22, 2026 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Question]: Is there a way to log the http requests/responses from fabric to OpenAI

1 participant