Skip to content

Logger calls eagerly construct large payload objects on publish/failure #27

@iifawzi

Description

@iifawzi

Summary

Multiple log sites pass the entire message body as part of the structured log fields. Even when the configured log level discards the line, the payload reference is captured and most structured-log transports will JSON.stringify it. For high-throughput publishers / heavy retry traffic, this is real GC pressure.

Where

  • src/core/RunMQ.ts:76-80publish() logs { topic, correlationId, message } at info level
  • src/core/consumer/processors/RunMQFailureLoggerProcessor.ts:13-16 — failure logs include message: message.message
  • src/core/consumer/processors/RunMQRetriesCheckerProcessor.ts:38-45 — max-retries log also includes message: message.message

Failure mode

For an MB-sized payload at 1k msg/s, this is GB/s of allocations the GC has to clean up — independent of whether the line is actually written.

Proposed fix

  • Drop message from the publish info log (correlationId is enough for tracing).
  • For failure logs, log a truncated preview or just the size:
    this.logger.error('Message processing failed', {
        correlationId: message.correlationId,
        size: typeof message.message === 'string' ? message.message.length : undefined,
    }, e instanceof Error ? e.stack : undefined);
  • Optional: lazy-evaluate via a () => ({ ... }) thunk if the logger supports it.

Acceptance criteria

  • No log site captures the full message payload by default.
  • Optional opt-in (e.g., logFullMessage: true in processor config) for debugging.

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance / hot-path issue

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions