Skip to content

logging updates (SYN-8012)#4686

Merged
vEpiphyte merged 69 commits intomasterfrom
visi-logging
Apr 13, 2026
Merged

logging updates (SYN-8012)#4686
vEpiphyte merged 69 commits intomasterfrom
visi-logging

Conversation

@invisig0th
Copy link
Copy Markdown
Contributor

@invisig0th invisig0th commented Jan 20, 2026

Summary

Consolidate and modernize the Synapse logging infrastructure. Logging functionality previously scattered across synapse/common.py and synapse/lib/structlog.py is replaced by a dedicated synapse/lib/logging.py module with async-safe log pumping, structured output, and real-time log streaming APIs.

  • New synapse/lib/logging.py module with JsonFormatter, TextFormatter, async StreamHandler, and log lifecycle management
  • Async log pump task ensures logging never blocks the asyncio event loop
  • Structured log output consistently formatted with system metadata that cannot be overwritten by user-provided information
  • New logs() and watch() admin APIs on all Synapse services for retrieving recent log entries and streaming live log events
  • Cell.getLogExtra() is now synchronous (no longer requires await)
  • User context automatically included in log entries via s_scope
  • Rich exception serialization including tracebacks, chained exceptions, and exception groups
  • Removed synapse/lib/structlog.py and logging helpers from synapse/common.py
  • Updated devops documentation with structured logging format details and examples
  • Also includes $lib.lift.tagsByPref() Storm API (SYN-10503)

Key changes

Area Description
synapse/lib/logging.py New module (381 lines): setup(), shutdown(), reset(), logs(), watch(), getLogExtra(), setLogInfo(), excinfo(), formatters, async stream handler
synapse/lib/cell.py getLogExtra() now sync, new logs()/watch() admin APIs, logging setup via s_logging.setup(), service name registered globally
synapse/lib/view.py Storm execution logging uses s_scope.enter() for user context, getLogExtra() for structured params
synapse/common.py Removed setlogging(), normLogLevel(), _getLogConfFromEnv() and related imports
synapse/lib/structlog.py Deleted (replaced by synapse/lib/logging.py)
synapse/tests/utils.py Refactored LoggerStream with async expect() method and queue-based log capture
docs/synapse/devopsguide.rst Expanded structured logging documentation with field descriptions and JSON examples
60+ test files Updated logging assertions to use new APIs and test utilities

Environment variables

SYN_LOG_LEVEL, SYN_LOG_STRUCT, and SYN_LOG_DATEFORMAT continue to work as before.

Comment thread synapse/lib/logging.py Outdated
Comment thread synapse/tests/utils.py Outdated
Comment thread synapse/lib/logging.py Outdated
Comment thread synapse/lib/logging.py Outdated
@vEpiphyte vEpiphyte requested a review from MichaelSquires April 9, 2026 13:56
MichaelSquires
MichaelSquires previously approved these changes Apr 9, 2026
@invisig0th invisig0th requested a review from vEpiphyte April 9, 2026 14:10
Comment thread synapse/lib/processpool.py
@vEpiphyte vEpiphyte modified the milestones: v2.238.0, v2.239.0 Apr 9, 2026
MichaelSquires
MichaelSquires previously approved these changes Apr 10, 2026
vEpiphyte
vEpiphyte previously approved these changes Apr 10, 2026
Comment thread synapse/lib/agenda.py Outdated
Comment thread synapse/datamodel.py Outdated
Comment thread synapse/datamodel.py Outdated
Comment thread synapse/datamodel.py Outdated
Co-authored-by: OCBender <181250370+OCBender@users.noreply.github.com>
@vEpiphyte vEpiphyte dismissed stale reviews from MichaelSquires and themself via 57c8b42 April 10, 2026 20:37
@vEpiphyte vEpiphyte merged commit c04d2ef into master Apr 13, 2026
6 checks passed
@vEpiphyte vEpiphyte deleted the visi-logging branch April 13, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants