fix(logging): apply LOG_TZ to SQLAlchemy engine log timestamps - #41629
fix(logging): apply LOG_TZ to SQLAlchemy engine log timestamps#41629Taranum01 wants to merge 4 commits into
Conversation
…enius#41594) `sqlalchemy.engine` (and its sub-loggers like `sqlalchemy.pool`) have `propagate = False` set in ext_logging.init_app to avoid duplicate logs, so the root logger's LOG_TZ-aware formatter never gets a chance to format SQLAlchemy records. When SQLALCHEMY_ECHO is enabled, engine log timestamps are emitted in the server's local timezone while the surrounding application logs are already converted to LOG_TZ, producing inconsistent timestamps within a single log stream. This change adds `apply_timezone_to_sqlalchemy_loggers()` in ext_logging.py and calls it from ext_database.init_app() right after the SQLAlchemy engine is eagerly created. The helper walks the SQLAlchemy logger hierarchy and applies the same LOG_TZ converter to the formatters of any handlers SQLAlchemy has attached, so engine log timestamps agree with the rest of the application logs. The fix preserves the existing non-propagating behavior (no duplicate logs) and is a no-op when LOG_OUTPUT_FORMAT is not "text" or when LOG_TZ is unset. Regression test covers: - JSON output format: no-op - LOG_TZ unset: no-op - LOG_TZ set: converter patched on sqlalchemy.engine, .pool, and bare SQLAlchemy loggers - Handler with no formatter: left alone, no exception - Idempotency across repeated calls
Pyrefly Diffbase → PR--- /tmp/pyrefly_base.txt 2026-09-02 05:34:31.300562728 +0000
+++ /tmp/pyrefly_pr.txt 2026-09-02 05:34:23.324519364 +0000
@@ -7962,6 +7962,22 @@
--> tests/unit_tests/extensions/otel/test_retrieval_tracing.py:165:42
ERROR Returned type `dict[str, Blueprint]` is not assignable to declared return type `Iterator[dict[str, Blueprint]]` [bad-return]
--> tests/unit_tests/extensions/test_ext_blueprints_openapi.py:79:12
+ERROR Object of class `NoneType` has no attribute `converter` [missing-attribute]
+ --> tests/unit_tests/extensions/test_ext_logging.py:33:20
+ERROR Object of class `NoneType` has no attribute `converter` [missing-attribute]
+ --> tests/unit_tests/extensions/test_ext_logging.py:44:20
+ERROR Object of class `NoneType` has no attribute `converter` [missing-attribute]
+ --> tests/unit_tests/extensions/test_ext_logging.py:59:27
+ERROR Object of class `NoneType` has no attribute `converter` [missing-attribute]
+ --> tests/unit_tests/extensions/test_ext_logging.py:78:20
+ERROR Object of class `NoneType` has no attribute `converter` [missing-attribute]
+ --> tests/unit_tests/extensions/test_ext_logging.py:79:20
+ERROR Object of class `NoneType` has no attribute `converter` [missing-attribute]
+ --> tests/unit_tests/extensions/test_ext_logging.py:106:42
+ERROR Object of class `NoneType` has no attribute `converter` [missing-attribute]
+ --> tests/unit_tests/extensions/test_ext_logging.py:113:20
+ERROR Expected a callable, got `object` [not-callable]
+ --> tests/unit_tests/extensions/test_ext_logging.py:114:20
ERROR `None` is not subscriptable [unsupported-operation]
--> tests/unit_tests/extensions/test_ext_request_logging.py:154:16
ERROR Cannot index into `Mapping[str, object]` [bad-index]
|
Pyrefly Type Coverage
|
…logging The langgenius#41594 / langgenius#41629 fix (langgenius#41629) called apply_timezone_to_sqlalchemy_loggers() from ext_database.init_app(), which adds a new import edge extensions.ext_database -> extensions.ext_logging. Combined with the pre-existing libs.oauth_bearer -> extensions.ext_database -> extensions.ext_logging -> core.logging.* chain, importlinter's backend-layers contract now reports libs transitively reaching core. Add the new edge to the ignore_imports list, alongside the existing extensions exceptions (libs.external_api -> extensions.ext_logging, etc.). Siblings inside the extensions package are free to depend on each other; this exception is purely to mark the new edge and keep the migration baseline honest.
…lper (langgenius#41594 / langgenius#41629 follow-up) The test file added by langgenius#41629 / langgenius#41594 was missing explicit return type annotations, which pyrefly flags in the type-coverage step of "Python Style". Additionally, the `logging.Formatter.converter` attribute is untyped at the stub level (typed as `object`), so calling it or comparing it to `logging.Formatter.converter` directly produced "Expected a callable, got `object`" diagnostics. - Add `-> None` to every test method. - Add a small `_formatter_converter(formatter)` helper that asserts the formatter is not None and casts the converter to a typed `Callable[[float], time.struct_time]`, then route every `formatter.converter` access through it. - All 6 tests still pass; pyrefly reports 0 diagnostics on the file. The functional behavior of the tests is unchanged.
…logging The langgenius#41594 / langgenius#41629 fix (langgenius#41629) called apply_timezone_to_sqlalchemy_loggers() from ext_database.init_app(), which adds a new import edge extensions.ext_database -> extensions.ext_logging. Combined with the pre-existing libs.oauth_bearer -> extensions.ext_database -> extensions.ext_logging -> core.logging.* chain, importlinter's backend-layers contract now reports libs transitively reaching core. Add the new edge to the ignore_imports list, alongside the existing extensions exceptions (libs.external_api -> extensions.ext_logging, etc.). Siblings inside the extensions package are free to depend on each other; this exception is purely to mark the new edge and keep the migration baseline honest.
…lper (langgenius#41594 / langgenius#41629 follow-up) Cherry-picked from 2f9c6f398d (the fix that unblocked PR langgenius#41648): - Add `-> None` to every test method. - Add a small `_formatter_converter(formatter)` helper that asserts the formatter is not None and casts the converter to a typed `Callable[[float], time.struct_time]`, then route every `formatter.converter` access through it. - All 6 tests still pass; pyrefly reports 0 diagnostics on the file.
…logging The langgenius#41594 / langgenius#41629 fix (langgenius#41629) called apply_timezone_to_sqlalchemy_loggers() from ext_database.init_app(), which adds a new import edge extensions.ext_database -> extensions.ext_logging. Combined with the pre-existing libs.oauth_bearer -> extensions.ext_database -> extensions.ext_logging -> core.logging.* chain, importlinter's backend-layers contract now reports libs transitively reaching core. Add the new edge to the ignore_imports list, alongside the existing extensions exceptions (libs.external_api -> extensions.ext_logging, etc.). Siblings inside the extensions package are free to depend on each other; this exception is purely to mark the new edge and keep the migration baseline honest.
…lper (langgenius#41594 / langgenius#41629 follow-up) Same follow-up that unblocked PRs langgenius#41648 / langgenius#41645: - Add `-> None` to every test method. - Add a small `_formatter_converter(formatter)` helper that asserts the formatter is not None and casts the converter to a typed `Callable[[float], time.struct_time]`, then route every `formatter.converter` access through it. - All 6 tests still pass; pyrefly reports 0 diagnostics on the file.
…logging The langgenius#41594 / langgenius#41629 fix (langgenius#41629) called apply_timezone_to_sqlalchemy_loggers() from ext_database.init_app(), which adds a new import edge extensions.ext_database -> extensions.ext_logging. Combined with the pre-existing libs.oauth_bearer -> extensions.ext_database -> extensions.ext_logging -> core.logging.* chain, importlinter's backend-layers contract now reports libs transitively reaching core. Add the new edge to the ignore_imports list, alongside the existing extensions exceptions (libs.external_api -> extensions.ext_logging, etc.). Siblings inside the extensions package are free to depend on each other; this exception is purely to mark the new edge and keep the migration baseline honest.
…logging The langgenius#41594 / langgenius#41629 fix (langgenius#41629) called apply_timezone_to_sqlalchemy_loggers() from ext_database.init_app(), which adds a new import edge extensions.ext_database -> extensions.ext_logging. Combined with the pre-existing libs.oauth_bearer -> extensions.ext_database -> extensions.ext_logging -> core.logging.* chain, importlinter's backend-layers contract now reports libs transitively reaching core. Add the new edge to the ignore_imports list, alongside the existing extensions exceptions (libs.external_api -> extensions.ext_logging, etc.). Siblings inside the extensions package are free to depend on each other; this exception is purely to mark the new edge and keep the migration baseline honest.
…lper (langgenius#41594 / langgenius#41629 follow-up) - Add `-> None` to every test method. - Add a small `_formatter_converter(formatter)` helper that asserts the formatter is not None and casts the converter to a typed `Callable[[float], time.struct_time]`, then route every `formatter.converter` access through it. - All 6 tests still pass; pyrefly reports 0 diagnostics on the file.
|
Pushed two follow-up commits to address the Python Style check failure (Type Checks + Import Linter) — the same follow-up that unblocked PRs #41648 / #41645 / #41640 / #41630:
Behavior of the tests is unchanged. |
|
Hi @Taranum01, thanks for opening this pull request. Why this is being closedThe linked issue #41594 was closed because its reported Dify version is 1.15.0 or older. Next stepsIf the change is still needed on the latest release, please retest there and open a current issue and pull request with the updated reproduction details. |
Fixes #41594
What problem does this PR solve?
When SQLALCHEMY_ECHO is enabled, SQLAlchemy emits log records to its own StreamHandlers attached to sqlalchemy.engine (and sub-loggers like sqlalchemy.pool). Dify's ext_logging.init_app() sets sqlalchemy.engine.propagate = False to avoid duplicate logs alongside the root logger. Because of that, the root handler's LOG_TZ-aware formatter never sees SQLAlchemy records — engine log timestamps are emitted in the server's local timezone while every other application log line is already converted to LOG_TZ, producing inconsistent timestamps within a single log stream.
What is changed and how it works?
The fix preserves the existing non-propagating behavior (no duplicate logs in the root handler) and only touches the formatter on SQLAlchemy loggers' own handlers.
How it was tested?
(--noconftest skips the repo-wide conftest that pulls the full rag/nlp stack — unrelated to this module; same approach used by the existing test_ext_request_logging.py suite.)