Skip to content

feat(logging): central taskito logger with AM/PM timestamps#155

Merged
pratyush618 merged 2 commits into
masterfrom
feat/centralized-logging
May 8, 2026
Merged

feat(logging): central taskito logger with AM/PM timestamps#155
pratyush618 merged 2 commits into
masterfrom
feat/centralized-logging

Conversation

@pratyush618
Copy link
Copy Markdown
Collaborator

Summary

One canonical entry point — `taskito.configure_logging()` — that owns the `taskito` logger and bridges Rust `log::*` records into the same sink. Used by every CLI subcommand and `Queue.run_worker` so all 'app runs' paths share one format and one destination.

  • New `taskito.log_config` module with a single public `configure(level=, stream=)` that's idempotent, thread-safe, and respects `TASKITO_LOG_LEVEL`.
  • Default format kept as `[%(asctime)s] %(levelname)s %(message)s`; timestamp now `%Y-%m-%d %I:%M:%S %p` so log lines carry AM/PM (per request).
  • `pyo3-log` bridges every `log::*` record from `taskito-core`, `taskito-python`, `taskito-async` and `taskito-workflows` into Python's `logging` module under matching logger names; `configure()` attaches a managed handler to all five.
  • `PyQueue::new` now releases the GIL during storage init. Without this, r2d2 pool builders that emit `log::error!("database is locked")` from worker threads deadlock against the main thread holding the GIL during retries. The pyo3-log bridge would have surfaced this latent issue immediately; releasing the GIL is the correct production fix regardless.
  • `pyo3-log` is activated lazily via a `_init_rust_logging` PyO3 function called from `configure()` rather than from module init, so cold imports don't trip the same flush path.
  • The previously buried `logging.basicConfig` in `mixins/lifecycle.py` is replaced by `configure_logging()`. `taskito.configure_logging` is exported from the top-level package for embedded users (Django/FastAPI) who don't go through the CLI.

Sample output

```
[2026-05-08 04:08:18 PM] INFO smoke message from python
[2026-05-08 04:08:18 PM] INFO smoke from sub-logger
```

Test plan

  • `uv run python -m pytest tests/ --ignore=tests/worker/test_prefork.py` — 506 passed, 8 skipped
  • `uv run python -m pytest tests/worker/test_prefork.py` — 8 passed, 1 skipped
  • `uv run python -m pytest tests/observability/test_log_config.py` — 8 new tests passed
  • `uv run ruff check`, `uv run mypy` — clean
  • `cargo check --workspace`, `--features postgres`, `--features redis` — clean
  • `cargo test --workspace` — 89 passed
  • Manual smoke: `taskito.configure_logging(level="DEBUG")` produces AM/PM-timestamped lines on `taskito` and `taskito.` child loggers.

pyo3-log forwards Rust log::* records into Python's logging module.
Activated lazily via _init_rust_logging() so cold imports don't deadlock
against r2d2 pool builders, and PyQueue::new releases the GIL during
storage init so connection-pool worker threads can flush log records.
One configure() call attaches a single managed handler to the taskito
Python logger and the four taskito_* loggers populated by pyo3-log. The
default format keeps today's [ts] LEVEL message shape; the timestamp
gains AM/PM via %I:%M:%S %p. Idempotent and thread-safe.

Wired into the CLI entry and run_worker so every 'app runs' path uses
the same sink. configure_logging is exported from the package for
embedded users (Django/FastAPI) to call themselves.
@pratyush618 pratyush618 merged commit 595a928 into master May 8, 2026
19 checks passed
@pratyush618 pratyush618 deleted the feat/centralized-logging branch May 8, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant