Skip to content

v1.3.0

Choose a tag to compare

@janbjorge janbjorge released this 27 Jul 12:47
200d2ed

v1.3.0

Features:

  • Web dashboard (pgq web): built-in FastAPI + htmx UI for queue insights and job management: backlog age, throughput, duration percentiles, failures with tracebacks, worker liveness, schedules, table health, plus requeue/cancel actions. Live updates ride the existing LISTEN/NOTIFY channel. Install with pgqueuer[web,asyncpg].
  • First-class Postgres schema support: --schema / PGQUEUER_SCHEMA installs PgQueuer's tables, triggers, and functions into a named schema instead of public.
  • Automatic log aggregation: QueueManager rolls pgqueuer_log into pgqueuer_statistics on a timer (log_aggregation_interval, default 30s) instead of requiring a cron job. Coordinated across workers via advisory lock; skipped when a worker did no work since the last tick.
  • pgq sql command group: emit install/uninstall/upgrade/durability/autovac SQL to stdout with no DB connection. Pipe into psql or capture as a migration file for Flyway, sqitch, Alembic.
  • Unified connection handling: ConnectionSettings plus shared connect_asyncpg/connect_psycopg factories bring consistent PGQUEUER_* env var support to the CLI, MCP server, and web dashboard.

Performance:

  • In-memory driver dequeue replaced a full-table scan/sort with per-entrypoint priority heaps. Drain throughput roughly 4x (373 → 1610 jobs/s at 50k jobs).

Fixes:

  • Several dequeue/shutdown edge cases: batch jobs no longer stranded on shutdown, lifecycle tasks torn down on run() failure, dequeue timeout now probes eligible queued work before deferred ETA, batch cap respects the tightest entrypoint concurrency limit, dequeue picks capped to remaining per-entrypoint/worker slots, and drain no longer shuts down on a stale queued-work count (which could strand a retried job).
  • Schema upgrade rebuilds the statistics aggregation index.

Deprecations:

  • --dry-run deprecated in favor of pgq sql install|uninstall|upgrade|durability.
  • add_prefix() deprecated; prefix is now a DBSettings field reading PGQUEUER_PREFIX directly.

Documentation:

  • New guides for the web dashboard and log-aggregation gating/backlog caveat; connection docs clarify the bring-your-own-connection model; general docs cleanup.

What's Changed

  • fix(qm): confirm empty queue with uncached count before drain shutdown by @janbjorge in #692
  • fix(qm): dispatch remaining batch jobs on shutdown instead of stranding picked rows by @janbjorge in #693
  • fix(qm): tear down lifecycle tasks on run() failure by @janbjorge in #694
  • fix(qm): probe eligible queued work before deferred eta in dequeue timeoutmeout by @janbjorge in #695
  • fix(queries): cap dequeue picks to remaining per-entrypoint and worker slots by @janbjorge in #696
  • fix(qm): drop batch cap to tightest entrypoint concurrency limit by @janbjorge in #697
  • build: check formatting in make lint by @janbjorge in #698
  • perf(inmemory): replace full-scan dequeue with priority heaps by @janbjorge in #699
  • docs: clean up dash-heavy phrasing across docs by @janbjorge in #700
  • feat: unify connection config across CLI, MCP, and pool factories by @janbjorge in #702
  • feat(web): add built-in web dashboard for queue insights and management by @janbjorge in #691
  • refactor(settings): make prefix a DBSettings field, deprecate add_prefix by @janbjorge in #704
  • feat(schema): add first-class Postgres schema support by @janbjorge in #703
  • feat(cli): add pgq sql command group, deprecate --dry-run by @janbjorge in #705
  • test: bump testcontainers to >=4.14.2 to drop deprecation warning by @janbjorge in #706
  • feat(stats): periodic log->statistics aggregation without cron by @janbjorge in #707
  • docs(agents): ban module-level globals outright by @janbjorge in #708
  • fix(schema): rebuild aggregation index on upgrade by @janbjorge in #670
  • fix(qm): skip log aggregation tick when worker did no work by @janbjorge in #713
  • docs(reliability): document log-aggregation gating and backlog caveat by @janbjorge in #714
  • fix(ci): drop pytest-xdist workers from 4 to 2 by @janbjorge in #715

Full Changelog: v1.2.0...v1.3.0