Skip to content

v0.5.0

Choose a tag to compare

@tokibito tokibito released this 05 Sep 08:00
· 32 commits to main since this release
82fc8c4

Workers recover from being killed, PostgreSQL gets a broker that needs no extra service, and the worker reports what happened in a form a job scheduler and a log aggregator can read.

Existing projects need no changes, with one exception: get_auth_handler() (singular), deprecated in 0.4, is now gone — see Removed below.

Added

  • Recovery of tasks left in RUNNING (manage.py requeue_stale_database_tasks --older-than 15m). A worker killed outright — SIGKILL, the OOM killer, a node failure — never writes a result, so the task it held stays RUNNING and no other worker picks it up. The new command puts those tasks back in READY. --older-than is required and has to be longer than the longest task takes to run; --max-attempts (default 3) marks a task FAILED rather than requeueing one that kills its own worker forever, and --mark-failed is there for tasks that are not safe to run twice. Also available as django_database_task.requeue_stale_tasks() and as a Django admin action.
  • PostgreSQL LISTEN/NOTIFY broker (django_database_task.postgres.PostgresNotifyDatabaseBackend, pip install django-database-task[postgres]). Notifies a channel of the database the tasks are already stored in, so a waiting worker starts the task in milliseconds instead of on the next poll. No queue, no credentials, no extra service. The notification is sent with pg_notify() inside the transaction that inserted the task, so a worker is never told about a task it cannot yet see or one whose transaction was rolled back.
  • Exit codes for job schedulers: run_database_tasks --empty-exit-code and --failed-exit-code. Both default to 0, so nothing changes for an existing cron line or Kubernetes Job until they are set.
  • Structured log fields. Log records now carry task_id, task_path, queue_name, priority, backend_alias, worker_id, plus status, duration_ms and error_class where they apply, instead of only interpolating them into the message. Task started, Worker started and Worker finished records are new.

Changed

  • A broker's enqueue() method is now called notify(). The old name read as if it enqueued the task, which is the backend's job: a broker is only told about a task the database already holds.

Deprecated

  • TaskBroker.enqueue(). A broker that overrides it is still called, with a DeprecationWarning, and stops being called in 0.6. Rename it to notify(). Bundled brokers and the BROKER option are unaffected.

Removed

  • get_auth_handler() (singular), deprecated in 0.4 and removed here as announced. A backend that overrides it is no longer called, and unless it also overrides get_auth_handlers() or sets AUTH_HANDLERS its endpoints are left unauthenticated. Override get_auth_handlers() instead. The AUTH_HANDLERS and CLOUD_TASKS_* options and the bundled backends are unaffected.

See CHANGELOG.md for the full entry, including the documentation added in this release.

Full Changelog: v0.4.0...v0.5.0