celery-manager 0.1.0a4

Fourth alpha of celery-manager: a redesigned dashboard, pausing whole schedules (indefinitely or until a set time), Insights, workers and queue controls, and versioned schema migrations.

pip install --pre -U celery-manager

Upgrading: this release adds schema migrations 0002–0005. By default they run automatically at startup, and that needs a database user with DDL rights. To run them as a deploy step instead, set auto_migrate=False and run celery-manager migrate --db-url .... Databases from 0.1.0a1–0.1.0a3 are adopted automatically and keep their data.

Not yet verified against real infrastructure: workers, rate limits and queue controls (tested against stand-ins plus a throwaway Redis); the workflow tree from a live chord; and the MySQL migration lock.

Added

• Pause and resume a whole schedule. POST /api/schedules/{name}/pause
  and POST /api/schedules/{name}/resume (also
  celery_manager.control.schedules.pause_schedule / resume_schedule).
  While a schedule is paused, every firing is recorded as a skipped run and
  the task body does not run — the same mechanism as an occurrence skip.
  Both endpoints are idempotent, return 404 for an unknown schedule, and
  write pause_schedule / resume_schedule audit rows. GET /api/schedules
  items gain paused, paused_at and paused_by; each occurrence from
  GET /api/schedules/{name}/occurrences gains paused. The frontend API
  client gains pauseSchedule() / resumeSchedule(). Limitations:

  • only @track-decorated tasks are affected — celery-manager does not
    own beat, so beat keeps firing and an undecorated task still runs;
  • resume waits for the next regular beat firing. It doesn't run the task
    now, doesn't reset beat's interval clock, and doesn't catch up firings
    missed while paused (they were recorded as skipped).

  The pause survives restarts: the startup sync from beat_schedule leaves
  it alone.

• Pause/Resume in the dashboard. Each schedule on the Schedules page has
  a Pause button (with a confirm step) or a Resume button and a Paused badge.
  A paused schedule's page shows who paused it and when, and marks upcoming
  runs as "Will be skipped".

• Task list filters in the URL, saved views, copy link. Status tab,
  search, group-by, date range and history-mode page are reflected in the
  hash-route query string (#/?status=failed&q=smtp), restored on load and
  on back/forward. Discrete choices push a history entry; typing in search
  replaces. Copy link copies the view URL. Views saves named filter
  sets per browser in localStorage (rename/delete supported; no backend).
  Filter state moved into useTaskFilters.

• Export task runs. GET /api/tasks/export?format=csv|json streams every
  run matching the list filters, capped at the new
  CeleryManagerConfig.export_max_rows (default 50000) with an
  X-Export-Truncated header, CSV formula-injection escaping, a timestamped
  filename, and an export_tasks audit row. The task list gains an Export
  menu.

• Insights: trends, per-task pages and grouped failures. New Insights page
  with 1h/24h/7d/30d ranges, KPIs, SVG charts (runs by status, p50/p95
  runtime, each with a text summary and table fallback) and a Failures tab
  that groups failed runs by a normalised exception signature, with open,
  show-runs and retry-all-in-group actions. A task page at
  /task-types/:taskName (linked from the task detail title and the Insights
  top-task lists) shows the same trends for one task plus recent runs and top
  errors. Backed by GET /api/analytics/overview, /api/analytics/failures
  and /api/analytics/tasks/{task_name}, computed at query time (no schema
  change; scans are capped, see Insights).

• Start a task from the dashboard. A Start task button on the Task
  runs page opens a sheet with a searchable task picker, args/kwargs JSON
  editors prefilled from the task's signature with live validation, and an
  optional queue and countdown/ETA; submitting opens the new run. New
  GET /api/tasks/registered lists registered tasks (excluding celery.*)
  with best-effort signature, queue and docstring summary.
  POST /api/tasks/start now accepts optional queue, countdown and eta,
  and seeds a queued run row so the run's page exists straight away.

• Retry with changed arguments. POST /api/tasks/{id}/retry takes an
  optional {"args", "kwargs"} body (no body behaves as before); retry
  lineage is recorded as usual and the audit row notes the edit and the
  arguments sent. The task detail page gains Retry with changes.

• Workflow tree for chains, groups and chords. Runs now record Celery's
  parent_id, root_id and group id (schema revision 0003; captured from
  task events and from @track's request). GET /api/tasks/{id}/workflow
  returns the whole tree (capped at 500 nodes), the task detail payload gains
  parent_id/root_id/group_id, and the task detail page shows a
  Workflow card: an indented, collapsible tree with a status summary.

• Workers page. A new Workers page lists every worker that answers the
  broker: status, PID, uptime, pool, processed count, active/reserved/
  scheduled counts (active tasks link to their run), and consumed queues. It
  refreshes every 10 seconds while the tab is visible. From each worker card
  you can shut the worker down (with a confirm step), add or remove a pool
  process, and start or stop consuming a queue. Backed by GET /api/workers
  and POST /api/workers/{worker}/shutdown, .../pool/grow,
  .../pool/shrink, .../queues/add and .../queues/cancel (also
  celery_manager.control.workers). Every control is audited. If the broker
  can't be reached the endpoints answer 503 with a readable message, not a
  500. New config field inspect_timeout_seconds (default 1.0) caps how
  long each broadcast waits for replies. See Workers.

• Runtime rate limits. POST /api/tasks/rate-limit changes a task's rate
  limit on running workers (all, or a destination list) through
  control.rate_limit. The rate format (10/s, 10/m, 100/h, or "" to
  remove the limit) is checked before anything is sent, and the change is
  audited. GET /api/workers/rate-limits lists registered tasks with the
  limit declared in code. On the Workers page a Rate limits table lets you
  edit a limit inline and shows which workers confirmed it. These limits are
  not persisted: a worker that restarts goes back to the limit declared
  in code.

• Queue depth. GET /api/queues lists every queue celery-manager knows
  of (default, task_queues, task_routes, queues seen on runs, and
  queues live workers consume). For each one it gives the broker's message
  count and consumer count, read with a passive queue_declare, next to the
  number of runs recorded as queued there. It always answers 200: an
  unreachable or slow broker (3s deadline) or a memory:// broker gives
  depth: null and a reason. A new Queues page in the dashboard shows
  this, refreshes every 5 seconds (paused while the tab is hidden) and draws
  a sparkline of depth over the session. See Queues.

• Purge a queue. POST /api/queues/{name}/purge removes the ready
  messages from one queue and returns how many. If it removed any, runs
  recorded as queued in that queue without an ETA are marked revoked,
  since no event will ever arrive for them. Audited as purge with both
  counts. In the dashboard you confirm by typing the queue name.

• Bulk cancel. POST /api/tasks/cancel-bulk revokes a list of
  task_ids, or every queued/started/retried run matching the task
  list's filters. It is capped at 1,000 (above that it's refused, not cut
  short), has a dry_run count and an opt-in terminate, and writes one
  cancel_bulk audit row. The task list gains Cancel N selected on the
  Queued, Started and Retried tabs, and Cancel all matching… with the
  count shown before you confirm.

• Pause a schedule until a set time. POST /api/schedules/{name}/pause
  takes an optional {"until"} (future, UTC; past is a 422) and the
  schedule resumes by itself afterwards, with no background job: readers
  clear the stale pause lazily and write one automatic resume_schedule
  audit row. Re-pausing with a new until updates it. paused_until joins
  the schedule payloads, and each occurrence's paused is now per occurrence.
  The dashboard's pause button becomes a menu (indefinitely, 1 hour, until
  tomorrow 9:00, custom time). Schema revision 0004.

Changed

• Dashboard redesign. shadcn/ui-style look (zinc theme, Geist fonts) in
  light and dark. The sidebar collapses to icons (Ctrl/Cmd+B, remembered per
  browser) and slides in over the page on phones. The task list gains a run
  strip (one bar per run, coloured by status), status tabs and a search and
  filter bar. Task detail puts the failure and logs first with execution
  details alongside. Tables stack on narrow screens.
• Schema migrations with Alembic. celery-manager's tables are now
  versioned, and the migrations ship inside the package (no alembic.ini
  needed). By default init_db() upgrades the database at startup, under a
  lock so the API, workers and listener can start together. Databases from
  0.1.0a1–0.1.0a3 are adopted automatically and keep their data. The new
  auto_migrate config field (default True) turns this off for teams that
  migrate as a deploy step with the new celery-manager migrate --db-url ...
  command (--check reports status and exits 1 if an upgrade ...

celery-manager 0.1.0a3

Third alpha of celery-manager: stable interval-schedule skips and race-free task-run writes (verified against real PostgreSQL and MySQL).

pip install --pre -U celery-manager

Fixed

• Interval-schedule occurrences (run_every / seconds / timedelta) are
  now anchored to the schedule's last observed run instead of "now + interval".
  Skips on interval schedules display stably across page loads and actually
  fire against beat's real run times.
• Race-free task_runs writes. The events listener, the @track
  decorator, report_progress() and the retry button used to check-then-insert
  the same row. They now share one race-safe get-or-create (native upsert on
  SQLite/PostgreSQL/MySQL, SAVEPOINT fallback elsewhere), so:
  • retry no longer returns a 500 or loses its lineage when the worker's row
    lands first;
  • the listener no longer leaves worker/queue empty when it loses the race;
  • late or out-of-order events can no longer downgrade a run's status;
  • the producer's hostname from task-sent is no longer recorded as the worker;
  • event arguments are stored as JSON, so retries from listener-created rows
    decode correctly.

Added

• Documentation site: https://pythondeveloper6.github.io/celery-manager/.
• Dashboard screenshots in the README and docs, and an "Interval schedules"
  section in the dashboard guide.
• The race tests run against PostgreSQL in CI (CM_TEST_DB_URL).

Changed

• README and docs no longer say tasks can be started from the UI; starting is
  API-only (POST /api/tasks/start).

celery-manager 0.1.0a2

First PyPI release.

pip install --pre celery-manager

No code changes since 0.1.0a1 apart from the version; README and package metadata made PyPI-ready. Full notes: CHANGELOG.

celery-manager 0.1.0a1 (alpha 1)

First alpha of celery-manager: a dashboard and REST API for monitoring and controlling Celery tasks, with task history, live progress, retries, schedules and pause/resume.
This is an early pre-release; expect rough edges and read the Breaking and Known limitations sections below.

Install

pip install "git+https://github.com/Pythondeveloper6/celery-manager.git@v0.1.0a1"

Or download the wheel below and pip install celery_manager-0.1.0a1-py3-none-any.whl.

Changelog

The initial release. Before this alpha, celery-manager had never been published, so everything
below is new rather than changed — with a few exceptions, called out first
because anyone who pinned a git SHA during development will hit them.

Breaking

• GET /api/tasks now returns an envelope, not a bare array. It used to
  return a JSON array of runs, hard-capped at 50 rows, with no offset — so
  older history was unreachable and a client could not tell "50 results" from
  "the first 50 of 1,284". It now returns:

  {"items": [ ... ], "total": 1284, "limit": 50, "offset": 0}

  Callers must change res.json() to res.json().items. The items also
  gained runtime_seconds and the four progress_* fields. There is no
  compatibility shim and no version negotiation: the change was taken
  deliberately while the package was unreleased and its only consumers were
  the bundled dashboard and docs/rest-api.md. See
  the REST API reference.

• The session cookie is now cm_session, and create_app() refuses a
  default secret_key. Both are security fixes; see Security
  below for what changed and what you need to do.

Added

Task history, search and filtering

• GET /api/tasks gained q, date_from, date_to, limit and offset
  alongside the existing status and task_name. limit is clamped into
  1..500 (control.tasks.MAX_PAGE_SIZE) rather than rejected, and the
  clamped values are echoed back in the envelope.
• q is a case-insensitive substring search OR-ed across task_name,
  task_id, args and kwargs. Because args/kwargs are stored as
  serialized JSON text, this searches that text — there is no way to express
  kwargs.user == "alice". LIKE wildcards in the search term are escaped, so
  q=100% finds a literal 100% instead of matching every row.
• Both date bounds apply to started_at and are inclusive, which means a
  queued run (no started_at) is excluded by either bound, and date_to is
  an instant rather than a calendar day.
• control.count_task_runs() — the total behind "showing 50 of 1,284". It
  shares one filter builder with list_task_runs(), so a total can never be
  a count of a different query than the page it labels.

Live progress

• celery_manager.report_progress(current, total=None, message=None), called
  from inside a task body. Writes progress_current, progress_total,
  progress_message and progress_updated_at on the run's task_runs row
  while the task is still in flight; all four are served raw by both task
  endpoints so the client decides how to render them.
• Database writes are throttled to one per 0.5 s per run
  (progress.MIN_WRITE_INTERVAL_SECONDS), so it is safe in a tight loop. The
  completing call (current >= total) bypasses the throttle, so a finished
  task never sits at 99%.
• Omitting total means open-ended progress: a count and a message, no
  percentage. A total of 0, negative, or non-int-able is treated the same
  way — celery-manager never invents a denominator.
• Never raises into the calling task, and never blocks it: bad arguments are
  ignored, a missing task context or an unwired process each log one WARNING
  per process and return.
• Works on undecorated tasks too, via celery.current_task.

Retry lineage

• task_runs.retried_from_id, stamped by retry_task() on the run it
  dispatches, so the chain is visible immediately rather than whenever the
  events listener catches up.
• control.list_retries_of() and the retried_from_id / retried_by fields
  on GET /api/tasks/{task_id}. retried_by is a list — retrying the same
  failed run more than once is normal, and each attempt is its own run.

Branding settings

• GET / PUT /api/settings and the app_settings table, holding exactly
  three keys: brand_name, brand_accent, brand_logo_url. Deployment-wide,
  not per-browser.
• GET /api/settings is deliberately unauthenticated — the login screen
  has to render the deployment's branding, and the login screen is pre-auth.
  That is safe only because DEFAULT_SETTINGS is a closed allowlist of
  non-sensitive values; updated_by/updated_at are deliberately not
  returned. Treat adding a key to that dict as a security decision.
• Values are validated strictly and rejected with a 422 carrying a
  human-readable reason: the accent is an anchored hex allowlist (it lands in
  a CSS custom property), and the logo URL is a scheme allowlist that refuses
  SVG data URIs (it lands in an <img src>).
• Any authenticated user can rebrand — there is no role system in v1 — but
  every changed key writes an audit row with old and new values.
• control.get_settings() / control.update_settings() for the same
  operations from Python.

Task detail depth

• GET /api/tasks/{task_id} gained queue, worker, eta, started_at,
  finished_at, runtime_seconds and the four progress_* fields. (eta
  is served but nothing currently writes it — see the note in the REST API
  reference.)

Dashboard

• Task list rebuilt around a left filter panel (search, status, group-by,
  collapsible date range) with removable active-filter chips, repeated in the
  empty state so an empty table says which filter emptied it. The panel
  collapses behind a Filters toggle below ~1024px.
• Two explicit modes: live (polls every 5 s, Load more widens the
  page up to the 500-row ceiling) and history (explicit offset paging
  through the full filtered total, polling switched off so rows can't shift
  under you between clicks), with a Back to live bar.
• Per-row progress bars with percentage, message and a projected time
  remaining; a run that has stopped reporting says how long it has been quiet
  and drops the estimate rather than counting down as if nothing were wrong.
• Task detail gained a Run panel (progress, live elapsed, estimated finish),
  an Execution panel (worker, queue, ETA, timings, each with a reason rather
  than a blank cell), and a retry-lineage panel walking the chain in both
  directions.
• Light / dark / system theming as an explicit per-browser choice, stored
  in local storage, applied before first paint and synced across tabs.
• A branding settings screen with a live preview and a measured contrast
  readout for the accent colour.
• Dashboard: sign-out control in the nav.
• Dashboard: Cancel and Terminate controls on the task detail page (terminate asks for inline confirmation)

Framework integration

• Mount the dashboard under a prefix inside a FastAPI app
  (app.mount("/celery-manager", dashboard)) or a Flask app (via a2wsgi
  and werkzeug's DispatcherMiddleware, new flask extra).
• The SPA now uses relative asset and API URLs, so it works under any mount
  prefix.
• Runnable examples/fastapi_demo and examples/flask_demo.
• New docs page docs/integrations.md.

Packaging and docs

• MkDocs Material documentation site under docs/.
• Root README.md, PyPI classifiers, keywords and project URLs; uvicorn
  moved into a standalone extra since mounting the ASGI app into a server
  you already run needs no server of its own.
• MIT license.

Changed

• @track now writes a started row before calling your task body, not
  only a terminal row afterwards. This is what makes an in-flight task
  visible with no events-listener process running, and what gives
  report_progress() a row to write to. Cost: one extra INSERT/UPDATE and
  commit per tracked run. As with every other write in the decorator, a
  failure is logged and dropped and can never change the task's outcome.
• POST /api/schedules/{name}/occurrences/pause now rejects
  action="reschedule" with a 422. It used to accept and store it, which
  displayed the occurrence as paused while doing nothing — the decorator only
  ever acted on skip/cancel. The column still permits the value; the
  endpoint no longer does.
• The session cookie is now Secure by default (cookie_secure=True). Set
  it to False only for local plain-HTTP development, where otherwise you
  will log in successfully and then immediately appear logged out.
• beat_schedule entries are normalized through
  celery.schedules.maybe_schedule(), so a bare number of seconds or a
  timedelta works the same as an explicit schedule()/crontab().
• self.retry() is now recorded as retried rather than failed. It raises
  celery.exceptions.Retry, an ordinary Exception subclass, which used to
  land in the decorator's failure branch and give a run Celery considers
  RETRY a failed status, a "Retry in 30s: …" exception message and a finish
  time it did not have.
• A retry attempt now clears what the previous attempt left on the shared row
  (exception, traceback, finished_at, runtime_seconds, the four
  progress_* columns). A task that failed once and then succeeded used to
  render as succeeded with a stale traceback and a stale progress bar.
• POST /api/tasks/{id}/retry returns 404 for an unknown task id instead of
  an unhandled ValueError (a 500); retry-bulk does the same.
  control.TaskRunNotFound is a ValueError subclass, so Python callers
  already catching ValueError are unaffected.

Fixed

• The events listener no longer downgrades a decorator-owned skipped status.
  A skipped occurrence returns normally, so the worker emits an ordinary
  task-succeeded for ...