From 8390ae6dba91ba55b0e013d7d424242dc610906c Mon Sep 17 00:00:00 2001 From: Simon Kelly Date: Thu, 6 Aug 2026 17:06:18 +0200 Subject: [PATCH 1/5] Document task nesting and error context providers for v2.5.0 Co-Authored-By: Claude Opus 5 (1M context) --- docs/changelog.md | 10 +++ docs/data_model.md | 13 +++- docs/python-celery.md | 29 ++++++++ docs/python-decorator.md | 17 +++++ docs/python-procrastinate.md | 13 ++++ docs/python.md | 126 +++++++++++++++++++++++++++++++++++ 6 files changed, 207 insertions(+), 1 deletion(-) diff --git a/docs/changelog.md b/docs/changelog.md index 0108a5f..6575ac3 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -7,6 +7,16 @@ hide: Full release notes for the Python SDK are available on [GitHub](https://github.com/taskbadger/taskbadger-python/releases). +## v2.5.0 + +**2026-08-06** + +**Python SDK** + +* **NEW** [Parent and child tasks](python.md#parent-and-child-tasks). Tasks can be nested one level deep via the [`parent`](data_model.md#parent) field on `create_task` / `update_task`, and `list_tasks` can filter by parent. The [`@track` decorator](python-decorator.md#nested-tasks), [Celery](python-celery.md#subtasks) and [Procrastinate](python-procrastinate.md#subtasks) integrations set it automatically for tasks enqueued from within a tracked task. +* **NEW** [Context providers](python.md#error-context-providers), a pluggable way to attach extra data to a task when it errors, along with a [Sentry provider](python.md#sentry) that links a failed task to its Sentry issue. Install with `pip install 'taskbadger[sentry]'`. +* **NEW** `Task.error` accepts an `exception` argument, which is passed to the configured context providers. + ## v2.4.0 **2026-07-30** diff --git a/docs/data_model.md b/docs/data_model.md index 25318a0..4dcee1d 100644 --- a/docs/data_model.md +++ b/docs/data_model.md @@ -44,6 +44,7 @@ The main attributes or a task are: : This is a computed percentage which is equivalent to `100 * value / value_max`. This will be `null` if **value** is null. + `data` : This can be used to store arbitrary JSON data that may be useful to store along with the task such @@ -79,6 +80,15 @@ The main attributes or a task are: with its logs. Set it when creating or updating a task, then filter tasks by exact match on this value in the web UI. + +`parent` + +: The ID of the task this task is part of. Tasks can only be nested one level deep, so the parent + must be a task that is not itself a child. A task's parent can not be changed once it has been + set. The `track` decorator and the Celery and Procrastinate integrations set this automatically + for tasks enqueued from within a tracked task, and it can also be set explicitly when creating or + updating a task. + ### Example Task ```json @@ -106,7 +116,8 @@ The main attributes or a task are: {"environment": "production"} ], "queue": "default", - "external_id": "celery-abc-123" + "external_id": "celery-abc-123", + "parent": null } ``` diff --git a/docs/python-celery.md b/docs/python-celery.md index 1e81644..c07bb1a 100644 --- a/docs/python-celery.md +++ b/docs/python-celery.md @@ -269,6 +269,35 @@ Task Badger will create task records for each inner invocation with metadata sim } ``` +## Subtasks + +==Since v2.5.0== + +A Celery task published from inside a tracked task is automatically nested under it via the +[`parent`](data_model.md#parent) field, so you can see the work a task spawned. + +Tasks nest a single level deep. A task published by a task that is itself a child becomes a sibling +of that child rather than a grandchild. + +What gets nested: + +- Tasks published from the body of a tracked task via `.delay()` or `.apply_async()`, including when + Celery runs them eagerly. +- Retries, which are nested under the first attempt. + +What does not get nested: + +- The next link of a `chain`, and `link` callbacks. These are successors of the task rather than work + it chose to enqueue. +- Tasks produced by [canvas primitives](#canvas-primitives-map-starmap-chunks) (`map`, `starmap`, + `chunks`) when they run on a worker. Their Task Badger tasks are created in the worker rather than + at publish time, so the enclosing task isn't visible there. They do nest when Celery runs eagerly. + +!!! note + + Known edge case: if a chain's next link is *also* called directly from the task body, that direct + call is not nested. + ## External ID The Celery task ID is automatically recorded on the Task Badger task's diff --git a/docs/python-decorator.md b/docs/python-decorator.md index 5aa49a5..f727926 100644 --- a/docs/python-decorator.md +++ b/docs/python-decorator.md @@ -17,6 +17,23 @@ status to `success` when the function completes or `error` if an exception is ra The decorator also applies the `taskbadger.Session` context manager to the function. See [connection management](python.md#connection-management). +When the function raises, the exception is recorded on the task data along with any context from the +configured [context providers](python.md#error-context-providers). + +## Nested tasks + +==Since v2.5.0== + +Tasks tracked by another integration while a decorated function is running are nested under its task +via the [`parent`](data_model.md#parent) field. That means other `@track` decorated functions called +from the body, as well as [Celery](python-celery.md#subtasks) and +[Procrastinate](python-procrastinate.md#subtasks) tasks enqueued from it. A bare `Task.create` in the +function body is not nested unless you pass `parent` yourself. + +Tasks nest a single level deep, so anything created by a decorated function that is itself a child +becomes a sibling of that child rather than a grandchild. Passing `parent` to the decorator explicitly +overrides the automatic nesting. + ## API Docs ::: taskbadger.track \ No newline at end of file diff --git a/docs/python-procrastinate.md b/docs/python-procrastinate.md index 2cfc3ac..20aea01 100644 --- a/docs/python-procrastinate.md +++ b/docs/python-procrastinate.md @@ -206,6 +206,19 @@ Task Badger task, so you can monitor the history and health of your scheduled jo The name of the Procrastinate queue a task is deferred to is automatically recorded on the Task Badger task's [`queue`](data_model.md#queue) field. +## Subtasks + +==Since v2.5.0== + +A job deferred from inside a tracked task is automatically nested under it via the +[`parent`](data_model.md#parent) field, so you can see the work a job spawned. + +Tasks nest a single level deep, so a job deferred by a task that is itself a child becomes a sibling +of that child rather than a grandchild. + +Nesting relies on the same wrapping as the rest of the integration, so the deferrals listed under +[Known Limitations](#known-limitations) are neither tracked nor nested. + ## External ID The Procrastinate job ID is automatically recorded on the Task Badger task's diff --git a/docs/python.md b/docs/python.md index 2865fd2..e570de6 100644 --- a/docs/python.md +++ b/docs/python.md @@ -33,6 +33,7 @@ page in the [Task Badger dashboard](https://taskbadger.net). | tags | Global tags which are added to all tasks. | | systems | System integrations such as [Celery](python-celery.md) | | before_create | A function that is called before a task is created. See [Before Create Callback](#before-create-callback) | +| context_providers | Providers that attach extra data to a task when it errors. See [Error Context Providers](#error-context-providers) | | organization_slug | The organization identifier. Only required for legacy API keys. | | project_slug | The project identifier. Only required for legacy API keys. | @@ -71,6 +72,36 @@ task = Task.get(task_id) The task object provides methods for updating the properties of a task and adding custom data. +### Parent and child tasks + +==Since v2.5.0== + +A task can be nested under another task by passing the parent's ID as the +[`parent`](data_model.md#parent) field: + +```python +from taskbadger import Task + +parent = Task.create("import") +child = Task.create("import.chunk", parent=parent.id) +``` + +Tasks nest a single level deep, so `parent` must be the ID of a task that is not itself a child. +A task's parent can not be changed once it has been set. + +Use `list_tasks` to fetch the children of a task: + +```python +import taskbadger + +children = taskbadger.list_tasks(parent=parent.id).results +``` + +`Task.create` and `create_task` only nest a task when you pass `parent` explicitly. The +[function decorator](python-decorator.md#nested-tasks), [Celery](python-celery.md#subtasks) and +[Procrastinate](python-procrastinate.md#subtasks) integrations do it for you: a task enqueued while +another tracked task is running is nested under it automatically. + ### Connection management The SDK will open a new connection for each request and close it when the request is complete. For instances @@ -139,6 +170,93 @@ def before_create(task_data: dict) -> dict: ==Since v1.5.0== +## Error Context Providers + +==Since v2.5.0== + +Context providers attach extra data to a task when it errors, for example a link back to the system +that reported the exception. They are consulted whenever a tracked task fails — via the +[function decorator](python-decorator.md), the [Celery](python-celery.md) or +[Procrastinate](python-procrastinate.md) integrations, or `Task.error(exception=...)` — and whatever +a provider returns is stored on the task [`data`](data_model.md#data) under the provider's identifier. + +Providers are registered with `taskbadger.init`: + +```python +import taskbadger +from taskbadger.context_providers.sentry import SentryContextProvider + +taskbadger.init( + token="YOUR_API_KEY", + context_providers=[SentryContextProvider(organization_slug="acme")], +) +``` + +A provider that raises is logged to the `taskbadger` logger and skipped, so it can never break the +task update. + +### Sentry + +`SentryContextProvider` links a failed task to the Sentry issue for the same exception. It requires +the `sentry-sdk` package, available via the `sentry` extra: + +```bash +uv add 'taskbadger[sentry]' +# or: pip install 'taskbadger[sentry]' +``` + +```python +import taskbadger +from taskbadger.context_providers.sentry import SentryContextProvider + +taskbadger.init( + token="YOUR_API_KEY", + context_providers=[SentryContextProvider(organization_slug="acme")], +) +``` + +Failed tasks then carry the Sentry event ID in their data, plus a link to the issue when +`organization_slug` is given: + +```json +{ + "exception": "bad input", + "sentry": { + "event_id": "5f8a...", + "url": "https://sentry.io/organizations/acme/issues/?query=5f8a..." + } +} +``` + +Pass `base_url` if you are running a self-hosted Sentry. + +!!! note + + The provider does not report the exception to Sentry itself. It assumes your application already + does that (e.g. via a framework integration) and reads back the event ID, which avoids reporting + the same exception twice. If the exception never reaches Sentry, no context is added. + +### Custom providers + +To attach context from another system, subclass `ContextProvider`, set an `identifier` and implement +`capture_error_context`: + +```python +from taskbadger.context_providers import ContextProvider + + +class RequestIdProvider(ContextProvider): + identifier = "request" + + def capture_error_context(self, exception, snapshot=None): + return {"id": get_current_request_id()} +``` + +Providers that read back state captured by another system, rather than capturing it themselves, +should also implement `snapshot`. It is called when a tracked task starts and its return value is +passed back as `snapshot`, so the provider can tell a fresh capture from a stale one left over from +something unrelated. + ## Python Reference ::: taskbadger.Task @@ -153,6 +271,14 @@ access to the API: ::: taskbadger.update_task +::: taskbadger.list_tasks + +## Context Provider Reference + +::: taskbadger.context_providers + +::: taskbadger.context_providers.sentry.SentryContextProvider + ## Safe functions For instances where you prefer not to handle errors you can use the following function which will handle From 02499b47933c73b4ef6f9108b7e7762d242f8291 Mon Sep 17 00:00:00 2001 From: Simon Kelly Date: Thu, 6 Aug 2026 17:08:22 +0200 Subject: [PATCH 2/5] Fix apply_async task args in Celery examples Co-Authored-By: Claude Opus 5 (1M context) --- docs/python-celery.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/python-celery.md b/docs/python-celery.md index c07bb1a..3313a39 100644 --- a/docs/python-celery.md +++ b/docs/python-celery.md @@ -120,14 +120,14 @@ def my_task(arg1, arg2): # using individual keyword arguments my_task.apply_async( - arg1, arg2, + args=[arg1, arg2], taskbadger_name="my task", taskbadger_value_max=1000, taskbadger_data={"custom": "data"}, ) # using a dictionary -my_task.apply_async(arg1, arg2, taskbadger_kwargs={ +my_task.apply_async(args=[arg1, arg2], taskbadger_kwargs={ "name": "my task", "value_max": 1000, "data": {"custom": "data"} From afccfce82d07ea52568a50908a2a4adb0cbfb1cb Mon Sep 17 00:00:00 2001 From: Simon Kelly Date: Thu, 6 Aug 2026 17:51:12 +0200 Subject: [PATCH 3/5] Note that taskbadger_ apply_async args need base=Task, document headers alternative Co-Authored-By: Claude Opus 5 (1M context) --- docs/index.md | 2 +- docs/python-celery.md | 56 +++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 55 insertions(+), 3 deletions(-) diff --git a/docs/index.md b/docs/index.md index 10ad1d3..503ad6f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -31,7 +31,7 @@ task.update(status=StatusEnum.SUCCESS, value=100) Or monitor Celery tasks automatically with the [Celery integration](python-celery.md): ```python -from taskbadger.systems import CelerySystemIntegration +from taskbadger.systems.celery import CelerySystemIntegration taskbadger.init( token="YOUR_API_KEY", diff --git a/docs/python-celery.md b/docs/python-celery.md index 3313a39..7b1b410 100644 --- a/docs/python-celery.md +++ b/docs/python-celery.md @@ -19,7 +19,7 @@ task that is executed by the Celery workers (except the internal Celery tasks), ```python import taskbadger -from taskbadger.systems import CelerySystemIntegration +from taskbadger.systems.celery import CelerySystemIntegration taskbadger.init( token="YOUR_API_KEY", @@ -107,9 +107,19 @@ The Task Badger task will also be updated when the task completes. ### Task Customization You can pass additional parameters to the Task Badger `Task` class which will be used when creating the task. -This can be done by passing keyword arguments prefixed with `taskbadger_` to the `.appy_async()` function or +This can be done by passing keyword arguments prefixed with `taskbadger_` to the `.apply_async()` function or to the task decorator. +!!! warning "`taskbadger_` arguments require the task base class" + + It is `taskbadger.celery.Task` that intercepts `taskbadger_`-prefixed arguments to `apply_async`, so + they only work on tasks declared with `base=Task`. On a plain Celery task they are silently ignored: + the task still publishes, and is still tracked if the + [system integration](#celery-system-integration) tracks it, but the options have no effect. + + To customize a task that is tracked by the system integration alone, pass the options in the + message headers instead — see [Customization without the task base class](#customization-without-the-task-base-class). + ```python # using the task decorator @@ -151,6 +161,48 @@ my_task.apply_async(args=[arg1, arg2], taskbadger_kwargs={ ==Since v1.4.0== +### Customization without the task base class + +Task Badger options can also be passed in the Celery message headers, which works for any task, +including plain Celery tasks tracked by the `CelerySystemIntegration`: + +```python +my_task.apply_async( + args=[arg1, arg2], + headers={"taskbadger_kwargs": { + "name": "my task", + "value_max": 1000, + "data": {"custom": "data"}, + }}, +) +``` + +Unlike the `taskbadger_`-prefixed arguments, the header is read when the task is published rather than +by the task class, so no `base=Task` is needed. It takes the same options as `taskbadger_kwargs`, minus +the prefix, and takes precedence over values set on the task decorator. + +If the task would not otherwise be tracked — it isn't using the base class and doesn't match the system +integration's tracking rules — add `taskbadger_track` to the headers to track it anyway: + +```python +my_task.apply_async( + args=[arg1, arg2], + headers={ + "taskbadger_track": True, + "taskbadger_kwargs": {"name": "my task"}, + }, +) +``` + +!!! note + + `record_task_args` is a header of its own rather than an entry in `taskbadger_kwargs`: + `headers={"taskbadger_record_task_args": True}`. + + The `taskbadger_task_id` attribute and `get_taskbadger_task()` method of the + [result object](#basic-usage) are added by `taskbadger.celery.Task`, so they are not available on + the result when using headers alone. + ### Accessing the Task Object The `taskbadger.celery.Task` class provides access to the Task Badger task object via the `taskbadger_task` property From 5f3574c2943a49480a6f74d217087f79ea428f68 Mon Sep 17 00:00:00 2001 From: Simon Kelly Date: Thu, 6 Aug 2026 17:57:03 +0200 Subject: [PATCH 4/5] Changelog and docs for v2.5.1 Co-Authored-By: Claude Opus 5 (1M context) --- docs/changelog.md | 10 ++++++++++ docs/python-celery.md | 18 +++++++++++++++++ docs/python.md | 45 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 72 insertions(+), 1 deletion(-) diff --git a/docs/changelog.md b/docs/changelog.md index 6575ac3..570fd0f 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -7,6 +7,16 @@ hide: Full release notes for the Python SDK are available on [GitHub](https://github.com/taskbadger/taskbadger-python/releases). +## v2.5.1 + +**2026-08-06** + +**Python SDK** + +* **CHANGED** [`list_tasks`](python.md#listing-tasks) returns a `TaskList` of `taskbadger.Task` objects, which can be iterated over directly. It previously returned the generated `PaginatedTaskList`, whose `results` were internal models without the SDK's update methods. Note that an empty `TaskList` is falsy, where `PaginatedTaskList` was always truthy. +* **FIX** Eager and canvas Celery tasks honour an explicit [`taskbadger_parent`](python-celery.md#subtasks), including `taskbadger_parent=None` to opt out of nesting. +* **FIX** Copying or pickling a `Task` no longer recurses until the stack overflows. + ## v2.5.0 **2026-08-06** diff --git a/docs/python-celery.md b/docs/python-celery.md index 7b1b410..4c5cd50 100644 --- a/docs/python-celery.md +++ b/docs/python-celery.md @@ -350,6 +350,24 @@ What does not get nested: Known edge case: if a chain's next link is *also* called directly from the task body, that direct call is not nested. +To override the automatic nesting, pass `taskbadger_parent` for the call. An explicit `None` makes the +task a root task even though it was published from inside a tracked task: + +```python +# nest under a different task +my_task.apply_async(taskbadger_parent=other_task.id) + +# opt out of nesting +my_task.apply_async(taskbadger_parent=None) +``` + +As with the other `taskbadger_` arguments this requires the task to use `base=Task`; without it, pass +`headers={"taskbadger_kwargs": {"parent": None}}` instead. See +[Customization without the task base class](#customization-without-the-task-base-class). + +==Since v2.5.1== eager tasks and [canvas primitives](#canvas-primitives-map-starmap-chunks) honour +`taskbadger_parent` too; before that they always nested under the enclosing task. + ## External ID The Celery task ID is automatically recorded on the Task Badger task's diff --git a/docs/python.md b/docs/python.md index e570de6..7ea69ee 100644 --- a/docs/python.md +++ b/docs/python.md @@ -72,6 +72,46 @@ task = Task.get(task_id) The task object provides methods for updating the properties of a task and adding custom data. +### Listing tasks + +`list_tasks` returns a [`TaskList`](#taskbadger.TaskList), a single page of tasks that you can iterate +over directly. The tasks it yields are ordinary `Task` objects, so they can be updated in place: + +```python +import taskbadger +from taskbadger import StatusEnum + +for task in taskbadger.list_tasks(page_size=50): + if task.status == StatusEnum.PENDING: + task.canceled() +``` + +The `next_` and `previous` attributes hold the URL of the adjacent page, or are unset if there isn't +one. To fetch the next page, pass its `cursor` query parameter back to `list_tasks`: + +```python +from urllib.parse import parse_qs, urlparse + +page = taskbadger.list_tasks(page_size=100) +while True: + for task in page: + ... + if not page.next_: + break + cursor = parse_qs(urlparse(page.next_).query)["cursor"][0] + page = taskbadger.list_tasks(page_size=100, cursor=cursor) +``` + +!!! note "Changed in v2.5.1" + + `list_tasks` previously returned the generated `PaginatedTaskList` whose `results` were + `taskbadger.internal.models.Task` objects, without the SDK's `update()` / `safe_update()` methods. + It now returns a `TaskList` of `taskbadger.Task` objects. + + A `TaskList` also has a length, so an empty page is falsy where `PaginatedTaskList` was always + truthy. If you have code like `if taskbadger.list_tasks(...):`, note that it now tests whether the + page has any tasks in it. + ### Parent and child tasks ==Since v2.5.0== @@ -94,7 +134,8 @@ Use `list_tasks` to fetch the children of a task: ```python import taskbadger -children = taskbadger.list_tasks(parent=parent.id).results +for child in taskbadger.list_tasks(parent=parent.id): + print(child.name, child.status) ``` `Task.create` and `create_task` only nest a task when you pass `parent` explicitly. The @@ -273,6 +314,8 @@ access to the API: ::: taskbadger.list_tasks +::: taskbadger.TaskList + ## Context Provider Reference ::: taskbadger.context_providers From f05371b2d910d2e16a0f4546a0fd6b190fc44d94 Mon Sep 17 00:00:00 2001 From: Simon Kelly Date: Fri, 7 Aug 2026 11:15:12 +0200 Subject: [PATCH 5/5] Correct review findings and document the v2.5.2 opt-out fix Co-Authored-By: Claude Opus 5 (1M context) --- docs/changelog.md | 12 +++++- docs/data_model.md | 9 ++-- docs/python-celery.md | 79 +++++++++++++++++++++++++++++++----- docs/python-decorator.md | 6 ++- docs/python-procrastinate.md | 2 +- docs/python.md | 48 +++++++++++++++++----- 6 files changed, 129 insertions(+), 27 deletions(-) diff --git a/docs/changelog.md b/docs/changelog.md index 570fd0f..db4e07e 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -7,14 +7,22 @@ hide: Full release notes for the Python SDK are available on [GitHub](https://github.com/taskbadger/taskbadger-python/releases). +## v2.5.2 + +**2026-08-07** + +**Python SDK** + +* **FIX** [`taskbadger_track=False`](python-celery.md#opting-out) now opts a single Celery execution out of tracking, as an `apply_async` argument or in the message headers. Previously it could only ever enable tracking: it was ignored when the `CelerySystemIntegration` was auto-tracking, and `apply_async` overwrote it on tasks using `base=Task`. + ## v2.5.1 -**2026-08-06** +**2026-08-07** **Python SDK** * **CHANGED** [`list_tasks`](python.md#listing-tasks) returns a `TaskList` of `taskbadger.Task` objects, which can be iterated over directly. It previously returned the generated `PaginatedTaskList`, whose `results` were internal models without the SDK's update methods. Note that an empty `TaskList` is falsy, where `PaginatedTaskList` was always truthy. -* **FIX** Eager and canvas Celery tasks honour an explicit [`taskbadger_parent`](python-celery.md#subtasks), including `taskbadger_parent=None` to opt out of nesting. +* **FIX** Eager Celery tasks honour an explicit [`taskbadger_parent`](python-celery.md#subtasks), including `taskbadger_parent=None` to opt out of nesting. * **FIX** Copying or pickling a `Task` no longer recurses until the stack overflows. ## v2.5.0 diff --git a/docs/data_model.md b/docs/data_model.md index 4dcee1d..126f5a7 100644 --- a/docs/data_model.md +++ b/docs/data_model.md @@ -85,9 +85,12 @@ The main attributes or a task are: : The ID of the task this task is part of. Tasks can only be nested one level deep, so the parent must be a task that is not itself a child. A task's parent can not be changed once it has been - set. The `track` decorator and the Celery and Procrastinate integrations set this automatically - for tasks enqueued from within a tracked task, and it can also be set explicitly when creating or - updating a task. + set. Both rules are enforced by the API, so breaking either one causes the request to fail. + + It can be set explicitly when creating or updating a task. The Celery and Procrastinate + integrations also set it automatically for tasks enqueued from within a tracked task, and the + `track` decorator sets it on a decorated function called from within one. Each integration + excludes some cases — see [Python SDK](python.md#parent-and-child-tasks) for details. ### Example Task diff --git a/docs/python-celery.md b/docs/python-celery.md index 4c5cd50..5d051ac 100644 --- a/docs/python-celery.md +++ b/docs/python-celery.md @@ -110,14 +110,17 @@ You can pass additional parameters to the Task Badger `Task` class which will be This can be done by passing keyword arguments prefixed with `taskbadger_` to the `.apply_async()` function or to the task decorator. -!!! warning "`taskbadger_` arguments require the task base class" +!!! warning "`taskbadger_` arguments to `apply_async` require the task base class" It is `taskbadger.celery.Task` that intercepts `taskbadger_`-prefixed arguments to `apply_async`, so - they only work on tasks declared with `base=Task`. On a plain Celery task they are silently ignored: + those only work on tasks declared with `base=Task`. On a plain Celery task they are silently ignored: the task still publishes, and is still tracked if the [system integration](#celery-system-integration) tracks it, but the options have no effect. - To customize a task that is tracked by the system integration alone, pass the options in the + `taskbadger_` arguments on the **task decorator** are not affected. They are read off the task class + when the task is published, so they apply whether or not the task uses `base=Task`. + + To set options per call on a task that is tracked by the system integration alone, pass them in the message headers instead — see [Customization without the task base class](#customization-without-the-task-base-class). ```python @@ -181,6 +184,16 @@ Unlike the `taskbadger_`-prefixed arguments, the header is read when the task is by the task class, so no `base=Task` is needed. It takes the same options as `taskbadger_kwargs`, minus the prefix, and takes precedence over values set on the task decorator. +!!! warning "Eager tasks read fewer options from the header" + + That applies to tasks which are actually published. When Celery runs a task [eagerly][always_eager] + nothing is published, so the Task Badger task is created as the task starts instead, reading the + header directly. Only `parent`, `heartbeat_interval` and `stale_timeout` are honoured there — + `name`, `value_max` and `data` are ignored. + + `taskbadger_track` is also *required* for an eager task that doesn't use `base=Task`, even if the + system integration would otherwise track it. + If the task would not otherwise be tracked — it isn't using the base class and doesn't match the system integration's tracking rules — add `taskbadger_track` to the headers to track it anyway: @@ -355,18 +368,30 @@ task a root task even though it was published from inside a tracked task: ```python # nest under a different task -my_task.apply_async(taskbadger_parent=other_task.id) +my_task.apply_async(args=[arg1, arg2], taskbadger_parent=parent_task.id) # opt out of nesting -my_task.apply_async(taskbadger_parent=None) +my_task.apply_async(args=[arg1, arg2], taskbadger_parent=None) ``` -As with the other `taskbadger_` arguments this requires the task to use `base=Task`; without it, pass -`headers={"taskbadger_kwargs": {"parent": None}}` instead. See +`taskbadger_parent` takes a **Task Badger** task ID, not a Celery task or result ID. + +As with the other `taskbadger_` arguments to `apply_async` this requires the task to use `base=Task`; +without it, pass `headers={"taskbadger_kwargs": {"parent": None}}` instead. See [Customization without the task base class](#customization-without-the-task-base-class). -==Since v2.5.1== eager tasks and [canvas primitives](#canvas-primitives-map-starmap-chunks) honour -`taskbadger_parent` too; before that they always nested under the enclosing task. +==Since v2.5.1== eager tasks honour `taskbadger_parent`; before that they always nested under the +enclosing task. + +!!! warning "Canvas primitives ignore `taskbadger_parent`" + + [Canvas primitives](#canvas-primitives-map-starmap-chunks) (`map`, `starmap`, `chunks`) are + published under Celery's own `celery.map` / `celery.starmap` task rather than your own, so + `taskbadger_parent` is never intercepted and the `taskbadger_kwargs` header doesn't reach the + worker. On a worker their parent can't be set per call, and they are not nested at all. + + Run eagerly they do nest under the enclosing task, and there the parent can be overridden with + `headers={"taskbadger_kwargs": {"parent": ...}}`. ## External ID @@ -376,8 +401,42 @@ originating Celery task. ## Opting out -If you want to prevent TaskBadger from tracking a particular execution, set the `taskbadger_track` header (False) when publishing: +To stop Task Badger tracking a single execution, set `taskbadger_track` to `False`, either as an +argument to `apply_async` or in the message headers: ```python +# as an argument — needs base=Task +my_task.apply_async(args=[arg1, arg2], taskbadger_track=False) + +# in the headers — works for any task +my_task.apply_async(args=[arg1, arg2], headers={"taskbadger_track": False}) + +# canvas primitives take the header form only add.map([(1, 2), (2, 3)]).apply_async(headers={"taskbadger_track": False}) ``` + +This takes precedence over the `CelerySystemIntegration`, so it opts out even when `auto_track_tasks` +is on, and over the `base=Task` class. + +The value must be exactly `False`. Omitting it leaves tracking to the normal rules. + +!!! warning "The argument form needs the task base class" + + As with the other [`taskbadger_` arguments to `apply_async`](#task-customization), + `taskbadger_track=False` is intercepted by `taskbadger.celery.Task`, so it only works on tasks + declared with `base=Task`. On a plain Celery task, or on a + [canvas primitive](#canvas-primitives-map-starmap-chunks), it is silently ignored and the task + stays tracked. + + The header form works in all three cases, so prefer it unless you know the task uses `base=Task`. + +To exclude a task from tracking on every call rather than per execution, use the `excludes` argument to +[`CelerySystemIntegration`](#celery-system-integration), which matches on task name. + +!!! warning "Only works from v2.5.2" + + ==Since v2.5.2== + + Before that this header could only ever *enable* tracking, never suppress it. `False` was + indistinguishable from omitting the header, and `apply_async` on a `base=Task` task overwrote it + with `True`. On earlier versions use `excludes` instead. diff --git a/docs/python-decorator.md b/docs/python-decorator.md index f727926..e535489 100644 --- a/docs/python-decorator.md +++ b/docs/python-decorator.md @@ -30,10 +30,14 @@ from the body, as well as [Celery](python-celery.md#subtasks) and [Procrastinate](python-procrastinate.md#subtasks) tasks enqueued from it. A bare `Task.create` in the function body is not nested unless you pass `parent` yourself. +Each integration carves out some exceptions — Celery, for instance, doesn't nest chain successors, +`link` callbacks, or canvas primitives running on a worker. The [Celery](python-celery.md#subtasks) and +[Procrastinate](python-procrastinate.md#subtasks) pages list what does and doesn't get nested. + Tasks nest a single level deep, so anything created by a decorated function that is itself a child becomes a sibling of that child rather than a grandchild. Passing `parent` to the decorator explicitly overrides the automatic nesting. ## API Docs -::: taskbadger.track \ No newline at end of file +::: taskbadger.track diff --git a/docs/python-procrastinate.md b/docs/python-procrastinate.md index 20aea01..367c939 100644 --- a/docs/python-procrastinate.md +++ b/docs/python-procrastinate.md @@ -216,7 +216,7 @@ A job deferred from inside a tracked task is automatically nested under it via t Tasks nest a single level deep, so a job deferred by a task that is itself a child becomes a sibling of that child rather than a grandchild. -Nesting relies on the same wrapping as the rest of the integration, so the deferrals listed under +Nesting relies on the same wrapping as the rest of the integration, so the cases listed under [Known Limitations](#known-limitations) are neither tracked nor nested. ## External ID diff --git a/docs/python.md b/docs/python.md index 7ea69ee..80896c9 100644 --- a/docs/python.md +++ b/docs/python.md @@ -86,8 +86,8 @@ for task in taskbadger.list_tasks(page_size=50): task.canceled() ``` -The `next_` and `previous` attributes hold the URL of the adjacent page, or are unset if there isn't -one. To fetch the next page, pass its `cursor` query parameter back to `list_tasks`: +The `next_` and `previous` attributes hold the URL of the adjacent page, or `None` if there isn't one. +To fetch the next page, pass its `cursor` query parameter back to `list_tasks`: ```python from urllib.parse import parse_qs, urlparse @@ -127,7 +127,8 @@ child = Task.create("import.chunk", parent=parent.id) ``` Tasks nest a single level deep, so `parent` must be the ID of a task that is not itself a child. -A task's parent can not be changed once it has been set. +A task's parent can not be changed once it has been set. Both rules are enforced by the API rather +than by the SDK, so breaking either one surfaces as a failed request. Use `list_tasks` to fetch the children of a task: @@ -141,7 +142,11 @@ for child in taskbadger.list_tasks(parent=parent.id): `Task.create` and `create_task` only nest a task when you pass `parent` explicitly. The [function decorator](python-decorator.md#nested-tasks), [Celery](python-celery.md#subtasks) and [Procrastinate](python-procrastinate.md#subtasks) integrations do it for you: a task enqueued while -another tracked task is running is nested under it automatically. +another tracked task is running is nested under it automatically. Because nesting is capped at one +level, a task enqueued from within a *child* becomes a sibling of that child rather than a grandchild. + +Each integration excludes a few cases from the automatic nesting — see the linked pages for what +does and doesn't get nested. ### Connection management @@ -217,9 +222,24 @@ def before_create(task_data: dict) -> dict: Context providers attach extra data to a task when it errors, for example a link back to the system that reported the exception. They are consulted whenever a tracked task fails — via the -[function decorator](python-decorator.md), the [Celery](python-celery.md) or -[Procrastinate](python-procrastinate.md) integrations, or `Task.error(exception=...)` — and whatever -a provider returns is stored on the task [`data`](data_model.md#data) under the provider's identifier. +[function decorator](python-decorator.md), or the [Celery](python-celery.md) or +[Procrastinate](python-procrastinate.md) integrations — and whatever a provider returns is stored on +the task [`data`](data_model.md#data) under the provider's identifier. + +You can also call `Task.error(exception=...)` yourself, but read the caveat below first. + +!!! warning "`Task.error()` on its own has no baseline" + + Those three integrations record the state of each provider as the task *starts*, so a provider can + tell context belonging to this task from context left over from something unrelated. + + Calling `task.error(exception=...)` directly does consult the providers, but nothing took that + baseline, so they have nothing to compare against. With + [`SentryContextProvider`](#sentry) that means the task gets a link to whatever + `sentry_sdk.last_event_id()` happens to be — quite possibly a stale, unrelated issue. + + Prefer letting an integration own the error path. If you must call `Task.error` yourself and the + link matters, pass the context in explicitly via `data` instead of relying on a provider. Providers are registered with `taskbadger.init`: @@ -238,8 +258,9 @@ task update. ### Sentry -`SentryContextProvider` links a failed task to the Sentry issue for the same exception. It requires -the `sentry-sdk` package, available via the `sentry` extra: +`SentryContextProvider` links a failed task to the Sentry issue for the same exception. It needs the +`sentry-sdk` package, available via the `sentry` extra. If the package isn't installed the provider is +a silent no-op — it adds no context and reports no error, so install the extra: ```bash uv add 'taskbadger[sentry]' @@ -271,11 +292,18 @@ Failed tasks then carry the Sentry event ID in their data, plus a link to the is Pass `base_url` if you are running a self-hosted Sentry. +The `exception` value is `str(exception)`, except on the Celery path, where Celery's own exception info +is used and the value is a full traceback. + !!! note The provider does not report the exception to Sentry itself. It assumes your application already does that (e.g. via a framework integration) and reads back the event ID, which avoids reporting - the same exception twice. If the exception never reaches Sentry, no context is added. + the same exception twice. + + To avoid linking to an unrelated event, the provider records Sentry's current event ID when the + task starts and only attaches context if it has changed by the time the task errors. So no context + is added when the exception never reaches Sentry, or when Sentry saw nothing new while the task ran. ### Custom providers