feat: structured notes + LogLevel enum#164
Merged
Merged
Conversation
JSON-encoded notes field on jobs/dead_letter/archived_jobs across SQLite, Postgres, and Redis; exposed as a Python kwarg on enqueue/enqueue_batch and as a getter on PyJob.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two related additions to the public surface:
LogLevelenum for structured task logs.JobContext.log()andQueue.query_logs()now take aLogLevelinstead of a free-form string.JobContext.publish()writes withLogLevel.RESULT. Subclassesstrso the wire format to the Rust column is unchanged. Exported from the package root.notesfield on jobs. Newnotes: dict | Nonekwarg onQueue.enqueue(),Queue.enqueue_many()(uniform + per-jobnotes_list), andTaskWrapper.apply_async(). Validated at the Python boundary against a tight contract: ≤ 15 top-level fields, ≤ 64-char keys, ≤ 500-char string leaves, ≤ 3 levels nesting, ≤ 4 KiB encoded. Stored as a canonical JSON-encoded string in a newnotescolumn onjobs,dead_letter, andarchived_jobsfor SQLite + Postgres, and on the RedisJob/DeadJobEntryJSON. Read back as a parsed dict viaJobResult.notes. Survives DLQ round-trips and replay. Distinct from the existing free-formmetadatastring blob.Why notes ≠ metadata
metadatais a power-user escape hatch — opaque string, unbounded, unstructured.notesis the dashboard-renderable equivalent for user-readable annotations (customer IDs, business reasons, priority comments). Bounded field count means the dashboard can render the full set as a fixed-size key/value table.Implementation outline
notes: Option<String>onJob/NewJob/JobRow/NewJobRow/DeadLetterRow/ArchivedJobRow/DeadJob. Newnotes TEXTcolumn added tocreate_tables()andalter_statements()(idempotent on Postgres viaIF NOT EXISTS). All call-sites updated (diesel_common/jobs.rs, sqlite/postgres dead_letter + archival, redis backend, scheduler, workflow ops).notesparameter onenqueue/enqueue_batchsignatures;notesgetter onPyJob; type stubs updated.taskito.notesmodule withvalidate_and_encode_notes+ size/depth/key constants. NewNotesValidationError(subclasses bothTaskitoErrorandValueError). Wired throughQueue.enqueue/enqueue_many,TaskWrapper.apply_async,JobResult.notes/to_dict.noteson theJobDTO; newNotesCardcomponent on the job overview tab rendering as a key/value table (falls back to raw<pre>if the column can't be parsed).docs/content/docs/guides/observability/notes.mdxguide;notesdocumented onqueue.enqueue/enqueue_manyreference.LogLeveldocs refreshed in the existing logging guide and streaming/context references.Cross-backend parity verified by the existing contract suite at
crates/taskito-core/tests/rust/storage_tests.rs(Postgres + Redis kick in via CI whenTASKITO_POSTGRES_TEST_URL/TASKITO_REDIS_TEST_URLare set).Test plan
cargo check --workspace(default /+postgres/+redis/+native-async) — cleancargo clippy --workspace --all-targets --all-features -- -D warnings— cleancargo fmt --all -- --check— cleancargo test --workspace— 91 tests pass, includes 2 new SQLite tests covering notes round-trip and DLQ retry preservationuv run maturin develop— wheel builds cleanuv run python -m pytest tests/— 680 passed, 9 skipped (21 new tests intests/core/test_notes.py)uv run ruff check py_src/ tests/— cleanuv run ruff format --check py_src/ tests/— cleanuv run mypy py_src/taskito/ tests/ --no-incremental— clean (203 files)pnpm --dir dashboard lint/typecheck/test— 97 vitest tests pass, biome + tsc cleanpnpm --dir docs lint— clean