Skip to content

feat: structured notes + LogLevel enum#164

Merged
pratyush618 merged 8 commits into
masterfrom
feat/structured-job-notes
May 14, 2026
Merged

feat: structured notes + LogLevel enum#164
pratyush618 merged 8 commits into
masterfrom
feat/structured-job-notes

Conversation

@pratyush618
Copy link
Copy Markdown
Collaborator

Summary

Adds two related additions to the public surface:

  • LogLevel enum for structured task logs. JobContext.log() and Queue.query_logs() now take a LogLevel instead of a free-form string. JobContext.publish() writes with LogLevel.RESULT. Subclasses str so the wire format to the Rust column is unchanged. Exported from the package root.
  • Structured notes field on jobs. New notes: dict | None kwarg on Queue.enqueue(), Queue.enqueue_many() (uniform + per-job notes_list), and TaskWrapper.apply_async(). Validated at the Python boundary against a tight contract: ≤ 15 top-level fields, ≤ 64-char keys, ≤ 500-char string leaves, ≤ 3 levels nesting, ≤ 4 KiB encoded. Stored as a canonical JSON-encoded string in a new notes column on jobs, dead_letter, and archived_jobs for SQLite + Postgres, and on the Redis Job/DeadJobEntry JSON. Read back as a parsed dict via JobResult.notes. Survives DLQ round-trips and replay. Distinct from the existing free-form metadata string blob.

Why notes ≠ metadata

metadata is a power-user escape hatch — opaque string, unbounded, unstructured. notes is the dashboard-renderable equivalent for user-readable annotations (customer IDs, business reasons, priority comments). Bounded field count means the dashboard can render the full set as a fixed-size key/value table.

Implementation outline

Layer What changed
Rust core New notes: Option<String> on Job / NewJob / JobRow / NewJobRow / DeadLetterRow / ArchivedJobRow / DeadJob. New notes TEXT column added to create_tables() and alter_statements() (idempotent on Postgres via IF NOT EXISTS). All call-sites updated (diesel_common/jobs.rs, sqlite/postgres dead_letter + archival, redis backend, scheduler, workflow ops).
PyO3 bindings notes parameter on enqueue/enqueue_batch signatures; notes getter on PyJob; type stubs updated.
Python New taskito.notes module with validate_and_encode_notes + size/depth/key constants. New NotesValidationError (subclasses both TaskitoError and ValueError). Wired through Queue.enqueue/enqueue_many, TaskWrapper.apply_async, JobResult.notes/to_dict.
Dashboard notes on the Job DTO; new NotesCard component on the job overview tab rendering as a key/value table (falls back to raw <pre> if the column can't be parsed).
Docs New docs/content/docs/guides/observability/notes.mdx guide; notes documented on queue.enqueue / enqueue_many reference. LogLevel docs refreshed in the existing logging guide and streaming/context references.

Cross-backend parity verified by the existing contract suite at crates/taskito-core/tests/rust/storage_tests.rs (Postgres + Redis kick in via CI when TASKITO_POSTGRES_TEST_URL / TASKITO_REDIS_TEST_URL are set).

Test plan

  • cargo check --workspace (default / +postgres / +redis / +native-async) — clean
  • cargo clippy --workspace --all-targets --all-features -- -D warnings — clean
  • cargo fmt --all -- --check — clean
  • cargo test --workspace — 91 tests pass, includes 2 new SQLite tests covering notes round-trip and DLQ retry preservation
  • uv run maturin develop — wheel builds clean
  • uv run python -m pytest tests/680 passed, 9 skipped (21 new tests in tests/core/test_notes.py)
  • uv run ruff check py_src/ tests/ — clean
  • uv run ruff format --check py_src/ tests/ — clean
  • uv run mypy py_src/taskito/ tests/ --no-incremental — clean (203 files)
  • pnpm --dir dashboard lint / typecheck / test — 97 vitest tests pass, biome + tsc clean
  • pnpm --dir docs lint — clean

@pratyush618 pratyush618 merged commit 9118ba0 into master May 14, 2026
22 checks passed
@pratyush618 pratyush618 deleted the feat/structured-job-notes branch May 14, 2026 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant