Skip to content

Feat/advanced features#12

Merged
pratyush618 merged 15 commits intomasterfrom
feat/advanced-features
Mar 7, 2026
Merged

Feat/advanced features#12
pratyush618 merged 15 commits intomasterfrom
feat/advanced-features

Conversation

@pratyush618
Copy link
Copy Markdown
Collaborator

No description provided.

…ings

- Add Redis storage backend with Lua scripts for atomic operations
- Add archival and queue_state modules for sqlite and postgres
- Extend storage traits with namespace, timezone, custom retry delays
- Update Python bindings to support new core features
- Add Rust integration tests
- Add events system and webhook support
- Add Flask, Prometheus, and Sentry contrib integrations
- Update serializers, dashboard, CLI, and app modules
- Update type stubs for new Rust bindings
- Enhance Django admin integration
- Update and add Python tests
- Update project config and lockfile
- Add OpenSSL setup steps for macOS (brew) and Windows (choco)
- Unify wheel build to use --features postgres on all platforms
- Set manylinux 2_28 for Linux with openssl-devel
- Remove Linux-only warnings from postgres and installation docs
- Add circuit breakers, events/webhooks, and logging guides
- Add integration docs for Django, FastAPI, Flask, OTel, Prometheus, Sentry
- Update FAQ, index, dashboard, monitoring, and advanced guides
- Add cargo fmt and clippy to lint job with rust-cache
- Use pip install -e ".[dev]" for pinned lint tool versions
- Extend ruff check/format to cover tests/ directory
- Remove redundant --ignore-missing-imports from mypy
- Split Rust tests into dedicated job (run once, not per Python version)
- Reduce Python test matrix from 15 to 9 jobs (min/max versions on macOS/Windows)
- Add rust-cache to all Rust-compiling jobs
- Add macos-13 for Intel wheel builds
- Add redis feature to wheel builds
- Use dynamic OpenSSL path glob on Windows
- Sync publish.yml build-wheels with ci.yml improvements
…base

- Fix Redis enqueue_unique() race condition with atomic Lua scripts
- Add dependency validation and storage to Redis enqueue_batch()
- Recover from poisoned mutex in scheduler instead of panicking
- Log warnings on unexpected migration failures (SQLite/Postgres)
- Validate EncryptedSerializer key type and size before use
- Wrap result deserialization errors with descriptive messages
- Add Postgres integration test wiring with env var + graceful skip
- Add pytest-cov to dev dependencies
- Add contrib middleware tests (OTel, Sentry, Prometheus) using mocks
- Validate canvas chunks() chunk_size > 0 and starmap() non-empty args
- Add graceful shutdown to PrometheusStatsCollector via threading.Event
- Log warning when async event loop thread doesn't stop within timeout
- Remove phantom to_dict() from PyJob type stubs
- Fix documentation URLs to point to canonical domain
…or handling

- Prevent i64 overflow in Python→Rust timeout/delay/expires/result_ttl conversions
- Add thread-safe double-checked locking for Prometheus metrics init and dashboard SPA cache
- Return 400 Bad Request for non-numeric dashboard query parameters
- Log warnings on purge_metrics/purge_task_logs failures instead of silent fallback
- Add proper worker fixture cleanup in test conftest
- Validate positive timeout in result()/aresult()
- Handle non-serializable extra dict in context.py task logging
- Skip webhook retries on 4xx client errors and log failure status codes
- Use urllib.parse for robust URL password masking
- Replace hardcoded status ints with JobStatus enum values in archival SQL
- Extract dashboard path segment helper to reduce fragile string slicing
- Add missing __repr__ stub for PyJob
…b integrations

Guard NaN/Infinity/negative floats before casting to i64 in delay_seconds,
expires, retry_backoff, and retry_delays. Add progress 0-100 range check.
Fix Django admin page param parsing, singleton race condition, and broad
exception catch. Refresh stale job status in FastAPI after aresult()
timeout. Use per-signature timeouts in canvas group/chord. Fix Redis
dequeue_score i32 overflow. Log cascade_cancel failures in DLQ. Validate
worker payload tuple length. Narrow version import exception.
…ype stub/API mismatches

- Validate limit/offset are non-negative in list_jobs, dead_letters, list_archived, query_task_logs
- Use saturating_mul/saturating_sub for timestamp arithmetic to prevent overflow
- Use saturating_add for circuit breaker failure_count increments
- Guard Redis log truncation against negative limit cast
- Fix archive_old_jobs type stub parameter name (cutoff_ms -> older_than_seconds)
- Reject negative values in dashboard _parse_int_qs and Django admin page params
- Log exceptions in Django admin _jobs_view instead of silently swallowing
- Match _FakeJobResult signature to real JobResult (poll_interval, max_poll_interval)
- Replace deprecated asyncio.get_event_loop() with get_running_loop()
…nd harden input validation

Cap float-to-i64 casts to prevent silent overflow in delay_seconds, expires,
retry_delays, and retry_backoff. Saturate now_millis() u128-to-i64 cast. Add
defensive clamping for Redis offset/limit pagination. Convert sync-calling
FastAPI handlers from async to sync, replace blanket error suppression with
logging, validate encrypted data length, reject empty items in chunks(), close
event loop on shutdown, fix falsy empty-string queue check, and expose
drain_timeout in Django settings.
… harden middleware/Redis paths

- Make _poll_once treat "cancelled" as terminal so result() raises immediately instead of hanging
- Use checked_add for TTL expiry arithmetic in SQLite/Postgres to prevent i64 overflow
- Log expire_pending_jobs errors in scheduler instead of silently discarding
- Validate enqueue_many kwargs_list/args_list length mismatch
- Add .max(0) guard on Redis list_archived offset/limit casts
- Replace deprecated asyncio.get_event_loop() with get_running_loop() in arun_worker
- Use saturating_add for Redis retry_count increment
- Keep dequeue_score arithmetic in i64 with clamped priority
- Replace Redis KEYS with SCAN in purge_job_errors
- Only call middleware after() for those whose before() succeeded
- Clamp percentile index to n-1 in task_metrics to prevent IndexError
…-safety and polling edge cases

Use checked_add/saturating_sub for timeout detection, worker reaping, scheduler cleanup, circuit breaker timing, and Redis TTL purging. Fix webhook thread-safety race on concurrent list mutation, wrap serializer decryption/decode errors, validate progress bounds and canvas concurrency, protect OTel span.end() with finally, guard Sentry scope leak on partial failure, handle stopped async loops, and use do-while polling in result waiters.
@pratyush618 pratyush618 self-assigned this Mar 7, 2026
@pratyush618 pratyush618 merged commit ab1b223 into master Mar 7, 2026
11 checks passed
@pratyush618 pratyush618 deleted the feat/advanced-features branch March 31, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant