You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Upgrading from 2.0
Struct literals.WorkerConfig gained result_cleanup_enabled and result_cleanup_interval, and PayloadEncryptionConfig gained key_rotation. Code
that builds these structs with a full struct literal must add the new fields or end
with ..Default::default(). Configuration files, from_file, Default and the
builders are unaffected. (#81
will make such structs #[non_exhaustive] in 3.0.)
Result cleanup is on by default. Worker pools now delete expired job results every
5 minutes. Expired results were already unreadable through get_job_result, but Job::result_data from get_job no longer shows them once cleaned. Set result_cleanup_enabled = false or call without_result_cleanup() to opt out.
Batch claims are opt-in.with_batch_processing_enabled(true) alone still claims one
job at a time; use with_batch_size(n) to claim several per poll.
Added
Batch claims.DatabaseQueue::dequeue_batch(queue, n) and dequeue_batch_leased(queue, weights, lease, n) claim up to n runnable jobs in one
round trip, with the same guarantees as a single claim: each job's lease is written by
the claiming statement, attempts is incremented, paused queues, unfinished
dependencies and disabled recurring jobs are honoured, jobs come back in priority DESC, scheduled_at ASC order, and concurrent claimers never get the same job.
PostgreSQL claims with one UPDATE ... WHERE id IN (SELECT ... LIMIT n FOR UPDATE SKIP LOCKED) RETURNING ...; MySQL with SELECT ... FOR UPDATE SKIP LOCKED LIMIT n and one UPDATE in the claim transaction (READ COMMITTED, retried on deadlock); TestQueue
matches them. Strict priority fills a batch in strict order; weighted priorities pick a
level by weight per batch, fill from it and top up from other levels. At most queue::MAX_DEQUEUE_BATCH_SIZE (1000) jobs per call.
DatabaseQueue::release_job_run hands a claimed job that never ran back to Pending, taking back the claim's attempt, so it can run elsewhere at once.
Batch-mode workers.Worker::with_batch_size(n) makes a worker claim up to n jobs
per poll (with_batch_processing_enabled(true) alone keeps worker::DEFAULT_BATCH_SIZE
= 1, so existing workers still claim one job at a time), and Worker::with_batch_concurrency(m) runs up to m of them at once (default 1). Each job
is processed exactly as in single-job mode (timeouts, retries, hooks, events, spawning,
result storage, encryption, statistics). Jobs waiting for their turn have their leases
renewed by heartbeats, so the stale job reaper never reclaims them while the worker is
alive. A throttle's max_concurrent permit is held per claimed job, and a batch claims
no more jobs than it has rate limit tokens (unused tokens are refunded). On shutdown,
claimed jobs that have not started are released at once. Workers without batch
processing claim one job at a time as before. New getters: is_batch_processing_enabled, batch_size, batch_concurrency. In a local benchmark, batch claims of 50 were 5-6x
faster than single claims on PostgreSQL and MySQL.
Automatic cleanup of expired job results.WorkerPool now clears expired job
results (DatabaseQueue::cleanup_expired_results) every 5 minutes in a background task,
as the README promised. Configure it with WorkerPool::with_result_cleanup(interval) / without_result_cleanup(), or with the new [worker] settings result_cleanup_enabled (default true) and result_cleanup_interval (default "5m"),
which WorkerPool::from_config and from_hammerwork_config apply. It is on by default
because expired results are already unreadable through get_job_result; clearing them
only frees space (and removes them from Job::result_data). Configuration files without
the new keys keep loading.
Key rotation from configuration.WorkerPool::with_key_rotation(key_manager, interval) rotates KeyManager keys as they fall due and stops on shutdown. The new [encryption.key_rotation] section (enabled, default false; master_key_source, check_interval_secs, max_key_versions, and the matching HAMMERWORK_ENCRYPTION_KEY_ROTATION_* variables) turns it on for pools built with from_hammerwork_config, with a KeyManager on the pool's database.
Key audit log API.KeyManager::audit_log(&KeyAuditFilter) reads hammerwork_key_audit_log, newest first, filtered by key, operation, outcome and time
range, with limit/offset paging, on PostgreSQL and MySQL. Also new: KeyOperation::ALL, parse_key_operation and FromStr/Copy/PartialEq for KeyOperation.
cargo hammerwork encryption audit shows the key audit log as a table or JSON, with --key-id, --operation, --since/--until (RFC 3339 or 30m/24h/7d), --success/--failed, --limit and --offset.
Fixed
Concurrent key rotation. Rotating the same key from several processes at once could
fail with a duplicate-key error, and an automatic rotation could rotate a key that another
process had just rotated. A rotation now writes its version only if the key is still at
the version it read (and, for automatic rotation, still due), in one transaction; the
unique (key_id, key_version) index decides between simultaneous writers. Automatic
rotation skips keys rotated elsewhere, and rotate_key reports "rotated concurrently"
without writing anything. A key never has two active versions.
KeyManager::start_rotation_service runs each pass in its own task, so dropping or
aborting the service no longer cancels a rotation part-way through its transaction.