Repository navigation
wurk v1.11.0
Fixes a 1.10.0 boot regression under wurkswarm: upgrade if your config/application.rb relies on the swarm's preload. Native periodics now keep firing through leader deaths, and push_bulk is cheaper.
Added
wurk preflight: what a live Redis holds and what carries over, before a cutover. Read-only:SCANand bounded sampling through a client that refuses every other command, neverLLEN. Reports Redis version, eviction, topology and Dragonfly; live processes by version; everyqueue:*|*key as owner-alive, reclaimable or unparseable; timestamp shapes andretry_forjobs; in-flight batches with nocallbacks;encryptpayloads without an envelope;uniquejobs:*andunique:*counts including SHA1-length digests.--app PATHboots the app to listlock:classes with no gem and known-broken add-ons, and runs your initializers.--json,--limit(a truncated scan is a finding),--sample,--pause-ms. Exit 0 clean, 1 findings, 2 could not run. About 10s at 1M keys with Redis latency under 5ms in a local test.wurk verify: the post-cutover check. Queues with jobs have a subscriber, no non-Wurk process is live, private-list count is sane, a live Wurk process holds the leader lock, every unpaused loop has a next fire. Read-only;--canary CLASSexplicitly pushes one job per subscribed queue.docs/migrate-from-sidekiq.md§10: a path for Sidekiq 7.x Pro/Enterprise apps that keep their add-on gems, with what was verified per gem, and what a 7.x rollback shows.Sidekiq::WebHelpers(alias ofWurk::Web::Extension::Helpers) withjob_params,parse_key,parse_params,qparams,to_query_string,display_args,filtering,redirect_with_query,text_direction,rtl?,locale,stats,workset,processes,sorted_processes,SAFE_QPARAMS,RETRY_JOB_KEYS. A subset of free §25.7, not all of it.Sidekiq::Paginator(page,page_items), andpageon the extension Action. Action also gainsconfig.erbin extensions honourslocals:; built-in_pagingand_job_infopartials; upstream string keys read as English (DeleteAll→Delete All).Sidekiq.gem_version,Sidekiq.freeze!.- sidekiq-unique-jobs, sidekiq-failures and sentry-sidekiq in
test:ecosystem, and a real-worker boot check for all four harnesses, coveringsidekiqandwurkswarmwith preload on, off, and fake mode at load. bin/repin-anchors: re-pins the ledger's anchor records, syncs both of the ledger's reference styles andtest/conformance/cases.ymlafter lines move, with--checkto report without writing.mgr.grace = 300inconfig.periodiclets a slot fire later than the default 90s, for a cut-over fromsidekiq-cron. Process-wide, default unchanged.
Changed
mount Sidekiq::Webin a Rails app now shows the dashboard. Extension routes still serve at their own paths; every other path goes to the engine, through the engine's ownWurk::Web.usemiddleware, authorization hook and read-only gate. The machine API is not served under this mount. Upgrade note: a host that kept this mount as a 404 (so it only exposed extension pages) now exposes the whole dashboard. Gate it as you wouldmount Wurk::Engine. As with stock Sidekiq, a mount with no auth is open.- A worker started by
wurk,sidekiq, a swarm child orWurk::Embeddednow refusesSidekiq::Testingfake or inline mode: it logs an ERROR and disables the mode for that process. Previously, as on stock Sidekiq, every enqueue went to an in-memory array and was lost. Upgrade note: enqueues that were being dropped start reaching Redis. Moverspec-sidekiqto your Gemfile's test group. A Rails web process enqueuing in fake mode is not covered, and the guard stands down underRAILS_ENV=testorRACK_ENV=test. - A job with
encryptset and a last argument that is not a Wurk envelope no longer reachesperform. It goes to the dead set withencryption_error: Wurk::Encryption::NotSealedError, without retries, and death handlers fire. That kill, like every decrypt failure's, no longer trims the dead set, and logs an ERROR with class and jid, so a misconfigured producer cannot destroy a backlog throughdead_max_jobs. Before, another implementation's ciphertext was passed toperformas if decrypted. Upgrade note: this also catches a plaintext job from a producer that hasencrypt: trueon the class but crypto not enabled, which used to run in the clear. Enable crypto on producers first. A dead-set retry seals the argument as it stands, which recovers plaintext jobs; do not retry foreign ciphertext. Only applies with crypto enabled on the worker. Sidekiq::Webas a bare Rack app answers a non-extension path with a 404 whose body names the fix, and logs one WARN.- Extension
truncatecuts like upstream:text[0..n]followed by..., not…. - An extension's tab on a path the SPA also renders natively (sidekiq-cron's Cron) is no longer hidden. With sidekiq-cron, the nav shows the native Cron entry and the gem's Cron tab.
- An extension view that names a view that doesn't exist now raises
extension <name> has no view <view>, instead ofhas no root_dir. push_bulkis cheaper: 15.0µs to 9.4µs per job (stock Sidekiq 6.8µs, 10 paired rounds). It normalizes once per slice instead of once per job. Payload key order is unchanged. Upgrade note: jobs in one bulk slice now share acreated_at; they used to differ by the milliseconds the loop took. Stock Sidekiq shares one across the whole bulk. A per-jobatkeeps the old walk, since expiry and deadline are measured from it.--helpdiffers per binary and names environment variables and the docs.bin/rails g wurk:installprints next steps, including that the dashboard has no login of its own.- Each release gets a CycloneDX SBOM asset and an SBOM attestation, from a separate
attest.ymlthat can't reach publishing.
Fixed
docs/migrate-from-sidekiq.mdno longer claims::while_executingmaps tounique_until: :start; "worst case one extra run" on dropping sidekiq-unique-jobs; leftover gem locks "do nothing" (they silently block enqueues after a rollback; UNLINK them); same encryption keys carry Enterprise jobs over; the default scheduler has a job-loss window;reliable_scheduler!must stay on; the Sidekiq web UI "keeps working" unqualified.docs/compatibility.mdpointed the parity-divergence rule atdocs/target; it is the ledger.docs/sentry.mdno longer sayssentry-sidekiqreports the wrong errors on Wurk. Against a dummy transport it reported the right exception per attempt; it has not been tested with a real Sentry.wurkswarmno longer crashes apps whoseconfig/application.rbrelies on the swarm's preload. Since 1.10.0, an app that skipsBundler.requirebecause the swarm already ran it, then uses a preloaded gem, died at boot withwurkswarm: uninitialized constant Dotenv(or whichever gem) and exit 1. It now restarts itself once with the preload first, underbundle exec, a binstub orload Gem.bin_path, keeping its pid, and logs a warning naming the first error. Gems that need Rails at require time keep working. Such an app does start twice per boot; aBundler.requireinconfig/application.rbavoids that.- Boot's
lock:check no longer dies on a class that redefinesself.name. With a gem like Faker loaded (Faker::Travel::Airport),ArgumentError: missing keywordsskipped the whole check, so jobs setting alock:that did nothing got no warning.Sidekiq::Testing's queue check had the same flaw. - Native periodics survive a leader's death. A follower rechecked the lock on a flat 60s, so a killed leader left up to 90s with no lease; it now rechecks when the lease it read runs out, about a second after a lapse (31s at most after a
SIGKILL). The lock and its TTL are unchanged. A new leader ticks the moment it wins rather than at its poller's next minute. Under repeated leader kills, periodics fired 8-9 of 10 slots, up from 0-1. - A leader's first tick is no longer spent on a stale slot. A slot over the grace is still skipped and logged, and the occurrence due now fires in its place, under the same claim, so nothing fires twice.
- The leader lease survives
maxmemory. Renewal and a new leader's token go throughallow-oomscripts, as the heartbeat does. Dragonfly refuses those, and falls back to what it did: the lease lapses until memory returns. - A leader that fails one renewal retries in 15s, not 60s, while it may still hold its lease.
wurkandwurkswarmprint one line for failures they fully understand: a bad flag, a bad config path, an unreachable Redis. Those traces now need-v. Any other error, including anything raised by the app, still prints its backtrace. Exit codes are unchanged.- A stray word on the command line (
wurk foo) now warns. It still boots as before. - A missing
-rpath is logged at ERROR and named. It was a generic INFO message. - No more
Signal INFO not supportedwarning at every Linux boot. docs/migrate-from-sidekiq.md, the wiki andllms.txtno longer say the railtie starts a swarm in every Rails process or that each clustered Puma worker forks its own. It boots only inside a running web server or withWURK_EMBED=1, and a preforking server refuses with a warning.- The same docs no longer promise a threads-only fallback on JRuby, TruffleRuby and Windows.
wurkswarmand the railtie's swarm callProcess.forkwith no fallback, and no CI leg runs those platforms.
Upgrade notes
- A minutely loop that had been skipped on recovery now fires once on the first tick after an outage. Loops with no occurrence inside the grace behave as before.
- A process that wins the lock ticks at once instead of after up to 60s, including just after boot.
- Followers send about 2 commands per 15-30s (
EVALSHA,PTTL) instead of 1 per 60s. - A Redis restored from a snapshot older than the last fire rewinds
nf, and the occurrence inside the grace can fire a second time. Before this change that held only for a slot within 90s; now also for a slot over it, if an occurrence is due.