Skip to content

v7.0.3

Choose a tag to compare

@mensfeld mensfeld released this 10 Jul 16:54
· 118 commits to main since this release
de22262
  • Feature: Shoryuken.active_job_fifo_message_deduplication to opt out of FIFO dedup id generation (mensfeld)

    • For FIFO queues the ActiveJob adapter derives a content-based message_deduplication_id from the
      serialized job minus job_id/enqueued_at (#457 / #750), so two distinct enqueues of the same job
      class and arguments within SQS's 5-minute window silently collapse into one - a "skipped message" trap
    • Set Shoryuken.active_job_fifo_message_deduplication = false to stop generating that id, so identical
      jobs are no longer silently dropped (rely on the queue's content-based deduplication or explicit ids)
    • Defaults to true, preserving the existing behavior; an explicit message_deduplication_id is still honored
  • Fix: Polling strategies are now thread-safe, and WeightedRoundRobin unpauses processed queues reliably (mensfeld)

    • message_processed runs on processor-completion threads (for FIFO queues) while next_queue/messages_found
      run on the dispatch thread; they mutate the same state with no synchronization, which is benign on MRI (GVL)
      but corrupts state on JRuby/TruffleRuby. Both WeightedRoundRobin and StrictPriority now serialize access
      with a mutex
    • WeightedRoundRobin#unpause_queues only checked the head of the paused list, so a queue marked ready by
      message_processed could stay stuck behind an earlier-paused queue; it now unpauses the first expired entry
      anywhere in the list
  • Fix: TimerTask#kill no longer deadlocks on Ruby 3.2 under concurrent callers (mensfeld)

    • kill called @thread.kill while holding @mutex; the timer loop's ensure block calls
      @mutex.synchronize to clear @running, so on Ruby 3.2 (where Thread#kill yields the GVL
      to the killed thread for cleanup before returning) both threads waited on each other forever
    • The thread is now killed after the mutex is released, so the ensure block can always acquire it
  • Fix: Stopping the launcher no longer destroys the process-global IO executor (mensfeld)

    • With no launcher_executor configured, Launcher#executor fell back to Concurrent.global_io_executor,
      and Launcher#stop/#stop! call shutdown (and kill) on it
    • Shutting down that process-wide pool broke anything else relying on concurrent-ruby's :io pool
      (including Shoryuken's own ShoryukenConcurrentSendAdapter) and prevented starting a fresh launcher
      in the same process
    • The launcher now owns a dedicated Concurrent::CachedThreadPool, so stopping it leaves the global
      IO executor untouched
  • Fix: Queue#delete_messages now logs every batch-delete failure and returns a robust boolean (mensfeld)

    • It used failed.any? { |f| logger.error ... }, which short-circuited after the first failure - so when
      a batch had multiple failures only the first was logged - and only returned truthy because Logger#error
      happens to return true (a custom logger returning falsey would have hidden the failure from callers)
    • It now logs each failure and returns failed.any?, so NonRetryableException/AutoDelete reliably see
      that some messages may remain and need reprocessing
  • Fix: Lifecycle events fired in reverse no longer alternate handler order (mensfeld)

    • Util#fire_event reversed the stored handler array in place with reverse!, so an event fired more
      than once with reverse: true (e.g. :shutdown via stop then stop!) flipped order each time
    • It now reverses a copy, leaving the stored handler order untouched
  • Fix: A fatal dispatch error no longer hard-kills an embedded host process (mensfeld)

    • Manager#handle_dispatch_error sent Process.kill('USR1', Process.pid) unconditionally after a
      dispatch error (e.g. SQS still failing once the fetcher exhausted its retries)
    • The CLI Runner traps USR1 and turns it into a graceful shutdown, but a host embedding
      Shoryuken::Launcher directly has USR1's default disposition, so the whole process - and its
      in-flight workers - was terminated
    • When embedded (no CLI Runner), the failing manager now just stops itself and Launcher#healthy?
      reports the failure; the USR1 signal is only sent in server (CLI) mode, preserving the
      supervisor-restart behavior there
  • Fix: Graceful stop is now bounded by the configured timeout (mensfeld)

    • Launcher#stop (the soft shutdown behind USR1/TSTP) called executor.wait_for_termination with no
      argument - an unbounded wait - so a single hung worker blocked shutdown forever
    • Both Launcher#stop and Launcher#stop! now share a shutdown_executor helper that waits up to
      Shoryuken.options[:timeout] seconds for in-flight workers, then force-kills the executor so the
      process can exit; the graceful stop still waits for workers, just no longer indefinitely
  • Docs: Correct the retry_intervals exponential backoff documentation (mensfeld)

    • The exponential_backoff? docstring claimed retries stop ("before giving up") after the last configured
      interval. They do not: once the intervals are exhausted, the last interval is reused for every later
      attempt, and SQS's redrive policy (maxReceiveCount) is what ultimately moves a message to a dead-letter queue
    • Added a spec pinning that far-later attempts keep reusing the last interval
  • Fix: CurrentAttributes.persist no longer drops a class when called once per class (mensfeld)

    • The storage key was derived from the per-call index, so registering classes across separate persist
      calls made the third call reuse cattr_0 and silently overwrite the second class - its attributes were
      then never serialized or restored
    • The key now uses the running registry size, so incremental and single-call registration both yield
      distinct, stable keys (single-call persist(A, B, C) keys are unchanged)
  • Fix: Busy-processor accounting no longer breaks when processor completion raises (mensfeld)

    • Manager#assign chained .then { processor_done }.rescue { processor_done }, so an exception inside
      processor_done (SQS lookups or a polling strategy's message_processed callback) ran completion twice
    • The busy counter was decremented twice for one message and drifted negative, inflating ready and
      silently breaking the configured concurrency limit for the life of the process
    • Completion now runs in an ensure around processing (exactly once), and processor_done logs
      instead of leaking exceptions from the FIFO bookkeeping
  • Fix: Repeated graceful stop no longer deadlocks the process (mensfeld)

    • Manager#await_dispatching_in_progress popped a signal queue that received exactly one token,
      so a second Launcher#stop blocked forever on an empty queue
    • Operationally this was the TSTP -> USR1 sequence: both signals trigger a graceful stop, leaving
      the process stuck in the signal loop where even TERM/INT were no longer handled (only SIGKILL worked)
    • The dispatch loop now closes the signal queue instead of pushing a token, releasing every pending
      and future waiter; the dispatch chain also releases waiters if the executor rejects a post during
      a racing hard shutdown
  • Fix: non_retryable_exceptions is no longer ignored when retry_intervals is also configured (mensfeld)

    • ExponentialBackoffRetry swallowed every exception after scheduling a retry, so NonRetryableException
      (which sits outside it in the default middleware chain) never saw non-retryable errors - poison messages
      were retried indefinitely (with the last interval repeated) instead of being deleted immediately
    • ExponentialBackoffRetry now re-raises exceptions classified as non-retryable so the message gets deleted
    • Exception classification is extracted to NonRetryableException.non_retryable? and shared by both middlewares
  • Enhancement: Use dynamic Ruby warning category opt-in in test helpers (mensfeld)

    • Replace version-gated Warning[:performance] with Warning.categories-based auto-enablement
    • Automatically enables all non-deprecated, non-experimental warning categories for forward compatibility
    • Applied to both spec/spec_helper.rb and spec/integrations_helper.rb