SolidJobs

SolidJobs is a Ractor-oriented Redis task runner for Ruby. It provides its own task API, Redis envelope, keyspace, interceptor model, failure policy, and inspection API.

SolidJobs is not compatible with Sidekiq. It does not read Sidekiq queues, does not write Sidekiq payloads, and does not expose Sidekiq's job API. Existing applications and queued work must be migrated explicitly.

gem "solid-jobs"

Then run bundle install.

class RecalculateAccount
  include SolidJobs::Task

  task_options channel: "critical", max_failures: 10

  def execute_task(account_id)
    Account.find(account_id).recalculate!
  end
end

RecalculateAccount.enqueue(42)
RecalculateAccount.enqueue_after(30, 42)
RecalculateAccount.schedule_at(Time.now + 300, 42)
RecalculateAccount.enqueue_many([[42], [43], [44]])

The persisted envelope is specific to SolidJobs:

{
  "id": "7ec33ed5-1b77-4bde-8960-bd77231f10ef",
  "task": "RecalculateAccount",
  "arguments": [42],
  "channel": "critical",
  "max_failures": 10,
  "created_ms": 1790981000000,
  "queued_ms": 1790981000001
}

All internal Redis keys use the solid_jobs: namespace. Ready work is stored under solid_jobs:channel:<name>; planned, retrying, discarded, node, claim, attempt, and metric data use separate namespaced keys.

Run executors:

bundle exec solid-jobs --require ./config/environment \
  --concurrency 8 --channel critical,3 --channel default

SolidJobs.configure do |config|
  config.channels = [["critical", 3], "default"]
  config.channel_order = :weighted
  config.concurrency = 8
  config.shutdown_timeout = 25
  config.default_task_options = {
    channel: "default",
    max_failures: 25,
  }
end

channel_order accepts :weighted for bounded weighted round-robin, :priority for declaration-order priority, and :shuffle for random selection from the weighted channel list.

Publication and execution use SolidJobs interceptors. An interceptor implements around(context) and yields to continue:

class TraceExecution
  def around(context)
    Telemetry.start(context.envelope.fetch("id"))
    yield
  ensure
    Telemetry.finish
  end
end

SolidJobs.configure do |config|
  config.execute_interceptors.use(TraceExecution)
end

Publication interceptors receive SolidJobs::Publisher::Publication; execution interceptors receive SolidJobs::Executor::Execution.

Tasks default to 25 failures. A task can customize the delay or final action:

class ImportCatalog
  include SolidJobs::Task

  task_options max_failures: 8, retry_within: 3_600

  retry_delay do |failure_count, error, envelope|
    :drop if error.is_a?(InvalidCatalog)
  end

  after_final_failure do |envelope, error|
    Alerts.catalog_import_failed(envelope.fetch("id"), error)
  end
end

retry_delay may return a delay in seconds, :drop, or :archive. Without an override, SolidJobs uses capped exponential backoff with equal jitter.

SolidJobs provides at-least-once delivery. An executor atomically claims a task before execution and completes the claim only after execute_task returns. Graceful shutdown requeues unfinished tasks. Claims owned by a crashed node are recovered into their original channels.

A process can still crash after an application side effect and before claim completion. The recovered task then runs again. Tasks must be idempotent or use application-level deduplication when duplicate effects are unsafe.

See docs/reliability.md for the state machine, claim fencing, recovery rules, and Ruby Ractor caveats.

node = SolidJobs::Nodes.new.first
node.request_control(:pause)
node.request_control(:backtraces)
node.request_control(:shutdown)

Control requests are asynchronous and use the selected node's namespaced Redis mailbox. Unknown actions raise KeyError without writing a request.

SolidJobs.testing!(:capture) do
  RecalculateAccount.enqueue(42)
  RecalculateAccount.captured
end

SolidJobs.testing!(:execute) do
  RecalculateAccount.enqueue(42)
end

Run the project suites:

bundle exec rake test
STRESS_JOBS=10000 bundle exec rake stress
SOLID_JOBS_TORTURE=1 STRESS_JOBS=100000 bundle exec rake torture
SOLID_JOBS_SOAK=1 SOLID_JOBS_SOAK_SECONDS=86400 bundle exec rake soak

There is no transparent migration path because compatibility is intentionally absent:

1. Replace include Sidekiq::Job with include SolidJobs::Task.
2. Replace sidekiq_options with task_options.
3. Replace perform_async, perform_in, and perform_bulk with enqueue, enqueue_after, and enqueue_many.
4. Replace middleware with SolidJobs interceptors.
5. Drain or export existing Sidekiq queues before switching. SolidJobs will not consume them.
6. Start SolidJobs with --channel; --queue is not accepted.

The Active Job adapter remains available as ActiveJob::QueueAdapters::SolidJobsAdapter, but it writes only SolidJobs envelopes and keys.

SolidJobs exposes an optional Ractor-local instrumenter and defaults to a no-op. Any configured instrumenter must respond to instrument(name, payload) and be Ractor-shareable before a conductor starts. Ordinary instrumentation failures are logged when possible and never change publication, execution, retry, acknowledgement, recovery, or shutdown behavior. Process-control interruptions still propagate so shutdown can requeue in-flight work.

SolidJobs emits the v1 lifecycle events job.enqueued, job.journaled, job.reserved, job.started, job.completed, job.failed, job.retry_scheduled, job.dead, job.acknowledged, and job.recovered. Ready jobs are not reservable until job.enqueued and job.journaled have been emitted; an expiring Redis publication barrier preserves this order without stranding work if a publisher exits. They carry correlation identifiers and structured errors, but never task arguments, business payloads, secrets, or textual logs. node_id is shared by all Ractors in a conductor; ractor_id and worker_id are integer processor identities. reservation_id is the reliable claim token.

The heartbeat Ractor also samples and emits process.observed, one ractor.observed per logical processor, and one redis.observed per processor every five seconds. Process observations are limited to process CPU time, RSS, GC count/time, and total Ruby allocations. Sources are local Ruby/OS reads made outside worker Ractors. A source that has never succeeded is unavailable; a later failure preserves the last value as stale with its increasing age. SolidJobs never estimates CPU, memory, GC, or allocations per Ractor.

Ractor observations expose only state, activity, integer ractor_id/worker_id/executor_id, and current job IDs. Quiet workers report waiting; shutdown reports stopping then stopped without inventing work. Redis connection state comes from worker operations already being performed: successful operations report connected and SolidRedis::ConnectionError reports disconnected. No monitoring-only Redis command is added. Redis latency, command, memory, and connection-count metrics remain explicitly unavailable until a reliable source exists.

Core Rails frameworks like ActiveRecord and ActionMailer rely on global variables, shared connection pools, and mutable global state. Referencing these modules directly inside a strict, isolated Ractor raises ArgumentError: can't share mutable object.

To keep SolidJobs' Ractor-native execution while preserving your existing Rails business logic, use a hybrid execution bridge: delegate mutable work to an isolated sub-process via bin/rails runner. The worker stays Ractor-native and the heavy Rails operations are offloaded to the operating system layer.

Important: These task classes must be loaded explicitly (e.g., inside config/boot.rb) outside of Zeitwerk management, as standard Rails autoloading is restricted inside isolated Ractors.

Trigger an email workflow using your existing Rails mailers and templates without memory isolation crashes:

# frozen_string_literal: true

require "open3"

class SendEmailTask
  include SolidJobs::Task

  # Define your processing channel and failure thresholds
  task_options channel: "default", max_failures: 3

  APP_ROOT = File.expand_path("../..", __dir__).freeze
  SCRIPT = "WelcomeMailer.hello(ARGV[0]).deliver_now"

  def execute_task(recipient = "nicolas.vandenbogaerde@gmail.com")
    output, status = Open3.capture2e(
      { "RAILS_ENV" => ENV.fetch("RAILS_ENV", "development") },
      "bin/rails", "runner", SCRIPT, recipient,
      chdir: APP_ROOT
    )

    unless status.success?
      raise "Email delivery failed: #{output.lines.grep_v(/^\s+from /).first(5).join}"
    end
  end
end

Load database records and run model mutations safely outside of the Ractor:

# frozen_string_literal: true

require "open3"

class ProcessUserDataTask
  include SolidJobs::Task

  # Dedicated data synchronization channel
  task_options channel: "data_sync", max_failures: 3

  APP_ROOT = File.expand_path("../..", __dir__).freeze
  SCRIPT = "User.find(ARGV[0]).update_metrics!"

  def execute_task(user_id)
    output, status = Open3.capture2e(
      { "RAILS_ENV" => ENV.fetch("RAILS_ENV", "development") },
      "bin/rails", "runner", SCRIPT, user_id.to_s,
      chdir: APP_ROOT
    )

    unless status.success?
      raise "ActiveRecord task failed: #{output.lines.grep_v(/^\s+from /).first(5).join}"
    end
  end
end

In complex production monoliths, a single service object often encapsulates multiple ActiveRecord updates, pricing mutations, external API calls, and email notifications. Running this orchestration layer directly inside a Ractor triggers memory isolation failures.

By serializing the service input into a lightweight, Ractor-safe JSON payload, SolidJobs can hand the task details to an isolated bin/rails runner sub-process.

A major benefit is automatic memory recovery: once the service has allocated thousands of transient Ruby objects, the sub-process terminates and the operating system reclaims all of its memory, keeping the worker immune to memory bloat and leaks.

Example: processing a heavy order checkout service.

# frozen_string_literal: true

require "open3"
require "json"

# Loaded explicitly (e.g., in config/boot.rb) outside of Zeitwerk.
# SolidJobs executors run within Ractors where standard Rails autoloading is restricted.
class ExecuteServiceTask
  include SolidJobs::Task

  # Dedicated high-priority business services channel
  task_options channel: "services", max_failures: 2

  APP_ROOT = File.expand_path("../..", __dir__).freeze

  # The runner script parses the input JSON from the command line and executes the service
  SCRIPT = <<~RUBY
    require 'json'
    params = JSON.parse(ARGV[0])
    OrderCheckoutService.call(user_id: params['user_id'], cart_id: params['cart_id'])
  RUBY

  def execute_task(params_hash)
    # Serialize the pure Hash into a strict JSON string for safe Ractor transmission
    json_payload = params_hash.to_json

    output, status = Open3.capture2e(
      { "RAILS_ENV" => ENV.fetch("RAILS_ENV", "development") },
      "bin/rails", "runner", SCRIPT, json_payload,
      chdir: APP_ROOT
    )

    unless status.success?
      raise "Service object execution failed: #{output.lines.first(5).join}"
    end
  end
end

Enqueuing from your Rails application:

# Construct a clean, 100% Ractor-safe arguments structure
task_payload = { user_id: 42, cart_id: 891 }

# Push into the Redis pipeline
ExecuteServiceTask.enqueue(task_payload)

SolidJobs is open source under the LGPL-3.0-or-later license.

Running multi-core clusters blind in staging or production makes monitoring hard. For visual insight into Ractor memory allocation, CPU thresholds, and live queue states, pair this gem with SolidTrace (commercial licenses available from $50 to $200 / month).

For commercial keys, custom architecture audits, or enterprise support, contact licensing@solidtrace.io.

SolidJobs uses Ruby Ractors for parallel execution across real CPU cores, while Sidekiq uses threads within a single Ruby process subject to the Global VM Lock (GVL).

The benchmark suite (benchmark_sidekiq_solid-jobs) measures throughput and resource usage under identical workloads against an isolated Redis server.

Sidekiq remains capped by Ruby's GVL near 1 core (~100% CPU), whereas SolidJobs scales linearly across cores:

At 8 concurrency, SolidJobs delivers 7.2× the throughput of Sidekiq on pure CPU workloads.

At 8 concurrency, SolidJobs delivers 3.1× the throughput of Sidekiq on mixed workloads.

Compared with SolidJobs 0.4.0 on the same benchmark matrix, individual enqueue throughput improved by 2.2–4.3%. Bulk throughput at 8 concurrency improved by 3.5%. CPU scaling efficiency at 8 Ractors improved from 94.0% to 96.2%.

• Peak RSS: Comparable processing footprint at 8 concurrency (42.5 MiB for SolidJobs and 41.9 MiB for Sidekiq on CPU tasks).
• Allocations: SolidJobs produces fewer object allocations per task (48 vs 108 allocations/job on CPU tasks, 41 vs 60 on individual enqueue), reducing GC pressure. CPU-task allocations fell by 36% from SolidJobs 0.4.0.

Environment: Ruby 4.0.1 (arm64-darwin25); Sidekiq 8.1.7; SolidJobs 0.4.1. Fresh process per measurement; medians of six runs; zero benchmark errors.