Skip to content
David Ciulla edited this page Nov 7, 2026 · 2 revisions

Queues

Laravex's queue system allows you to defer time-consuming tasks to be processed in the background. The queue worker is an OTP GenServer in Laravex.Queue.Worker and supports four connection drivers: sync, database, deferred, and redis.


Configuration

# config/queue.exs
%{
  "default" => Laravex.Env.get("QUEUE_CONNECTION", "database"),
  "connections" => %{
    "sync"     => %{"driver" => "sync"},
    "database" => %{"driver" => "database", "queue" => "default"},
    "deferred" => %{"driver" => "deferred"},
    "redis"    => %{"driver" => "redis", "connection" => "default", "queue" => "default"}
  }
}
Connection Behaviour
sync Executes jobs immediately in the current process (useful for local development).
database Persists jobs to the jobs table. A worker process (queue:work) consumes them.
deferred Stores jobs in an in-memory ETS table. Jobs run when the next worker poll fires (queue:work).
redis Pushes jobs via LPUSH and pops them via BRPOP over a plain TCP RESP connection.

Default: QUEUE_CONNECTION=database.


Redis Connection

To use the redis driver set the following environment variables:

QUEUE_CONNECTION=redis
REDIS_HOST=127.0.0.1
REDIS_PORT=6379

Jobs are stored as JSON payloads in a Redis list key laravex:queue:{queue_name}. The worker calls BRPOP with a 1-second timeout to block until a job arrives.

Limitations: no TLS, no Cluster, no password auth over the wire. The Redis driver uses :gen_tcp with the RESP protocol directly — no Hex package required.


Defining Jobs

Create a job module with Laravex.Queue.Job:

./artisan make:job ProcessPodcast
defmodule App.Jobs.ProcessPodcast do
  use Laravex.Queue.Job

  @impl true
  def handle([podcast_id]) do
    # Time-consuming work here
    App.Services.PodcastProcessor.process(podcast_id)
  end
end

use Laravex.Queue.Job injects:

  • should_queue?/0 — returns true by default
  • dispatch/2 — push this job onto the configured queue
  • dispatch_sync/1 — run this job immediately in the current process

To opt out of the queue entirely (always run synchronously), pass queue: false:

use Laravex.Queue.Job, queue: false

Dispatching Jobs

# Push to the default queue
App.Jobs.ProcessPodcast.dispatch([podcast.id])

# Push with options
App.Jobs.ProcessPodcast.dispatch([podcast.id],
  queue:      "high",
  delay:      60,       # seconds before the job becomes available
  tries:      3,        # maximum attempts before moving to failed_jobs
  backoff:    30        # seconds to wait between retries
)

# Run synchronously regardless of configuration
App.Jobs.ProcessPodcast.dispatch_sync([podcast.id])

Via Laravex.Queue.Worker.push/3

Laravex.Queue.Worker.push(App.Jobs.ProcessPodcast, [podcast.id])
Laravex.Queue.Worker.push(App.Jobs.ProcessPodcast, [podcast.id],
  connection: "redis",
  queue: "default"
)

ShouldQueue for Event Listeners

Event listeners can be queued by using the Laravex.Events.ShouldQueue macro. See Events for details.


Running the Worker

# Process jobs continuously (polls every second when queue is empty)
./artisan queue:work

# Process a single job and exit
./artisan queue:work --once

The worker:

  1. Drains the deferred ETS table (runs any in-memory jobs whose available_at has passed).
  2. For the database driver: queries the jobs table for the next available job (available_at <= now AND reserved_at IS NULL).
  3. For the redis driver: calls BRPOP on the configured list key.
  4. Marks the job as reserved, executes job_mod.handle(args), then deletes it on success.
  5. On failure: retries up to tries times with backoff seconds between attempts; on final failure, moves the job to failed_jobs.

Failed Jobs

When a job exhausts its retry attempts, it is inserted into the failed_jobs table with its exception message.

# List all failed jobs
./artisan queue:failed

# Retry a specific job by ID or UUID
./artisan queue:retry {id|uuid}

# Delete a failed job
./artisan queue:forget {id|uuid}

# Delete all failed jobs
./artisan queue:flush

Programmatic access:

Laravex.Queue.Worker.failed_jobs()
Laravex.Queue.Worker.retry_failed("some-uuid")
Laravex.Queue.Worker.forget_failed("some-uuid")
Laravex.Queue.Worker.flush_failed()

Job Chains

Laravex.Bus.chain/1 runs a list of jobs sequentially and synchronously — each job must complete before the next begins. If any job raises, the chain stops.

Laravex.Bus.chain([
  {App.Jobs.OptimizeAudio, [podcast.id]},
  {App.Jobs.TranscribePodcast, [podcast.id]},
  {App.Jobs.NotifySubscribers, [podcast.id]}
])

Each element is either {JobModule, args} or a bare JobModule (with no arguments).


Job Batches

Laravex.Bus.batch/2 dispatches a group of jobs tracked together in the job_batches table. The batch struct is returned immediately; jobs are pushed to the configured queue.

./artisan migrate   # creates the job_batches table
batch = Laravex.Bus.batch(
  [
    {App.Jobs.ImportRow, [1]},
    {App.Jobs.ImportRow, [2]},
    {App.Jobs.ImportRow, [3]}
  ],
  name: "CSV Import"
)

# batch.id         => "uuid-string"
# batch.total_jobs => 3
# batch.pending_jobs => 3

Batch Options

Option Type Description
:name string Human-readable batch name shown in queue:batches
:then {mod, fun, args} MFA called when all jobs succeed
:catch {mod, fun, args} MFA called when any job fails
:finally {mod, fun, args} MFA always called when the batch finishes
:connection string Queue connection for all batch jobs
:queue string Queue name for all batch jobs
:tries integer Max attempts per job
:delay integer Seconds before jobs become available
batch = Laravex.Bus.batch(
  [{App.Jobs.ImportRow, [n]} for n <- 1..100],
  name:    "Big Import",
  then:    {App.Services.Notifier, :import_complete, []},
  catch:   {App.Services.Notifier, :import_failed, []},
  finally: {App.Services.Metrics, :record_batch, []}
)

Callbacks receive the %Laravex.Bus.Batch{} struct as their first argument, followed by the extra args from the MFA tuple.

Attaching Callbacks After Dispatch

Laravex.Bus.then(batch, {App.Jobs.Notify, :handle, []})
Laravex.Bus.catch_callback(batch, {App.Jobs.Alert, :handle, []})
Laravex.Bus.finally(batch, {App.Jobs.Cleanup, :handle, []})

Inspecting Batches

./artisan queue:batches
./artisan queue:batch {id}
./artisan queue:batch cancel {id}
Laravex.Bus.find_batch("batch-uuid")
# => %Laravex.Bus.Batch{
#      id: "…", name: "CSV Import",
#      total_jobs: 3, pending_jobs: 1, failed_jobs: 0,
#      cancelled_at: nil, finished_at: nil, …
#    }

Laravex.Bus.list_batches()     # most recent 50
Laravex.Bus.cancel(batch)      # or cancel("batch-uuid")
Laravex.Bus.cancelled?(batch)  # => true | false

Dispatching via Laravex.Bus

Laravex.Bus also exposes standalone dispatch/3 and dispatch_sync/2 functions equivalent to calling through the worker:

Laravex.Bus.dispatch(App.Jobs.ProcessPodcast, [podcast.id])
Laravex.Bus.dispatch_sync(App.Jobs.ProcessPodcast, [podcast.id])

The bus/0 Helpers returns Laravex.Bus:

import Laravex.Support.Helpers
bus().dispatch(App.Jobs.ProcessPodcast, [podcast.id])

See Also

  • Events — queued listeners using Laravex.Events.ShouldQueue
  • Artisan-Console — queue:work, queue:failed, queue:batches, make:job
  • Database — jobs, failed_jobs, job_batches tables
  • Limitations — Redis transport constraints

Clone this wiki locally