Complete Queue lifecycle safety, durability, and Laravel parity

AGENTS.md

@@ -167,7 +167,7 @@ Build complete, long-term solutions, not MVPs or local workarounds. A broad chan
 - **Use `Sleep::usleep()` / `Sleep::sleep()` for delays in source code** — `Sleep` is fakeable in tests. Use raw `sleep()` / `usleep()` only where real time must pass, such as test harnesses and external-process polling.
 - **Use `xxh128` for internal non-cryptographic hashing** — cache and context keys, content checksums, and change detection. It is faster than `sha256`, which is reserved for trust boundaries: stored credential digests, signatures, and anything an attacker gains by forging. Seed it when the hashed value comes from user input, as `SwooleStore` does for its physical table keys.
 - **Use immutable dates by default** — Hypervel defaults to `Hypervel\Support\CarbonImmutable`, including where Laravel uses mutable Carbon. Create public or application-configurable dates through the `Date` facade or date helpers, and use exact `CarbonImmutable` for framework-owned internal or held values. Type configurable Carbon boundaries as `CarbonInterface` and native or third-party boundaries as `DateTimeInterface`. Capture the return value of every date modifier whose result must persist. Use `Hypervel\Support\Carbon` only for explicit mutable opt-out or conversion behavior.
-- **Use typed config getters, no call-site defaults** — prefer `$config->string()`, `$config->integer()`, `$config->float()`, `$config->boolean()`, `$config->array()` over `$config->get()` for any key that can't legitimately be null. Typed getters fail fast on misconfiguration — an `InvalidArgumentException` naming the key, instead of a wrong type propagating silently — and give phpstan the real type. Defaults live in config files: `LoadConfiguration` merges the framework base config, and package providers must merge their defaults through `mergeConfigFrom()`. A call-site default is then dead code that can drift from the real default. Declare every key in the package's config file and call the getter without a default — a missing key then fails loudly instead of being silently papered over. Exceptions: bootstrap code that runs before or without config merging (e.g. `LoadConfiguration` itself), and genuinely nullable keys, which keep `get()`.
+- **Use typed config getters and avoid duplicate defaults** — prefer `$config->string()`, `$config->integer()`, `$config->float()`, `$config->boolean()`, and `$config->array()` over `$config->get()` for values that cannot be null. Framework and package defaults are shallow-merged with application config. `mergeableOptions()` is only for named groups such as connections or stores: application entries replace matching defaults, while other default entries remain. Other nested arrays are replaced as a whole. Keep a fallback when a setting inside one of those replaced arrays is intentionally optional.
 - **Env var naming** — ported config keeps its upstream env var names. New Hypervel-specific keys start with the config file's domain prefix (`SERVER_`, `APP_`) with the clearest name for the rest, e.g. `SERVER_WORKERS` for the `Constant::OPTION_WORKER_NUM` server option. If a config value mirrors another config key, reuse that key's env var instead of defining a duplicate — e.g. the Slack log channel username falls back to `APP_NAME`.
 - **Use `resolve...Using` for Hypervel-owned config resolvers** — prefer this naming for callbacks that resolve config-derived values, unless an established Laravel domain convention already exists, such as `redirectUsing()`.
 - **Always use American English spelling** — E.g., "behavior" vs "behaviour", "utilize" vs "utilise".

src/boost/docs/queues.md

@@ -200,6 +200,37 @@ Adjusting this value based on your queue load can be more efficient than continu
 > [!WARNING]
 > Setting `block_for` to `0` will cause queue workers to block indefinitely until a job is available. This can delay worker restart and pause checks until the next job has been processed.
 
+<a name="sqs-overflow-storage"></a>
+#### SQS Overflow Storage
+
+Amazon SQS limits the maximum size of a queued message payload. If you need to dispatch jobs with payloads that may exceed this limit, you may configure Hypervel to store oversized SQS payloads in a cache store and send a pointer through SQS instead. To enable this feature, add an `overflow` array to your SQS queue connection configuration:
+
+```php
+'sqs' => [
+    'driver' => 'sqs',
+    'key' => env('AWS_ACCESS_KEY_ID'),
+    'secret' => env('AWS_SECRET_ACCESS_KEY'),
+    'prefix' => env('SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/your-account-id'),
+    'queue' => env('SQS_QUEUE', 'default'),
+    'suffix' => env('SQS_SUFFIX'),
+    'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
+    'after_commit' => false,
+    'overflow' => [
+        'enabled' => env('SQS_OVERFLOW_ENABLED', false),
+        'store' => env('SQS_OVERFLOW_STORE'),
+        'always' => false,
+        'delete_after_processing' => true,
+        'flush_on_clear' => env('SQS_OVERFLOW_FLUSH_ON_CLEAR', false),
+    ],
+],
+```
+
+When overflow storage is enabled, Hypervel will store payloads that are at least 1 MB in the configured cache store. If the `always` option is `true`, every SQS payload will be stored in the cache store regardless of its size. Since queued jobs will need to retrieve their payloads from the cache store when they are processed, you should choose a store that can retain the payloads until your workers process them. By default, stored payloads are deleted after their jobs have been successfully processed and deleted from SQS.
+
+If the `flush_on_clear` option is `true`, the configured overflow cache store will be flushed when the `queue:clear` command clears the SQS queue. Since flushing a cache store may remove all items from that store, you should configure SQS overflow storage to use a dedicated cache store when enabling this option.
+
+SQS may accept a message even when publication reports an error, so Hypervel keeps its overflow payload; payloads may also remain when a message expires before processing or a queue is cleared without `flush_on_clear`. You should use a dedicated store, monitor its growth, and make sure it will not evict these non-expiring payloads under memory or capacity pressure, since removing a live payload will fail its job.
+
 <a name="other-driver-prerequisites"></a>
 #### Other Driver Prerequisites
 
@@ -900,6 +931,16 @@ public function middleware(): array
 }
 ```
 
+The `backoff` method also accepts a closure, allowing the delay to be determined from the exception:
+
+```php
+use Throwable;
+
+return [(new ThrottlesExceptions(10, 5 * 60))->backoff(
+    fn (Throwable $exception) => $exception->getCode() === 429 ? 5 : 1
+)];
+```
+
 Internally, this middleware uses Hypervel's cache system to implement rate limiting, and the job's class name is utilized as the cache "key". You may override this key by calling the `by` method when attaching the middleware to your job. This may be useful if you have multiple jobs interacting with the same third-party service and you would like them to share a common throttling "bucket" ensuring they respect a single shared limit:
 
 ```php
@@ -1011,6 +1052,45 @@ The `connection` method may be used to specify which Redis connection the middle
 return [(new ThrottlesExceptionsWithRedis(10, 10 * 60))->connection('limiter')];
 ```
 
+<a name="releasing-jobs"></a>
+### Releasing Jobs
+
+The `Release` middleware allows you to release a job back onto the queue without executing it. The `Release::when` method will release the job if the given condition evaluates to `true`, while the `Release::unless` method will release the job if the condition evaluates to `false`:
+
+```php
+use Hypervel\Queue\Middleware\Release;
+
+/**
+ * Get the middleware the job should pass through.
+ */
+public function middleware(): array
+{
+    return [
+        Release::when($condition, releaseAfter: 60),
+    ];
+}
+```
+
+Releasing a job back onto the queue will still increment the job's total number of attempts. You may wish to tune your `Tries` and `MaxExceptions` attributes on your job class accordingly.
+
+You can also pass a closure to the `when` and `unless` methods for more complex conditional evaluation:
+
+```php
+use Hypervel\Queue\Middleware\Release;
+
+/**
+ * Get the middleware the job should pass through.
+ */
+public function middleware(): array
+{
+    return [
+        Release::when(function (): bool {
+            return ! $this->order->isPaid();
+        }, releaseAfter: 60),
+    ];
+}
+```
+
 <a name="skipping-jobs"></a>
 ### Skipping Jobs
 
@@ -2571,6 +2651,14 @@ The `--stop-when-empty` option may be used to instruct the worker to process all
 php artisan queue:work --stop-when-empty
 ```
 
+The `--stop-when-empty-for` option may be used to keep the worker alive until the queue has remained empty for a given number of seconds. The timer begins when the worker starts and resets whenever a job finishes:
+
+```shell
+php artisan queue:work --stop-when-empty-for=30
+```
+
+The worker will wait for any jobs it is already processing before exiting.
+
 <a name="processing-jobs-for-a-given-number-of-seconds"></a>
 #### Processing Jobs for a Given Number of Seconds
 
@@ -2870,6 +2958,18 @@ To store failed jobs in a file, set the failed job driver to `file`. If no custo
 QUEUE_FAILED_DRIVER=file
 ```
 
+You may customize the file path and maximum number of retained failed jobs in your `config/queue.php` configuration file:
+
+```php
+'failed' => [
+    'driver' => env('QUEUE_FAILED_DRIVER', 'file'),
+    'path' => storage_path('framework/cache/failed-jobs.json'),
+    'limit' => 100,
+],
+```
+
+Malformed queue payloads are stored as failed jobs and removed from their queue instead of being retried indefinitely. When using Horizon with the Redis queue, Horizon must decode a raw payload before it can decorate and publish it; malformed payloads are therefore failed without Horizon telemetry, while the original payload remains available in failed-job storage.
+
 When running a [queue worker](#running-the-queue-worker) process, you may specify the maximum number of times a job should be attempted using the `--tries` switch on the `queue:work` command. If you do not specify a value for the `--tries` option, jobs will only be attempted once or as many times as specified by the job class' `Tries` attribute:
 
 ```shell
@@ -2997,6 +3097,12 @@ To view all of the failed jobs that have been inserted into your `failed_jobs` d
 php artisan queue:failed
 ```
 
+To return the failed jobs as JSON, pass the `--json` option:
+
+```shell
+php artisan queue:failed --json
+```
+
 The `queue:failed` command will list the job ID, connection, queue, failure time, and other information about the job. The job ID may be used to retry the failed job. For instance, to retry a failed job that has an ID of `ce7bb17c-cdd8-41f0-a8ec-7b4fef4e5ece`, issue the following command:
 
 ```shell
@@ -3146,6 +3252,31 @@ php artisan queue:clear redis --queue=emails
 > [!WARNING]
 > Clearing jobs from queues is only available for the SQS, Redis, and database queue drivers. In addition, the SQS message deletion process takes up to 60 seconds, so jobs sent to the SQS queue up to 60 seconds after you clear the queue might also be deleted.
 
+<a name="inspecting-jobs"></a>
+## Inspecting Jobs
+
+The database and Redis queue drivers allow you to inspect pending, delayed, and reserved jobs without removing them from the queue:
+
+```php
+use Hypervel\Support\Facades\Queue;
+
+$queue = Queue::connection('redis');
+
+$pending = $queue->pendingJobs('emails');
+$delayed = $queue->delayedJobs('emails');
+$reserved = $queue->reservedJobs('emails');
+```
+
+You may inspect jobs across every queue on the connection using the corresponding `allPendingJobs`, `allDelayedJobs`, and `allReservedJobs` methods:
+
+```php
+$pending = $queue->allPendingJobs();
+$delayed = $queue->allDelayedJobs();
+$reserved = $queue->allReservedJobs();
+```
+
+These methods load every matching job into memory. Avoid using them against very large backlogs in latency-sensitive code.
+
 <a name="monitoring-your-queues"></a>
 ## Monitoring Your Queues
 
@@ -3157,6 +3288,12 @@ To get started, you should schedule the `queue:monitor` command to [run every mi
 php artisan queue:monitor redis:default,redis:deployments --max=100
 ```
 
+To return the queue sizes as JSON, pass the `--json` option:
+
+```shell
+php artisan queue:monitor redis:default,redis:deployments --max=100 --json
+```
+
 Scheduling this command alone is not enough to trigger a notification alerting you of the queue's overwhelmed status. When the command encounters a queue that has a job count exceeding your threshold, an `Hypervel\Queue\Events\QueueBusy` event will be dispatched. You may listen for this event within your application's `AppServiceProvider` in order to send a notification to you or your development team:
 
 ```php
@@ -3604,3 +3741,16 @@ Queue::looping(function () {
     }
 });
 ```
+
+Hypervel also dispatches a `Hypervel\Queue\Events\WorkerIdle` event when a queue worker is unable to retrieve a job from the queue:
+
+```php
+use Hypervel\Queue\Events\WorkerIdle;
+use Hypervel\Support\Facades\Event;
+
+Event::listen(function (WorkerIdle $event) {
+    // $event->connectionName
+    // $event->queue
+    // $event->workerOptions
+});
+```

src/boost/docs/redis.md

@@ -548,6 +548,8 @@ The `flushByPattern` method deletes all keys matching a pattern using Redis' `SC
 $deleted = Redis::flushByPattern('users:*');
 ```
 
+When calling `flushByPattern` on an already-held connection, acquire the connection with `transform: false`.
+
 <a name="transactions"></a>
 ### Transactions
 
@@ -610,7 +612,7 @@ Redis::pipeline(function (\Redis $pipe): void {
 > In Hypervel, the closure form of `pipeline` returns the pooled connection as soon as the pipeline is executed. Calling `Redis::pipeline()` without a closure pins a connection for the rest of the coroutine, so the closure form is preferred for application code.
 
 > [!WARNING]
-> The native `reset()` command is not available on pooled connections because it clears authentication and selected database state owned by the pool. Use `discard()` to abandon a transaction, `unwatch()` to stop watching keys, or `exec()` to complete a pipeline.
+> The native `reset()` command is not available on pooled connections because it clears authentication and selected database state owned by the pool. For facade-managed connections, use `Redis::discard()`, `Redis::unwatch()`, or `Redis::exec()`. Within a `withConnection` callback, call `discardTransaction()`, `unwatch()`, or `exec()` on the connection passed to the callback. Calling `discard()` on a held connection removes it from the pool instead of abandoning its Redis transaction.
 
 <a name="advanced-helpers"></a>
 ### Advanced Helpers
@@ -623,7 +625,7 @@ use Hypervel\Support\Facades\Redis;
 
 $keys = Redis::withConnection(function (RedisConnection $connection): array {
     return iterator_to_array($connection->safeScan('users:*'));
-});
+}, transform: false);
 ```
 
 The `evalWithShaCache` method executes a Lua script using `evalSha` and automatically falls back to `eval` when Redis has not cached the script yet:

src/contracts/src/Broadcasting/Broadcaster.php

@@ -5,7 +5,6 @@
 namespace Hypervel\Contracts\Broadcasting;
 
 use Hypervel\Http\Request;
-use Hypervel\Support\Collection;
 
 interface Broadcaster
 {
@@ -23,9 +22,4 @@ public function validAuthenticationResponse(Request $request, mixed $result): mi
      * Broadcast the given event.
      */
     public function broadcast(array $channels, string $event, array $payload = []): void;
-
-    /**
-     * Get all of the registered channels.
-     */
-    public function getChannels(): Collection;
 }

src/contracts/src/Notifications/Factory.php

@@ -4,7 +4,6 @@
 
 namespace Hypervel\Contracts\Notifications;
 
-use Hypervel\Support\Collection;
 use UnitEnum;
 
 interface Factory
@@ -17,10 +16,10 @@ public function channel(UnitEnum|string|null $name = null): mixed;
     /**
      * Send the given notification to the given notifiable entities.
      */
-    public function send(array|Collection $notifiables, mixed $notification): void;
+    public function send(mixed $notifiables, mixed $notification): void;
 
     /**
      * Send the given notification immediately.
      */
-    public function sendNow(array|Collection $notifiables, mixed $notification): void;
+    public function sendNow(mixed $notifiables, mixed $notification): void;
 }

src/contracts/src/Queue/Queue.php

@@ -15,7 +15,7 @@ interface Queue
     public function size(?string $queue = null): int;
 
     /**
-     * Get the current queue workload for the application.
+     * Get the number of pending jobs.
      */
     public function pendingSize(?string $queue = null): int;
 

src/database/src/ConnectionInterface.php

@@ -109,7 +109,7 @@ public function commit(): void;
     /**
      * Rollback the active database transaction.
      */
-    public function rollBack(): void;
+    public function rollBack(?int $toLevel = null): void;
 
     /**
      * Get the number of active transactions.

src/foundation/config/queue.php

@@ -88,6 +88,13 @@
             'suffix' => env('SQS_SUFFIX'),
             'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
             'after_commit' => false,
+            'overflow' => [
+                'enabled' => env('SQS_OVERFLOW_ENABLED', false),
+                'store' => env('SQS_OVERFLOW_STORE'),
+                'always' => false,
+                'delete_after_processing' => true,
+                'flush_on_clear' => env('SQS_OVERFLOW_FLUSH_ON_CLEAR', false),
+            ],
             'pool' => [
                 'min_retained_objects' => 1,
                 'max_objects' => 10,
@@ -133,7 +140,7 @@
     | can control how and where failed jobs are stored. Hypervel ships with
     | support for storing failed jobs in a simple file or in a database.
     |
-    | Supported drivers: "database-uuids", "file", "null"
+    | Supported drivers: "database", "database-uuids", "file", "null"
     |
     */
 

src/foundation/src/Bootstrap/LoadConfiguration.php

@@ -148,7 +148,11 @@ protected function loadConfigurationFile(RepositoryContract $repository, string
     }
 
     /**
-     * Get the options within the configuration file that should be merged again.
+     * Get configuration arrays whose entries should be merged by name.
+     *
+     * For example, an application Redis connection replaces the framework Redis
+     * connection, while framework connections not defined by the application
+     * remain. Nested arrays not listed here are replaced completely.
      *
      * @return array<int, string>
      */

src/foundation/src/Console/ChannelListCommand.php

@@ -34,6 +34,7 @@ class ChannelListCommand extends Command
      */
     public function handle(Broadcaster $broadcaster): void
     {
+        // @phpstan-ignore method.notFound (channel listing is an optional concrete broadcaster capability)
         $channels = $broadcaster->getChannels();
 
         if (! $channels->count()) {

src/horizon/src/Events/JobsMigrated.php

@@ -5,6 +5,7 @@
 namespace Hypervel\Horizon\Events;
 
 use Hypervel\Horizon\JobPayload;
+use Hypervel\Queue\InvalidPayloadException;
 use Hypervel\Support\Collection;
 
 class JobsMigrated
@@ -30,8 +31,12 @@ class JobsMigrated
     public function __construct(array $payloads)
     {
         $this->payloads = collect($payloads)->map(function ($job) {
-            return new JobPayload($job);
-        });
+            try {
+                return new JobPayload($job);
+            } catch (InvalidPayloadException) {
+                return null;
+            }
+        })->filter()->values();
     }
 
     /**