NativePHP Queue System with background processing

[dircm]

Overview

This PR introduces a queue processing system that allows NativePHP Mobile apps to dispatch and process Laravel jobs both while the app is active (foreground) and when it's in the background. Jobs are stored in the native SQLite database and processed by native coordinators (Swift/Kotlin) that automatically hand off between foreground and background processing based on app lifecycle.

Why This Matters

Mobile apps need background job processing for tasks like syncing data with remote servers, processing uploads/downloads, deferring processing and more..

Traditional Laravel queue workers dont currently run on nativephp mobile. This implementation provides a mobile-native solution that:

• Uses Laravel's familiar job dispatch API (MyJob::dispatch())
• Stores jobs persistently in SQLite
• Processes jobs without blocking UI interactions
• Continues processing jobs in the background using platform-native processing
• Dispatches events back to Livewire when jobs complete for real-time UI updates

Architecture

Features

1. One job at a time: Jobs are processed sequentially to avoid concurrent PHP execution (PHP is single-threaded in the embedded context)

2. Serial queue on iOS: All PHP execution (WebView requests + queue work) uses a shared DispatchQueue to prevent crashes from concurrent PHP access

3. Polling with notifications: The coordinator polls for jobs but also receives immediate notifications when new jobs are dispatched, reducing latency

4. Configurable throttling: Batch limits and delays prevent queue processing from starving UI responsiveness

5. Automatic lifecycle handoff: When the app backgrounds, the foreground coordinator pauses and schedules a background task. When foregrounding, background work is cancelled and foreground processing resumes. NOTE: Backgrounded tasks can be run very slowly ~15mins on an idle ios device or not at all if the device is doing other things..

Laravel queue features
Fully Supported ✅

• MyJob::dispatch() - Standard dispatch API
• ->delay() - Delayed jobs stored with available_at timestamp
• $tries property - Retries respected, fails after max attempts
• $backoff property - Backoff delays via release()
• $maxExceptions - Inherited from DatabaseQueue
• Failed job storage - Uses standard failed_jobs table
• Retry failed jobs - Via /_native/queue/retry/{id} endpoint
• Multiple queues - Pass queue name in dispatch
• Job chaining - Works via payload serialization
• Job middleware - Executes during $job->fire()
• Queue events - JobProcessing, JobProcessed, JobFailed dispatched
• after_commit - Configured in queue connection
• Encrypted payloads - If Laravel encryption configured

Partially Supported ⚠️

• Job batching - Requires job_batches table migration, untested
• $timeout - PHP embedded context doesn't enforce timeouts reliably
• Unique jobs (ShouldBeUnique) - Requires cache locks, depends on cache driver
• Rate limiting (RateLimited middleware) - Requires cache driver
• WithoutOverlapping - Requires cache locks

Not Supported ❌

• Laravel Horizon - Requires Redis + background daemon
• queue:work command - Native coordinator replaces this
• Queue priorities - FIFO only, no priority ordering
• Worker CLI flags - Not applicable
• queue:listen - Handled by native coordinator

File Structure

PHP Components

File	Purpose
src/Queue/NativeQueue.php	Extends DatabaseQueue, notifies native layer on job push
src/Queue/NativeQueueConnector.php	Laravel queue connector for the native driver
src/Queue/NativeQueueController.php	HTTP endpoints for queue operations (/work, /status, /retry)
src/Queue/NativeQueueServiceProvider.php	Registers the queue driver and internal routes
src/Queue/Events/JobCompleted.php	Event dispatched when a job completes successfully
src/Queue/Events/JobFailed.php	Event dispatched when a job fails
src/Queue/Events/QueueEmpty.php	Event dispatched when queue becomes empty

iOS (Swift) Components

File	Purpose
Queue/NativeQueueCoordinator.swift	Foreground coordinator that manages job processing lifecycle
Queue/BackgroundQueueWorker.swift	BGTaskScheduler integration for background processing
Bridge/Functions/QueueFunctions.swift	Bridge function Queue.JobsAvailable

Android (Kotlin) Components

File	Purpose
queue/NativeQueueCoordinator.kt	Foreground coordinator that manages job processing lifecycle
queue/BackgroundQueueWorker.kt	WorkManager CoroutineWorker for background processing
bridge/functions/QueueFunctions.kt	Bridge function Queue.JobsAvailable

Usage

1. Configure Queue Connection

In your app's .env:

QUEUE_CONNECTION=native

In config/queue.php, add the native connection:

'native' => [
    'driver' => 'native',
    'table' => 'jobs',
    'queue' => 'default',
    'retry_after' => 60,
],

2. Dispatch Jobs

Use Laravel's standard job dispatch API:

MyJob::dispatch($data);

// Or with delay
MyJob::dispatch($data)->delay(now()->addMinutes(5));

3. Listen for Events in Livewire

use Livewire\Attributes\On;

#[On('native:Native\Mobile\Queue\Events\JobCompleted')]
public function onJobCompleted($jobId, $jobName, $queue, $durationMs, $pending)
{
    // Update UI with job completion
}

#[On('native:Native\Mobile\Queue\Events\JobFailed')]
public function onJobFailed($jobId, $jobName, $error, $willRetry, $attempts, $pending)
{
    // Handle job failure
}

#[On('native:Native\Mobile\Queue\Events\QueueEmpty')]
public function onQueueEmpty($queue, $processedCount)
{
    // All jobs processed
}

4. Optional Configuration

In config/nativephp.php:

'queue' => [
    // Foreground processing
    'min_delay' => 100,        // ms between jobs
    'batch_size' => 10,        // jobs per batch before yielding
    'poll_interval' => 2000,   // ms to poll when empty

    // Background processing
    'background' => [
        'enabled' => true,
        'interval' => 15,              // minutes (Android minimum)
        'max_jobs_per_session' => 50,  // jobs per background session
    ],
],

---

Testing

Testing Background Processing

iOS (Xcode Debugger):

While the app is running in Xcode, use the debugger console to simulate a background task:

e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"<bundle-id>.queue-processing"]

Replace <bundle-id> with your app's bundle identifier (e.g., com.myapp.example).

Android (ADB):

Monitor background processing via logcat:

adb logcat | grep -E "(BackgroundQueueWorker|MainActivity)"

Look for:

• App stopped - scheduled background queue processing
• Processed job X: JobClassName
• App started - resumed foreground queue processing

Manual Testing Flow

1. Open your app
2. Dispatch several jobs (e.g., 5-10 jobs)
3. Immediately background the app before jobs complete
4. Wait for background processing (or use simulator commands above)
5. Return to foreground - completed jobs should appear in UI
6. Verify foreground processing resumes for new jobs

---

Limitations

1. Background timing is system-controlled: iOS and Android decide when to run background tasks based on battery, device state, and usage patterns. Jobs will eventually process, but not immediately.

2. No priority queues: All jobs processed in FIFO order. Multiple queue support exists but priority is not implemented.

3. pauseDuringScroll not implemented: The coordinator has this config option but scroll detection is not wired up yet.