systems background jobs

Background jobs

Active contributors: Douwe de Vries

Purpose

Background jobs retain long-running desktop work behind short Tauri command calls. The system supports CSV anonymization progress and cancellation, Local AI model download progress and cancellation, terminal-state cleanup, and panic-to-failure conversion.

Directory layout

Path	Role
src-tauri/src/job_registry.rs	Generic retained job registry and lifecycle primitives.
src-tauri/src/jobs.rs	CSV anonymization job status, store, cancellation, progress, and worker execution.
src-tauri/src/local_ai/download.rs	Local AI model download job status, store, progress parsing, and worker execution.
src-tauri/src/commands/job_commands.rs	Tauri commands for starting, polling, and canceling anonymization jobs.
src-tauri/src/commands/local_ai_commands.rs	Tauri commands for Local AI status, model download jobs, cancellation, and setup URL.

Key abstractions

• JobRegistry<J> stores retained jobs by generated IDs and prunes old terminal jobs.
• JobLifecycle<S> owns status snapshots, cancellation flags, terminal timestamps, and thread-safe updates.
• AnonymizeJobStore creates jobs with job-<pid>-<sequence> IDs and retains AnonymizeJobStatus.
• LocalAiDownloadStore creates jobs with ai-model-<pid>-<sequence> IDs and retains LocalAiDownloadStatus.
• ProcessControl bridges the core CSV streamer with progress callbacks and cancellation checks.
• Terminal states are succeeded, failed, and canceled; terminal snapshots are removed after retrieval or pruned by retention policy.

How it works

stateDiagram-v2
    [*] --> Running
    Running --> Succeeded: worker returns result
    Running --> Failed: worker returns error or panics
    Running --> Canceled: cancel flag observed
    Succeeded --> [*]: snapshot removes terminal job
    Failed --> [*]: snapshot removes terminal job
    Canceled --> [*]: snapshot removes terminal job

Loading

For CSV output, start_anonymize_job authorizes paths, creates a retained job, spawns blocking work, and immediately returns the initial status. run_anonymize_job calls the core service with ProcessControl; each processed row updates rowsProcessed, and cancellation is read from the lifecycle flag. The frontend polls until the status is terminal.

For Local AI downloads, start_local_ai_model_download verifies Ollama availability, creates a retained download job, and streams /api/pull progress lines. Each parsed line updates status text and optional byte counts. Cancellation asks the worker to stop, then reports canceled.

Integration points

• Frontend workflow state polls anonymization jobs through useAnonymizeJob.
• frontend/src/hooks/useLocalAi.ts polls Local AI download jobs and refreshes runtime status after completion.
• Tauri command shell registers job commands and owns both managed stores.
• csv-anonymizer-core provides AnonymizeParams, ProcessControl, and AnonymizeData.
• Smart replacement map is prepared before streaming rows when selected columns use Local AI.

Entry points for modification

• Change generic retention, TTL, status snapshots, or cancellation mechanics in src-tauri/src/job_registry.rs.
• Change anonymization job status fields or lifecycle handling in src-tauri/src/jobs.rs.
• Change model download progress parsing or terminal handling in src-tauri/src/local_ai/download.rs.
• Change Tauri job command request shapes in src-tauri/src/commands/job_commands.rs.
• Change Local AI download command behavior in src-tauri/src/commands/local_ai_commands.rs.

Key source files

• src-tauri/src/job_registry.rs
• src-tauri/src/jobs.rs
• src-tauri/src/local_ai/download.rs
• src-tauri/src/commands/job_commands.rs
• src-tauri/src/commands/local_ai_commands.rs