Skip to content

ETL processing documents permanently stuck after crash #2

@haydenk

Description

@haydenk

claim_pending in crates/etl/src/processor.rs only selects WHERE status = 'downloaded'. If the processor crashes mid-run, any document stuck in 'processing' can never be picked up again without manual SQL intervention.

Fix: Add a timeout-based reset query that moves stale 'processing' rows back to 'downloaded' (e.g., WHERE status = 'processing' AND etl_started_at < NOW() - INTERVAL '1 hour') before claiming new work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions