System Architecture

High-Level System Design (HLD)

Architecture decisions

1. The parent process owns CLI parsing, JSONL reading, validation, length-aware batching, process coordination and final output.
2. A spawned worker is assigned one device and owns a separate model and tokenizer (real mode); the mock mode substitutes a keyword rule.
3. All workers read from the shared task queue, which naturally balances uneven batch durations. None sentinel items indicate no further batches.
4. A result queue returns (ids, outputs, token_count), so the parent reconstructs original order even if batches complete out of order.
5. A model-load lock limits simultaneous model initialization; a shared event releases workers for processing after each reports readiness.

The code does not expose network endpoints, use DDP DistributedSampler, or split a model across GPUs. See Parallelism Explained.

flowchart TB
    A["CLI arguments + JSONL input"] --> B["Validate records / text-field"]
    B --> C["Compute length-ranked batch indices"]
    C --> D["Multiprocessing spawn context"]
    D --> E["Task queue + result queue + load lock + go event"]
    E --> W0["Worker 0 - GPU 0 or CPU"]
    E --> W1["Worker 1 - GPU 1 or CPU"]
    W0 --> O["Results queue: IDs, answers, token counts"]
    W1 --> O
    O --> M["Parent maps outputs to original indices"]
    M --> N["Normalize one-word sentiment labels"]
    N --> P["Write results JSONL + optional accuracy"]

Loading