Worker Lifecycle

Worker State and Lifecycle

• spawn avoids reusing a potentially unsafe CUDA context after fork.
• torch.cuda.set_device(index) runs in CUDA workers; CPU workers set a per-process thread count.
• Each worker reports readiness via result_queue, waits on go_event, then consumes tasks until None.
• Exceptions are reported as tracebacks; parent detects failed worker processes or timeouts.
• Parent finally block terminates any remaining workers and closes queues.

stateDiagram-v2
    [*] --> Spawned
    Spawned --> Initializing
    Initializing --> Ready: model loaded or mock setup
    Initializing --> Error: exception
    Ready --> Waiting: notify parent and await go_event
    Waiting --> Processing: task_queue.get(batch)
    Processing --> Processing: next batch
    Processing --> Finished: task_queue.get(None)
    Processing --> Error: exception
    Finished --> [*]
    Error --> [*]: error message on result queue

Loading