Low Level Design

Low-Level Design (LLD): Function Dependencies

Functions implemented in parallel_llm_inference.py

Function	Responsibility
parse_args(argv)	Validate CLI options and positive numeric thresholds
read_jsonl(path, text_field, max_chars)	Decode UTF-8 JSONL and validate review field
make_demo_rows(n)	Generate labeled synthetic reviews
checked_devices(...)	Choose/validate CUDA or CPU workers; reject duplicate GPU assignments
main(argv)	Build length-sorted batches, spawn workers, collect results, restore order and write JSONL
worker(...)	Device setup, load model, block on start event, process until sentinel
load_model(...)	Load tokenizer and Qwen model weights and place on one device
generate_batch(...)	Chat template, tokenization, generation and token accounting
mock_generate(texts)	Deterministic keyword-based smoke-test backend
receive(...)	Queue receive with timeout and worker-crash detection
normalized_label(answer)	Accept a strict one-word positive/negative label with optional punctuation

This is procedural module/function design, not an invented object-oriented class hierarchy. See Request Sequence for runtime interactions.

flowchart LR
    Main["main(argv)"] --> Args["parse_args(argv)"]
    Main --> Input["read_jsonl(path, field, limit)"]
    Main --> Demo["make_demo_rows(n)"]
    Main --> Devices["checked_devices(...)"]
    Main --> Receiv["receive(queue, processes, timeout)"]
    Main --> Worker["worker(id, device, args, ...)"]
    Worker --> Load["load_model(model_id, device, dtype)"]
    Worker --> Gen["generate_batch(model, tokenizer, texts, ...)"]
    Gen --> Prompt["build_messages(text)"]
    Worker --> Mock["mock_generate(texts)"]
    Main --> Normal["normalized_label(answer)"]

Loading