CLI Reference

Command-Line Interface

Option	Default	What it controls
--model	Qwen/Qwen2.5-0.5B-Instruct	Model repository ID or local model directory
--devices	All CUDA GPUs, otherwise CPU	Distinct cuda:0 cuda:1 IDs or cpu cpu
--input	Synthetic rows	Read JSONL file
--text-field	review	Input text field
--output	results.jsonl	Output JSONL path
--num-samples	40	Synthetic row count when no input supplied
--batch-size	2	Reviews per worker generate() call
--max-input-tokens	256	Prompt token maximum
--max-new-tokens	8	New output token maximum
--max-chars	1200	Optional input character clipping
--dtype	CUDA float16 / CPU float32	Requested weight dtype
--worker-timeout	900	Seconds before waiting for a worker fails
--mock	Off	Use heuristic backend instead of a model

All size and time flags must be positive. The entrypoint rejects unsupported devices and duplicate CUDA GPU IDs. For authoritative details, run:

python parallel_llm_inference.py --help