Model Loading and Generation

Model Loading and Generation Internals

Real mode (--mock absent)

• load_model() imports AutoTokenizer and AutoModelForCausalLM from Transformers.
• AutoTokenizer.from_pretrained(model_id) loads tokenizer and applies left padding with EOS-as-PAD if missing.
• The model loads via AutoModelForCausalLM.from_pretrained(..., dtype=...) and is moved to an explicit worker device.
• generate_batch() calls apply_chat_template(), truncates inputs to max_input_tokens, calls greedy model.generate(do_sample=False, ...) and decodes only new tokens.
• Model outputs are normalized to a strict one-word sentiment class; unexpected explanations yield predicted_label: null.

Mock mode

mock_generate() searches review text for positive/negative keywords and returns an answer without importing the Transformers model. It exercises file validation, batching, worker scheduling and result writing, not real LLM generation. Do not compare its throughput or accuracy against Qwen.