Execution Modes

Execution Modes

Mode	Inputs	Device	Typical configuration	Purpose
--smoke-test	80 synthetic examples, tiny tokenizer, random frozen embeddings	CPU only	train/val/test = 64/8/8, S=24, B=8, N=2, H=4, epochs=2	Offline correctness run
--train	Yelp Polarity + MiniLM frozen embeddings	CUDA if available, otherwise CPU	requested 1000/200/200, S=128, B=8, N=3, H=6, epochs=2	Small real-data experiment
--full	Yelp Polarity + MiniLM frozen embeddings	CUDA if available, otherwise CPU	requested 50000/5000/10000, S=256, B=8, epochs=5	Larger preset

Actual returned real-data counts can be lower after review-length filtering. The --full path overrides train_size, val_size, test_size, batch_size, max_len, and epochs even if those are supplied via the CLI.

Example commands

python transformer_from_scratch.py --smoke-test
python transformer_from_scratch.py --train --sdpa
python transformer_from_scratch.py --full --sdpa
python transformer_from_scratch.py --train --output .\runs\experiment_01