Troubleshooting

Troubleshooting

Python script complains about required mode

The entry point requires exactly one of --smoke-test, --train, or --full. In a notebook, run:

!python transformer_from_scratch.py --smoke-test

Python cannot import Torch, Transformers, or Datasets

Confirm the interpreter/environment and install the supplied dependencies:

python -m pip install -r requirements.txt
python -m pip check

PyTorch detects no CUDA

The PyTorch package itself must be built for CUDA and compatible with your NVIDIA GPU/driver. Real-data training falls back to CPU; smoke test deliberately uses CPU regardless of CUDA availability.

CUDA out of memory

Try --train --batch-size 1 --max-len 64 --sdpa. Avoid --full until smaller batches work; that preset overrides batch size and maximum sequence length.

Model/data download failures

--train and --full use Hugging Face. Internet access and relevant cache availability are required. --smoke-test has no Hugging Face dependency.

Graphviz image not created

--visualize is optional. Install torchview, graphviz (Python), and the Graphviz executable if you want PNGs. Mermaid diagrams are rendered directly by GitHub from the Wiki pages.

requirements.txt was missing in the source archive

The uploaded archive uses transformer_from_scratch_requirements.txt. This publishing package preserves it and adds an identical requirements.txt for the README's install commands.

transformer_from_scratch_corrected.py appears in a docstring

The original script's top docstring mentions an older filename. The actual script in the uploaded project and ZIP is transformer_from_scratch.py. Use the latter in your commands.