myModel

myModel is a from-scratch Transformer language-model project. It contains a small GPT-style model, byte-level BPE tokenizer utilities, custom training components, and text generation helpers.

Project Structure

myModel/
├─ scripts/                 # CLI entry points for training and generation
├─ src/mymodel/
|  ├─ configs/              # Dataclass configs for model, training, and generation
│  ├─ model/                # Transformer architecture and neural network modules
│  ├─ tokenizer/            # BPE tokenizer and byte mapping helpers
│  ├─ training/             # Training loop, optimizer, scheduler, loss, checkpoints
│  └─ generation/           # Model loading, sampling, and text generation
├─ data/                    # Local tokenizer assets and processed training data
├─ checkpoints/             # Local model checkpoints
└─ tests/                   # Lightweight smoke tests

Setup

pip install -r requirements.txt
pip install -e .

Train

python scripts/train.py --data_dir data/traindata --model_name myModel

Generate

python scripts/generate.py \
  --model_config_path checkpoints/model_config.json \
  --checkpoint_path checkpoints/checkpoint.pt \
  --vocab_path data/tokenizer/vocab.json \
  --merges_path data/tokenizer/merges.txt \
  --prompt "Once upon a time"

Test

pytest

Large local artifacts such as checkpoints and binary training data are ignored by git.