Checkpoints

Checkpoint and Adapter Lifecycle

flowchart LR
    A["Training loop completed"] --> R{"rank == 0?"}
    R -->|yes tiny mode| C["torch.save: tiny_transformer.pt"]
    R -->|yes qwen mode| D["PEFT save_pretrained + tokenizer.save_pretrained"]
    R -->|no| E["Skip checkpoint write"]
    C --> B["Distributed barrier if world_size > 1"]
    D --> B
    E --> B
    B --> X["destroy_process_group"]

Loading

Rank-0 artifacts

• Tiny mode: --save-dir/tiny_transformer.pt saved via torch.save with architecture metadata and model_state_dict.
• Qwen mode: PEFT LoRA adapters and tokenizer saved with save_pretrained. The original pretrained Qwen base weights are not copied into the output folder.

Both modes write only on rank 0 and then join a distributed barrier before teardown. The script performs a single save after the training loop, not periodic checkpointing. It does not save optimizer state, scheduler state, epoch/step progress, RNG state, or resumable distributed-training metadata.

Artifacts and local training outputs are excluded from Git with .gitignore in the publication package. Do not upload model checkpoints as source files without checking size, provenance and redistribution terms.