Repository navigation
MiniGPT-0
Release Date: 2025-05-03
Checkpoint: minigpt0.pth
License: MIT (code), research-only (weights)
Device: CPU or GPU compatible
Model Specs
| Feature | Value |
|---|---|
| Parameters | ~7.3M |
| Layers | 6 |
| Heads | 4 |
| Embedding Dim | 256 |
| Feedforward Dim | 1024 |
| Block Size | 128 tokens |
| Vocab Size | 5000 |
| Dropout | 0.1 |
| Framework | PyTorch |
Training Info
- Trained on: 20% OpenAssistant
- Model type: GPT-style decoder-only
- Epochs: 3
- Steps: 290,670 (96,890x3)
- Loss: ~2.63 (token-wise cross-entropy)