MiniGPT-0 (May 3, 2025)

MiniGPT-0

Release Date: 2025-05-03
Checkpoint: minigpt0.pth
License: MIT (code), research-only (weights)
Device: CPU or GPU compatible

---

Model Specs

Feature	Value
Parameters	~7.3M
Layers	6
Heads	4
Embedding Dim	256
Feedforward Dim	1024
Block Size	128 tokens
Vocab Size	5000
Dropout	0.1
Framework	PyTorch

---

Training Info

• Trained on: 20% OpenAssistant
• Model type: GPT-style decoder-only
• Epochs: 3
• Steps: 290,670 (96,890x3)
• Loss: ~2.63 (token-wise cross-entropy)

MiniGPT-0 (Research Preview – Apr 30, 2025)

⚠️ Research Preview

This release of MiniGPT-0-preview is intended for research and educational use only.
It is not aligned, safety-tested, or ready for deployment in production or user-facing applications.
Pretrained weights are licensed for non-commercial research.

MiniGPT-0 (Preview)

Release Date: 2025-04-30
Checkpoint: minigpt0-preview.pth
License: MIT (code), research-only (weights)
Device: CPU or GPU compatible

---

Model Specs

Feature	Value
Parameters	~7.3M
Layers	6
Heads	4
Embedding Dim	256
Feedforward Dim	1024
Block Size	128 tokens
Vocab Size	5000
Dropout	0.1
Framework	PyTorch

---

Training Info

• Trained on: 15% OpenAssistant
• Model type: GPT-style decoder-only
• Epochs: 3
• Steps: 211,740 (70,580x3)
• Loss: 2.375 (token-wise cross-entropy)