CrystalLM

🚀 Performance & Training Improvements

This release focuses on dramatically reducing memory usage during training and improving dataset scale, making CrystalLM usable on low-memory machines.

✨ What’s new

• 💾 Massive memory optimization

  • Replaced full in-memory training tensor construction with Dataset + DataLoader
  • Training RAM usage reduced from ~16 GB → ~250 MB
  • Enables smooth training on older devices (e.g. 8GB RAM MacBook Pro with i5-7360U)

• 📚 Larger training corpus

  • Increased total training text size and token count
  • Improved language continuity and semantic richness in generation

• ⚙️ Training stability

  • Mini-batch training instead of full-batch forward pass
  • Faster iteration speed and more predictable loss curves

🧠 Model

• Word-level language model
• Embedding + LSTM architecture (unchanged)
• Context window: 20 tokens

📦 Notes

• Model quality improves noticeably with additional text
• This version lays the foundation for future architecture upgrades (e.g. GRU / Transformer)