Repository navigation
Tinyapper 2.0.0
More Tinyapper, the LLM trained from scratch! This time the model is trained to be conversational, along with a saving KV cache.
Tech Changes
- New conversational demo at tinyapper.craisin.tech
- Updated attention conditional for multiple infills for chat
- Updated KV cache so chats can be cached
- Reiterated how datasets are made
- Retrained model for yapperpedia so the model is slightly more coherent
Future Development
- Standardizing datasets to use memmap for larger datasets
- Retrain conversational model
- Fine tune the model