This project is an AI chatbot developed with Python using the bigram learning model followed by a self-attention transformer model, following this tutorial using different data. The model is implemented into the Handler One Discord Bot project here.
- Batch Size: 128
- Block Size: 256
- Maximum Iterators: 5000
- Evaluation Interval: 500
- Learning Rate: 3e-4
- Evaluation Iterators: 200
- Number of Embeds: 384
- Number of Heads: 6
- Number of Layers: 6
- Dropout Rate: 0.3
This section contains an example output after training with the current model hyperparameters in this section.
