Simple implementation of mixture of experts transformer in pytorch. Model trained on personal dataset with okayish responses.

Training script - train.py Finetune script - finetune.py MoE implementation - utils.py test out model - httpchatbot.py or eval.py