A simple project that help visualize expert router choices for text generation
-
Updated
Apr 17, 2024 - Python
A simple project that help visualize expert router choices for text generation
This is the repo for the MixKABRN Neural Network (Mixture of Kolmogorov-Arnold Bit Retentive Networks), and an attempt at first adapting it for training on text, and later adjust it for other modalities.
Meet Moe, a discord bot, written in modern python!
Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Network (ACCV 2020)
Community Implementation of the paper: "Multi-Head Mixture-of-Experts" In PyTorch
Official repository for paper "MATERobot: Material Recognition in Wearable Robotics for People with Visual Impairments" at ICRA 2024, Best Paper Finalist on Human-Robot Interaction
[arXiv'24] Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization
Implementation of the "the first large-scale multimodal mixture of experts models." from the paper: "Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts"
[Preprint] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"
PyTorch implementation of moe, which stands for mixture of experts
pytorch open-source library for the paper "AdaTT Adaptive Task-to-Task Fusion Network for Multitask Learning in Recommendations"
[ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal, Shiwei Liu, Zhangyang Wang
Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Zeta
Batch download high quality videos from https://twist.moe
Add a description, image, and links to the moe topic page so that developers can more easily learn about it.
To associate your repository with the moe topic, visit your repo's landing page and select "manage topics."