Initial release.
- Add
SoftMoEVisionTransformer
class. - Add default ViT architectures
soft_moe_vit_tiny
,soft_moe_vit_small
,soft_moe_vit_base
,soft_moe_vit_large
andsoft_moe_vit_huge
functions. - Add
SoftMoELayerWrapper
class.
Full Changelog: https://github.com/bwconrad/soft-moe/commits/0.0.1