Implementation of the model "Hedgehog" from the paper: "The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry"
open-source
ai
ml
attention
multi-modal
neural-nets
feedforward
softmax
attention-is-all-you-need
attention-mechanisms
mlps
ffns
opensource-ai
-
Updated
Mar 11, 2024 - Python