This repository contains the public implementation of KromHC, a novel variant of manifold-constrained hyper-connections for training large language models.
The code will be available soon (in 1 week).
-
Notifications
You must be signed in to change notification settings - Fork 0
wz1119/KromHC
About
KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matrices
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published