Repository navigation
Exploring a new approach to make Mamba even stronger #1051
Closed
PrinceZaman-2005
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I've been experimenting with Mamba/State Space Models and recently built a small prototype around an idea I'm calling S-Mamba.
Mamba is already very strong at handling long sequences. My question is whether we can push it further by making the model learn when information in a sequence should be retained, released, or carried forward.
I've built an initial prototype and I'm currently experimenting with this idea.
I'd really appreciate thoughts from people who have worked with Mamba or SSMs, especially if there are related approaches I should look into.
Project: https://github.com/LiteBrains-AI/S-Mamba
I'd be interested to hear whether this direction seems worth exploring further.
All reactions