Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mamba #113

Merged
merged 30 commits into from
May 27, 2024
Merged

Mamba #113

merged 30 commits into from
May 27, 2024

Conversation

mrudat-iais
Copy link
Collaborator

  • added mamba implementation
  • adapted implementation to modalities interface
  • added tests for mamba classes

rrutmann and others added 14 commits April 8, 2024 16:16
@rrutmann
feat(mamba): Define Mamba model as in the original implementation of …
c666481 
…the authors
@rrutmann
feat(mamba): Add full language model based on mamba blocks
060aa44
@ajude2s
feat(models): Added main function for mamba model.
04c4046
@ajude2s
fix(models): Added seed as a parameter in the gpt2 model.
20d714b
@ajude2s
feat(models): Renamed the mamba files and Added MambaLMHead to the ma…
e897f0c 
…in().
@mrudat-iais
Merge branch 'main' into mamba
eb1f44f
@mrudat-iais
Merge remote-tracking branch 'origin/mamba' into mamba
bee993e
@mrudat-iais
refactor(mamba): delete comment
cb6b066
@mrudat-iais
feat(models): we can now train a mamba model
46238ca 
- new component "mamba" registered
- mamba config file with small training setting in config_mem_map_mamba.yaml
- mamba model implemented (with config pydantic)
- changed forward() abstract method in model.py
@mrudat-iais
feat(tests): tests for mamba blocks are written
108af99
@mrudat-iais
refactor(models): added and removed todo(s)
05b7d50
@ajude2s
feat(models):
fe591ff 
- Remove the kwargs from the forward function.
- Removed the from_pretrained() from mamba_model.py
@ajude2s
test(mamba): Added tests for mamba.
b4409d4
@mrudat-iais
Merge branch 'main' into mamba
373cf5f
@mrudat-iais mrudat-iais requested a review from le1nux April 24, 2024 14:10
@le1nux le1nux requested review from mali-git and fromm-m April 24, 2024 21:19
rrutmann and others added 6 commits April 29, 2024 15:46
@rrutmann
feat(mamba): Add generation capabilities to Mamba model
0a04463
@rrutmann
feat(mamba): Add config for small scale experiments
67280c5
@mrudat-iais
refactor(mamba):
9fded2a 
- generate method now in interface
- "input_ids" now as variable saved in models and configs
@mrudat-iais
refactor(mamba):
d9be945 
- generate method now in interface
- "input_ids" now as variable saved in models and configs
@mrudat-iais
refactor(mamba):
73a0c2b 
- generate method now in interface
- "input_ids" now as variable saved in models and configs
@mrudat-iais
refactor(mamba):
eab721f 
- generate method now in interface
- "input_ids" now as variable saved in models and configs
thomaschhh
thomaschhh reviewed May 13, 2024
View reviewed changes
Copy link
Member

@thomaschhh thomaschhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional type hints / annotations would be helpful.

src/modalities/models/mamba/mamba_block.py Show resolved Hide resolved
src/modalities/models/mamba/mamba_block.py Outdated Show resolved Hide resolved
ajude2s and others added 3 commits May 13, 2024 16:38
@ajude2s
feat(models):
a92193e 
- Adapted the tests for the new configs
- Changed the interface for generate and generate_text functions for all the models
@rrutmann
test(mamba): Use correct vocab sizes in tests
3163c69
@rrutmann
feat(mamba): Add missing scripts from mamba_ssm to modalities
575fd34
@rrutmann
Copy link
Collaborator

Additional type hints / annotations would be helpful.

We added the missing type hints

@rrutmann rrutmann merged commit d2f6dd9 into main May 27, 2024
@mrudat-iais mrudat-iais deleted the mamba branch May 28, 2024 11:33
@flxst flxst mentioned this pull request Jun 12, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rrutmann rrutmann rrutmann left review comments

@thomaschhh thomaschhh thomaschhh left review comments

@le1nux le1nux Awaiting requested review from le1nux

@mali-git mali-git Awaiting requested review from mali-git

@fromm-m fromm-m Awaiting requested review from fromm-m

Assignees

@rrutmann rrutmann

@mrudat-iais mrudat-iais

@ajude2s ajude2s

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mrudat-iais @rrutmann @thomaschhh @ajude2s