Skip to content

Issues: microsoft/mup

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

26 Open 36 Closed
26 Open 36 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

More options of input/output types in coord_check
#80 opened Oct 18, 2024 by francois-rozet
1
How to use with SSL methods like DINOv2?
#78 opened Jul 30, 2024 by josephcappadona
MuP for RNNs
#77 opened Jul 26, 2024 by norikazu99
Not getting perf improvements from muP at ~1.5B scale
#76 opened Jul 19, 2024 by gordicaleksa
MuP for Mamba
#74 opened May 30, 2024 by norikazu99
Increasing coord check for the network output
#71 opened Apr 11, 2024 by AkshitaB
2
mu parametrization for gated-mlp and group-query attention
#70 opened Mar 22, 2024 by ftgreat
Reproducing Figure 1 using 'examples/Transformer/main.py'
#69 opened Jan 16, 2024 by jndean
coord_check for model that returns loss function directly
#68 opened Jan 8, 2024 by ad8e
Reproducing the validation accuracy vs learning rates curve on ResNet
#67 opened Dec 21, 2023 by liulei277
1
About Learning rate decay
#64 opened Oct 17, 2023 by afcruzs
2
Usage with torch.compile in Pytorch 2?
#60 opened Sep 28, 2023 by dreavjr
2
FSDP support?
#59 opened Sep 25, 2023 by platers
1
3
Some questions about the implementation of muP.
#57 opened Sep 11, 2023 by lepodl
µTransfer across batch size && weight decay setting
#56 opened Aug 21, 2023 by PanYue2023
_rescale_parameters() inconsistent with the paper for the tied embedding scenario?
#55 opened Jul 12, 2023 by ofivite
2
Is it possible to also scale the depth of the model?
#54 opened Jul 6, 2023 by ricomnl
5
Warmup schedule when changing the number of tokens/steps (GPT-3 experiment detail)
#51 opened Jun 6, 2023 by sashaDoubov
Positional Embeddings should be MuReadout parameters ?
#48 opened May 31, 2023 by codedecde
2
Are Sequentials with list comprehension handled incorrectly?
#43 opened Apr 17, 2023 by RobertBaruch
2
in mlp example: 2 problems
#41 opened Mar 16, 2023 by yjjinjie
1
Has MuP been tested on segmentation models?
#26 opened Nov 6, 2022 by isdj
4
Should base=None be used in set_base_shapes for model used for tuning?
#25 opened Nov 3, 2022 by callumm-graphcore
2
Does mup support Swin Transformer v2 model?
#21 opened Jul 6, 2022 by shiyf129
2
integration with Flax?
#11 opened Apr 13, 2022 by nestordemeure
4
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.