Skip to content

eps is not passed in Mistral3RMSNorm #38025

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sergiopaniego opened this issue May 8, 2025 · 0 comments · May be fixed by #38026
Open

eps is not passed in Mistral3RMSNorm #38025

sergiopaniego opened this issue May 8, 2025 · 0 comments · May be fixed by #38026

Comments

@sergiopaniego
Copy link
Member

The class Mistral3RMSNorm can receive eps:

def __init__(self, hidden_size, eps=1e-6):

but when used in the creation of the Mistral3 model, it is not actually passed

self.norm = Mistral3RMSNorm(config.vision_config.hidden_size)

It could be solved as follows:

self.norm = Mistral3RMSNorm(config.vision_config.hidden_size, eps=config.text_config.rms_norm_eps)
@sergiopaniego sergiopaniego linked a pull request May 8, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant