Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow to override the checkpoint's droupout settings #2213

Merged
merged 2 commits into from Jan 26, 2023

Conversation

l-k-11235
Copy link
Contributor

@l-k-11235 l-k-11235 commented Oct 10, 2022

This PR provides the ability to override the checkpoint's dropout settings (dropout, attention_dropout) when finetuning a Transformer model.
A new flag override_opts is added. This overriding can thus be disabled with --override_opts False
It is not possible at the moment because the checkpoint's model_opts override those set in the config.
See the following discussion about this topic:
https://forum.opennmt.net/t/tranformer-change-dropout-when-finetuning/4625

@vince62s
Copy link
Member

@l-k-11235 @francoishernandez is this ok to merge ?

@francoishernandez
Copy link
Member

LGTM

@vince62s
Copy link
Member

vince62s commented Jan 26, 2023

I am just wondering if model_opt = opt is not too strong.
what if some options are missing in opt ? shoudn't we copy only the ones that are modified?

@francoishernandez
Copy link
Member

At this stage of the code opt should be a valid opts object.

@vince62s vince62s merged commit 47534e9 into OpenNMT:master Jan 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants