Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix importing unofficial TF models with extra optimizer weights #2652

Merged
merged 1 commit into from
Feb 7, 2020
Merged

Fix importing unofficial TF models with extra optimizer weights #2652

merged 1 commit into from
Feb 7, 2020

Conversation

monologg
Copy link
Contributor

@monologg monologg commented Jan 27, 2020

Hi:)

I was trying to convert the BERT tf model to torch model, and tf model has extra optimizer weights (This file is the one I've tried to convert).

But it encounters the error, and I've printed the parameters' name in tf model.

Screen Shot 2020-01-27 at 10 21 50 PM

I've found out the instead of "adam_v" for "adam_m, names are saved as "AdamWeightDecayOptimizer" or "AdamWeightDecayOptimizer_1".

There was a similar issue that also encountered the issue that I had. (Issue Link from DeepPavlov repo)

I can't find out the exact reason why the parameters are named as "AdamWeightDecayOptimizer" or "AdamWeightDecayOptimizer_1" instead of "adam_v" or "adam_m", but it might be safe to cover all the exceptional cases:)

Copy link
Member

@LysandreJik LysandreJik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@codecov-io
Copy link

Codecov Report

Merging #2652 into master will not change coverage.
The diff coverage is 0%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #2652   +/-   ##
=======================================
  Coverage   74.58%   74.58%           
=======================================
  Files          87       87           
  Lines       14892    14892           
=======================================
  Hits        11107    11107           
  Misses       3785     3785
Impacted Files Coverage Δ
src/transformers/modeling_bert.py 87.9% <0%> (ø) ⬆️
src/transformers/modeling_albert.py 79.14% <0%> (ø) ⬆️
src/transformers/modeling_t5.py 81.09% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update babd41e...d338eb0. Read the comment docs.

@LysandreJik LysandreJik merged commit 7336896 into huggingface:master Feb 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants