Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ValueError during the init of pretrained BERT #29

Open
shy1992 opened this issue Aug 8, 2019 · 4 comments
Open

ValueError during the init of pretrained BERT #29

shy1992 opened this issue Aug 8, 2019 · 4 comments

Comments

@shy1992
Copy link

shy1992 commented Aug 8, 2019

Hello!
I tried your code in a google colab and i encountered a problem i wasn't able to solve.
During the inititalization of the Bert encoder in your ipynb:
https://github.com/santhoshkolloju/Abstractive-Summarization-With-Transfer-Learning/blob/master/BERT_SUMM.ipynb

in cell 15 there occurs the following error

`Intializing the Bert Encoder Graph
loading the bert pretrained weights

ValueError Traceback (most recent call last)
in ()
35 init_checkpoint = os.path.join(bert_pretrain_dir, 'bert_model.ckpt')
36 #init_checkpoint = "gs://cloud-tpu-checkpoints/bert/uncased_L-12_H-768_A-12/bert_model.ckpt"
---> 37 model_utils.init_bert_checkpoint(init_checkpoint)

5 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py in _init_from_checkpoint(ckpt_dir_or_file, assignment_map)
344 "Assignment map with scope only name {} should map to scope only "
345 "{}. Should be 'scope/': 'other_scope/'.".format(
--> 346 scopes, tensor_name_in_ckpt))
347 # If scope to scope mapping was provided, find all variables in the scope
348 # and create variable to variable mapping.

ValueError: Assignment map with scope only name bert/position_embeddings should map to scope only bert/embeddings/position_embeddings. Should be 'scope/': 'other_scope/'.`

i checked the github repo of texar already and found the post:
asyml/texar#127

Basically the code for the encoder and decoder changed in the newer version of texar but i dont know how to adjust the code.

@santhoshkolloju
Copy link
Owner

Clone the older version of code it will be there in releases of texar repo

@shy1992
Copy link
Author

shy1992 commented Aug 8, 2019

So i tried the following:

  • Downloaded the 0.2.0 repo like you did with the git repo
  • installed the texar from source

`!wget https://github.com/asyml/texar/archive/v0.2.0.zip -P texar_repo;
!unzip texar_repo/v0.2.0.zip -d texar_repo
if not 'texar_repo' in sys.path:
sys.path += ['texar_repo']

!mv texar_repo/texar-0.2.0/* texar_repo`

and the installation of the old texar version:
!cd texar_repo && pip install -e .

the error still occurs

@santhoshkolloju
Copy link
Owner

Please download. 0.1.0

@shy1992
Copy link
Author

shy1992 commented Aug 9, 2019

Hello thank you for your reply.
i got it to work by just copy the repo
the thing i got wrong was that texar doesnt need to be installed i guess
with the command:
sys.path += ['texar_repo']
you get also access to the texar module so pip install is not required.

Thank you for your help!
Amazing work btw its great to get insights about a complex tensorflow project.

Off-Topic: Do you have any good ressources to learn tensorflow or did you just read the tensorflow documentation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants