Python 3.10 Colab

[ziyaad30]

RuntimeError: Error(s) in loading state_dict for CustomAlbert:
Unexpected key(s) in state_dict: "embeddings.position_ids".

[teopapad92]

In Utils/PLBERT/utils.py, delete the check

from:

if not hasattr(bert.embeddings, 'position_ids'):
    del new_state_dict["embeddings.position_ids"]
bert.load_state_dict(new_state_dict)

to:

# if not hasattr(bert.embeddings, 'position_ids'):
del new_state_dict["embeddings.position_ids"]
bert.load_state_dict(new_state_dict)

[ziyaad30]

In Utils/PLBERT/utils.py, delete the check

from:

if not hasattr(bert.embeddings, 'position_ids'):
    del new_state_dict["embeddings.position_ids"]
bert.load_state_dict(new_state_dict)

to:

# if not hasattr(bert.embeddings, 'position_ids'):
del new_state_dict["embeddings.position_ids"]
bert.load_state_dict(new_state_dict)

Thank you, Ill give it a go and test.

[ziyaad30]

It worked and closing, I can see this is still in early development.