how to integrate models not available via huggingface or gluon?

[pstroe]

hi there,

as i understand your library, it works with models which are available from huggingface or gluon.

question: for a model that is not available in the model zoos of those two frameworks, e.g. a model i trained myself, how can i get this to work with a config.json, a pytorch_model.bin and a vocab.txt file?

best,

phillip

[pstroe]

hello there, since there has not been a response as of yet, i'd like to reinvoke this issue.

[JulianSlzr]

Hi Phillip, thanks for reaching out. If I understand correctly from this and your (indirect) private message, your model is still defined in Transformers; it's just one you trained yourself.

At least for Transformers v3, you can see in commit f33f240 how I added ALBERT support in response to issue #2 :

• Here is where support for a zoo model or local weights (via --weights) is added: https://github.com/awslabs/mlm-scoring/blob/master/src/mlm/models/__init__.py#L94-L102
• (Optional) Here is where I subclass ALBERT's masked LM head to improve memory use: https://github.com/awslabs/mlm-scoring/blob/master/src/mlm/models/bert.py#L195-L199

I have been on leave + working on other projects, but hope to revisit usability and a switch to v4 in Jan-Feb 2022.

[pstroe]

hi julian,
thanks a lot for your response! this helped a lot and i could make it work!