How to use the model with sentence-transformer for inference?

[qhd1996]

Cannot load the model.
code
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("../../models/consbert/unsup-consert-base-atec_ccks") # the model path
Error message
Traceback (most recent call last):
File "/home/qhd/PythonProjects/GraduationProject/code/preprocess_unlabeled_second/sentence-bert.py", line 16, in
model = SentenceTransformer("../../models/cosbert/unsup-consert-base-atec_ccks")
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/sentence_transformers/SentenceTransformer.py", line 87, in init
modules = self._load_sbert_model(model_path)
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/sentence_transformers/SentenceTransformer.py", line 824, in _load_sbert_model
module = module_class.load(os.path.join(model_path, module_config['path']))
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/sentence_transformers/models/Transformer.py", line 123, in load
return Transformer(model_name_or_path=input_path, **config)
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/sentence_transformers/models/Transformer.py", line 30, in init
self.tokenizer = AutoTokenizer.from_pretrained(tokenizer_name_or_path if tokenizer_name_or_path is not None else model_name_or_path, cache_dir=cache_dir, **tokenizer_args)
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 445, in from_pretrained
return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1719, in from_pretrained
return cls._from_pretrained(
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1791, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/transformers/models/bert/tokenization_bert_fast.py", line 177, in init
super().init(
File "/home/qhd/anaconda3/envs/qhdpython39/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 96, in init
fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
Exception: No such file or directory (os error 2)

[yym6472]

From the error message, it seems that the transformers and sentence_transformers packages used by your code are the ones installed in the python library (site-packages), rather than the modified version in my repository. You should run your code in the root dir of my repository or copy the modified transformers and sentence_transformers folders to your root dir.

If you want to encode the sentence by averaging the embeddings of last two layers (last2avg) or the first and the last layers (firstlastavg), you can use the following function in eval.py:

def load_model(model_path: str, last2avg: bool = False, firstlastavg: bool = False):
    model = SentenceTransformer(model_path)
    if last2avg:
        model[1].pooling_mode_mean_tokens = False
        model[1].pooling_mode_mean_last_2_tokens = True
        model[0].auto_model.config.output_hidden_states = True
    if firstlastavg:
        model[1].pooling_mode_mean_tokens = False
        model[1].pooling_mode_mean_first_last_tokens = True
        model[0].auto_model.config.output_hidden_states = True
    logging.info("Model successfully loaded")
    return model

[qhd1996]

From the error message, it seems that the transformers and sentence_transformers packages used by your code are the ones installed in the python library (site-packages), rather than the modified version in my repository. You should run your code in the root dir of my repository or copy the modified transformers and sentence_transformers folders to your root dir.

If you want to encode the sentence by averaging the embeddings of last two layers (last2avg) or the first and the last layers (firstlastavg), you can use the following function in eval.py:

def load_model(model_path: str, last2avg: bool = False, firstlastavg: bool = False):
    model = SentenceTransformer(model_path)
    if last2avg:
        model[1].pooling_mode_mean_tokens = False
        model[1].pooling_mode_mean_last_2_tokens = True
        model[0].auto_model.config.output_hidden_states = True
    if firstlastavg:
        model[1].pooling_mode_mean_tokens = False
        model[1].pooling_mode_mean_first_last_tokens = True
        model[0].auto_model.config.output_hidden_states = True
    logging.info("Model successfully loaded")
    return model

Thanks a lot!