Model Request for BAAI/bge-m3 (XLMRoberta-based Multilingual Embedding Model)

[mofanke]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Feature Description

Supporting a multilingual embedding.
https://huggingface.co/BAAI/bge-m3

Motivation

There are some differences between multilingual embeddings and BERT

Possible Implementation

sorry, no idea. I tried , seems model arch is same as bert ,but tokenizer is XLMRobertaTokenizer , not bertTokenizer

[RoggeOhta]

Also request this model to be supported.

[vonjackustc]

Tried to support it, use BertModel & SPM tokenizer.
https://huggingface.co/vonjack/bge-m3-gguf

Tested cosine similarity between "中国" and "中华人民共和国":
bge-m3-f16: 0.9993230772798457
mxbai-embed-large-v1-f16: 0.7287733321223814

[vuminhquang]

I got error when using with langchain
"terminate called after throwing an instance of 'std::out_of_range'"

[ciekawy]

same here with llama.cpp, the full error:

libc++abi: terminating due to uncaught exception of type std::out_of_range: unordered_map::at: key not found

[ciekawy]

the _bert version does not crash, but the the embeddings do not seem to have any sense...

[ciekawy]

also tried to follow instructions on https://github.com/PrithivirajDamodaran/blitz-embed but after converting to gguf, getting error:

llama_model_quantize: failed to quantize: key not found in model: bert.context_length

[ciekawy]

@vonjackustc can you share params you used with llama.cpp?