BGE embedding issue related to Model Dimension

[nittulukose]

I have managed to deploy privateGPT with sagemaker endpoints. I have used the following models
LLM Model : Mistral 7B Instruct
Embeding Model: BGE Base En V1.5 (Its dimension is 768)
This worked fine.

But when I chang Embedding to any other bge model (dimensions other than 768), I get the following error

ValueError: shapes (4,768) and (384,) not aligned: 768 (dim 1) != 384 (dim 0)

The given error is for the model bge-small-en-v1.5 (Dimension 384).

How can I resolve? I want to use other BGE embedding models with dimensions other than 768.

It seems, while running the privategpt application with "sagemaker mode" is expecting the dimension 768.

How can I explicitly change that to our custom value?

I need to run the embedding model "bge-m3 (dimension 1024)" for multilingual feature.

Looking forward for a solution.

[dbzoo]

You may not mix and match multiple embedding models with different dimensions into the same storage backend without reinitializing them - specifically the vector db. Given that you must re-init the vector store to change dimensions and the index/doc store related to it, you need to reset all database stores.

If you are using simple, chromadb or postgres this will wipe the stores. - See #1772
$ make wipe

[nittulukose]

Hi,

Thank you for your response.

In this case, I am using "qdrant" vectorstore. Could you please let me know how to re-init "qdrant" vector store?

[dbzoo]

If you are using qdrant locally you can delete the directories and files within.
Otherwise you need this #1783

[nittulukose]

Hi @dbzoo

Thanks, that worked!!!

I have cleared the contents in the directory, local_data/private_gpt/qdrant. Now, the dimension issue fixed.

Please suggest an LLM - Embedding Model combination that support 'Query Doc' mode for multi language (Both input and output).

My Requirement.

1. Query Doc Mode
2. Support multi language (Especially Arabic and English)
3. Accuracy

Currently I'm using,
LLM Model : Mistral 7B Instruct
Embeding Model: BGE M3 (Dim 1024) - which claims multilanguage support.
But, in this case Query Doc Mode is not accurate.