.Net: New Feature: AddBertOnnxEmbeddingGenerator should support multi-langugage embedding models which do not have vocab.txt files

[philipag]

---

name: Feature request
about: Suggest an idea for this project

---

Multi-language text embeddings models like arctic_embed_m_v2_0 do not use vocab.txt files are currently not supported by AddBertOnnxEmbeddingGenerator(). The following throws an exception:

		builder.Services.AddBertOnnxEmbeddingGenerator(
			onnxModelPath: "....\arctic_embed_m_v2_0_model_int8.onnx",
			vocabPath: null
			);

The embeddings generator should support these newer models.

[rogerbarreto]

@philipag Thanks for opening the issue, this seems to be a limitation on the Onnx Runtime package.

If you can manage to access this model using the Native Onnxruntime C#, please share the code or Open a PR and I would be happy on adding this capability to our connector.