Can we add ReDimNet2 model for embeddings ?

[ashutosh-utsav]

Hi,

I’m using Embed Anything for large scale embedding workflows and wanted to ask about support for RedimNet2 (audio/speaker embeddings).

Currently, I don’t see a way to integrate it. Is there a recommended approach for adding custom embedding models, or are there plans to support audio models like this?

Happy to contribute if needed.

[akshayballal95]

Good to hear you are using EmbedAnything and are happy with it.

There are currently two ways to add models.

1. Using Candle. If the model is on HuggingFace and uses one of the architectures supported by Candle, it can be added easily
2. ONNX model. If the model is available as an ONNX model, we can use the ONNX Runtime feature in this library to infer on it.

You can start by exploring how the whisper model is added in rust/src/file_processor/audio /audio_processor.rs. We don't have any other audio models, so this isn't abstracted well. So there is work to be done towards that as well, similar to how image embedding models are abstracted.

Happy to accept contributions for other modalities, such as audio/video. It's quite useful for us. Feel free to explore and let me know if you face any issues.