[Benchmark] - embedding quantization

[michaelfeil]

System Info

0.0.40 shipped the first version of embedding quant.

--embedding-dtype int8

This Issue is looking for testers, to verify the real life performance of these features at real datasets.

Information

• Docker
• The CLI directly via pip

Tasks

• An officially supported command
• My own modifications

Reproduction

Expected behavior

E.g. delivering a benchmark of int8 quantization - can be added under ./docs

[michaelfeil]

@mahiro72 Expressed interest in working on this fyi!