Feature request: Show progress when encoding with multiprocessing (encode_multi_process)

[MikulasZelinka]

Hi,

when encoding sentences without multiprocessing, it's easy enough to just monitor the progress with tqdm by wrapping the loop (when encoding individual sentences) or just using the built-in progress bar of encode:

sentence-transformers/sentence_transformers/SentenceTransformer.py

Lines 140 to 142 in 46a1494

	def encode(self, sentences: Union[str, List[str], List[int]],
	batch_size: int = 32,
	show_progress_bar: bool = None,

However, when using encode_multi_process, that isn't easily possible because everything is hidden inside those methods:

sentence-transformers/sentence_transformers/SentenceTransformer.py

Line 230 in 46a1494

def start_multi_process_pool(self, target_devices: List[str] = None):

sentence-transformers/sentence_transformers/SentenceTransformer.py

Line 277 in 46a1494

def encode_multi_process(self, sentences: List[str], pool: Dict[str, object], batch_size: int = 32, chunk_size: int = None):

Would it please be possible to add progress bar(s) for multiprocess encoding?

Either one progress bar for the whole bunch of sequences, or a progress bar per process. Thank you!

[abhishekmamdapure]

Any update on this ?

[jaanli]

+1, this would be super helpful. encode supports this but encode_multi_process does not.