Translation speed is to slow on HDD

[ghost]

Up to 8 sec on trying translate A4
I think, it would be nice to cache database indexes in the RAM as the argosmodels collection has about 1Gb summary.

[ghost]

Now I understand, the issue mostly related to https://github.com/argosopentech/argos-translate engine as the LibreTranslate is web application based on that project. So maybe we need to close that question here or the engine is different from original version?

[pierotofy]

Translation speed is dependent on argos-translate.

[PJ-Finlay]

What do you mean by "cache database indexes in the RAM"? The majority of the time translation takes is performing the translation itself (longer for longer translations). CTranslate2 seems to be a pretty well optimized inference engine and we currently split sentences and then batch translate each one on its own core.

It's possible that preloading the models from disk into memory could speed things up but I'd want to see some evidence of a meaningful speed improvement before adding code complexity. If you're interested in looking into it I'd be curious to hear what you find.

@guillaumekln

[guillaumekln]

Disk speed only impacts the time it takes to load the model, or when the process is swapping on disk due to a lack of memory. Which one is it here?

[PJ-Finlay]

I meant loading the model from disk to RAM to run inference on it. Argos Translate currently keeps models in memory once they've been used but that's to prevent memory leaks not for performance:

• https://forum.qt.io/topic/120341/how-to-debug-a-hard-to-find-memory-leak-in-pyqt/16
• argosopentech/argos-translate@26802a3