Machine translation scoring behaviour and LLM-based automatic suggestion services

[michael-smt]

I'm trying to wrap my head around how the scoring works with machine translations, specifically these two sentences in the priority of machine translation and translation memory services docs:

For each string with translation score lower than the service’s maximum, the service is asked to produce a translation. Translations with a score exceeding the current one are accepted.

1. "each string" means a existing translation that somehow has a score (presumable from a previous machine translation round) attached to them?
2. "Translations" means the result from the translation service?
3. "the current one" means the stored translations' score?
4. "accepted" means that this result and the score from the machine translation service is the one that will get written to the database?

---

Additionally, if I'm reading the code right, for LLM-based services it seems to be that:

• they always produce the same score for their results which equals their maximum score
• all services are set to to the same maximum score

So what happens if you let multiple LLM-based services bulk translate at once? Which one "wins"?

[nijel]

The scoring for translation memory is based on the match. I think some external services (MyMemory) also utilize this. For all others the scoring is a static hint to order the result. This mostly comes from time when we could somehow rank the machine translation services based on our experience, but it is not really useful with LLMs.

There are existing issues that would cover ordering:

• Add-on ordering #3315
• Set priority for Machine translation engines in Automatic Translation Addon. #5770

[michael-smt]

Thanks for the explanation. I should rewrite these two sentences then 🙂

Another drawback with these fixed scores is that the translation memory can still outrank a machine translation when the similarity is close enough. So it's not possible to do a batch autotranslate run with both the translation memory for 100% matches and the LLMs for everything else...

[nijel]

Add-on ordering + multiple auto translate add-ons would address this.