Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve speed and result quality for the translation memory api #612

Closed
pascalchevrel opened this issue Feb 9, 2016 · 0 comments
Closed

Improve speed and result quality for the translation memory api #612

pascalchevrel opened this issue Feb 9, 2016 · 0 comments
Assignees
Labels

Comments

@pascalchevrel
Copy link
Member

@pascalchevrel pascalchevrel commented Feb 9, 2016

Not related to the Hackathon.

I found more perf improvements for the tm/global api and it also makes the code simpler. I also found out that with the simplification we now get a bit more results (not noticeable for big locales because looking for strings almost always returns a lot of results, but for locales that only do Gaia for example, a search that would yield 3 results now would return maybe 5 results).

One thing I wonder is if we should also set a minimum quality threshold (currently the value is set to 0, that means we return all results even if the quality is very low) to maybe a value of 50. It has no impact on the speed but maybe a higher default value would be a good thing.

@pascalchevrel pascalchevrel self-assigned this Feb 9, 2016
pascalchevrel added a commit to pascalchevrel/transvision that referenced this issue Feb 9, 2016
… memory api

* For a global search, don't loop through all repositories, only those a locale supports (major perf gain for small locales)
* Simplify ShowResults::getTranslationMemoryResults(), send a prepared array without entities as  keys
* Ignore empty localized strings in search results
pascalchevrel added a commit to pascalchevrel/transvision that referenced this issue Feb 9, 2016
… memory api

* For a global search, don't loop through all repositories, only those a locale supports (major perf gain for small locales)
* Simplify ShowResults::getTranslationMemoryResults(), send a prepared array without entities as  keys
* Ignore empty localized strings in search results
pascalchevrel added a commit to pascalchevrel/transvision that referenced this issue Feb 9, 2016
…ion memory api

* For a global search, don't loop through all repositories, only those a locale supports (major perf gain for small locales)
* Simplify ShowResults::getTranslationMemoryResults(), send a prepared array without entities as keys
* Ignore empty localized strings in search results (happens when the entity is in the repo but empty)

For locales that do everything, this has no perf impact, for most locales that will give a small perf boost (most locales don't do mozilla-central), for locales that have less repositories (just started and only have Aurora for example, or only translating Gaia, or not translating Firefox for iOS, the perf gain can be significant. For Guarani for example, request times are divided by 5 on my machine compared to our current code. Same for memory use.
pascalchevrel added a commit to pascalchevrel/transvision that referenced this issue Feb 9, 2016
…ion memory api

* For a global search, don't loop through all repositories, only those a locale supports (major perf gain for small locales)
* Simplify ShowResults::getTranslationMemoryResults(), send a prepared array without entities as keys
* Ignore empty localized strings in search results (happens when the entity is in the repo but empty)

For locales that do everything, this has no perf impact, for most locales that will give a small perf boost (most locales don't do mozilla-central), for locales that have less repositories (just started and only have Aurora for example, or only translating Gaia, or not translating Firefox for iOS, the perf gain can be significant).
For Guarani for example, request times are divided by 5 on my machine compared to our current code. Same for memory use.
pascalchevrel added a commit to pascalchevrel/transvision that referenced this issue Feb 9, 2016
…ion memory api

* For a global search, don't loop through all repositories, only those a locale supports (major perf gain for small locales)
* Simplify ShowResults::getTranslationMemoryResults(), send a prepared array without entities as keys
* Ignore empty localized strings in search results (happens when the entity is in the repo but empty)

For locales that do everything, this has no perf impact, for most locales that will give a small perf boost (most locales don't do mozilla-central), for locales that have less repositories (just started and only have Aurora for example, or only translating Gaia, or not translating Firefox for iOS, the perf gain can be significant).
For Guarani for example, request times are divided by 5 on my machine compared to our current code. Same for memory use.
pascalchevrel added a commit that referenced this issue Feb 9, 2016
issue #612: Improve speed and result quality for global translation memory api
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.