Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Black-list some locales by default + make it configurable #41

Closed
baptistejamin opened this issue Mar 15, 2019 · 3 comments
Closed

Black-list some locales by default + make it configurable #41

baptistejamin opened this issue Mar 15, 2019 · 3 comments
Assignees
@valeriansaliou

Comments

@baptistejamin
Copy link
Collaborator

Jav: Javanese
Orm: Oromoo
Hau: Hausa
Kur: Kurdish
Yor: : Yoruba
Uzb: Uzbek
Igbo: Ibo
Ceb: Cebuano
Tgl: Tagalog
Mlg : Malagasy
Nya : Chewa
Kin: Kinyarwanda
Zul: Zulu
Som: Soomaaliga
Ilo: Ilokano
Uig: Ouïghour
Hat: Haitian Creole
Aka: Akan
Sna: ChiShona
Afr: Afrikaans
Run: Ikirundi
Tuk: Turkmen
Epo: Esperanto
@valeriansaliou valeriansaliou self-assigned this Mar 15, 2019
@valeriansaliou valeriansaliou changed the title Remove locales Black-list some locales by default + make it configurable Mar 15, 2019
@valeriansaliou
Copy link
Owner

Black-listing the locales above from the ngram shows a performance increase of 50% on Latin locales.

@valeriansaliou
Copy link
Owner

We should blacklist some locales by default in the default configuration file (see above), and let the user blacklist locales at will.

@valeriansaliou
Copy link
Owner

Closed. With all --release optimizations, we go sub-millisec with a 50% difference in time to process lexer-based requests. It's now all right to keep the ngram as-is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@valeriansaliou valeriansaliou

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@baptistejamin @valeriansaliou