Skip to content

dev-20261015

@martincivan martincivan tagged this 15 Oct 02:24
### Summary

Slovak and Czech datasets are effectively BM25-blind today. The tokenizer only keeps ASCII letter runs whole, so every accented word is fragmented before indexing (`škola` -> `š kola`) and the keyword leg of hybrid search never matches. Neither language has a Snowball stemmer either, so the analyzer inherits whatever stemmer the previously processed dataset left behind.


Co-authored-by: mcivan <martin.civan@nases.gov.sk>
Assets 2
Loading