How can I search mixed language text?

[yeonns]

Hello.

My markdown page contains "Korean" and "English" text like below

### 코드code

So, I want to search text with mixed language like "코드code", but I think it only supports one language per file/directory.
Is there any way searching with mixed language?

[alex-shpak]

Hi!
For searching flexsearch library is used.
I never tried to make mixed search settings for it, but you can give it a shot.

There is param in i18m called bookSearchConfig, where you can specify flexsearch indexing settings.
https://github.com/alex-shpak/hugo-book/blob/master/i18n/zh.yaml#L13

Current settings are taken from their docs
https://github.com/nextapps-de/flexsearch#cjk-word-break-chinese-japanese-korean

[alex-shpak]

Related issues in flexsearch repo
nextapps-de/flexsearch#207
nextapps-de/flexsearch#73

[yeonns]

Hi!
I will try editing bookSearchConfig.
Thank you 👍

[yeonns]

Hi!
I could search mixed language(Korean & English) with below bookSearchConfig
It was simpler than I thought.

- id: bookSearchConfig
  translation: |
    {
      split: " "
    }

I think It is little strange because I do not need any custom tokenize function, and just use split function. 😢
But the problem has been resolved, and thank you again! 😄

[alex-shpak]

Cool 👍

[marshall-999]

it is not work now.

[kjs104901]

This is working for me.

function(str){
  let result = []
  result.push.apply(result, str.replace(/[\x00-\x7F]/g, "").split(""));
  str.split(/[^a-zA-Z0-9\u00C0-\u00ff]/g).forEach(t => {
    if (t.length > 0) {
      result.push.apply(result, t.toLowerCase().split(/[\p{Z}\p{S}\p{P}\p{C}]+/u));
    }
  });
  return result;
}