Korean tokenizer for Orama (garu-orama-tokenizer)

[ongjin]

Sharing a small integration in case it's useful to anyone doing Korean search with Orama.

Orama's default tokenizer segments Korean poorly, because Korean glues particles/endings onto stems — a search for 학교 ("school") misses 학교에 / 학교를, and 먹었다 won't match 먹다. garu-orama-tokenizer plugs a Korean morphological analyzer (Garu — runs in-browser via WebAssembly, 1 MB model bundled, no server) into Orama as a custom tokenizer, so inflected forms index to the same stem.

import { create, insert, search } from '@orama/orama'
import { createTokenizer } from 'garu-orama-tokenizer'

const db = await create({
  schema: { title: 'string' },
  components: { tokenizer: await createTokenizer() },
})
await insert(db, { title: '학교에서 점심을 먹었다' })
await search(db, { term: '먹다' }) // matches

• npm: https://www.npmjs.com/package/garu-orama-tokenizer
• Garu: https://github.com/ongjin/garu (MIT, deterministic, no neural net)

If it'd help readers, happy to see it linked from the tokenizer/plugins docs — but no pressure at all. Disclosure: I'm the author of Garu.