text

text

Utilities for working with Persian (Farsi) text.

import "github.com/amiranmanesh/go-persian-tools/text"

Normalization

text.FixArabic("علي كريم")  // علی کریم
text.Normalize("مقالهٔ من") // مقاله من

FixArabic is a conservative, meaning-preserving cleanup: it replaces the Arabic yeh and kaf with their Persian counterparts and strips vocalization marks. Use it on anything you are about to store.

Normalize goes further and is meant for comparison keys, not display. On top of FixArabic it folds the alef and hamza variants onto a plain alef, teh marbuta onto heh, and turns zero-width joiners into spaces:

text.Normalize("می‌روم") // "می روم"

It is idempotent, so you can safely normalize once when writing a record and again when querying it. Keep the original around for showing back to the user.

Why this matters

"علي" (Arabic yeh) and "علی" (Persian yeh) look identical in most fonts but are different code points, so a naive == says they differ. Normalizing both sides makes search, sorting and uniqueness behave the way a reader expects.

Keyboard layout

text.SwitchToPersianKey("sghl") // سلام
text.SwitchToEnglishKey("اثغ")  // hey

The usual fix for text typed with the wrong layout selected. Characters outside the layout, including uppercase letters and digits, are left as they are.

Transliteration

text.Finglish("سلام")     // salam
text.Finglish("خوب")      // khub
text.Finglish("خواهر")    // khahar
text.Finglish("رفتن")     // raftan
text.Finglish("سلام دنیا") // salam dania

How it works

Three passes, because Persian spelling and Latin spelling disagree about what a word is made of:

1. Split into words. Each word is transliterated on its own, so nothing leaks across a space.
2. Letters to phonemes. The letters that can be either a consonant or a vowel — ا و ی ه ع — are resolved from their position, and silent letters are dropped.
3. Phonemes to syllables. Persian does not write its short vowels, so most words arrive as consonants with no nucleus; this pass supplies one wherever a syllable needs it.

The rules this makes expressible:

Rule	Example
و between consonants is a long "u"	خوب → khub
و before or after a vowel is "v"	اهواز → ahvaz
ی after a consonant is a long "i"	خیابان → khiaban
ی after a vowel is "y"	چای → chay
the vav in خوا is silent	خواهر → khahar
a final ه after a consonant is "e"	خانه → khane
initial ای and او are long vowels	ایران → iran
every syllable gets a vowel	رفتن → raftan
r and n cannot close a word alone	مادر → madar

What it guesses

Persian does not write its short vowels, so nothing on the page separates "gol" from "gal". The transliterator supplies the most common one, an "a":

text.Finglish("گل")   // gal, not gol
text.Finglish("کتاب") // katab, not ketab

Homographs such as کرم (kerm, karam, kerem) have no single correct answer at all. Expect a readable approximation, not a reversible encoding.

Measured against the reference word lists that ship with the tests, the hit rate is 65% on the list the rules were developed against and 53% on a held-out list that was never used while tuning them. TestFinglishAccuracy fails if either figure drops.

Inspection

text.Reverse("سلام")                // مالس
text.CheckIsEnglish("ali")          // true
text.OnlyPersianAlpha("123شاهینhi") // شاهین

Constants

Constant	Value	Meaning
text.ZWNJ	U+200C	zero-width non-joiner, separates parts of a compound word
text.ZWJ	U+200D	zero-width joiner, forces a joining form

Notes

Every function is pure, holds no state and is safe for concurrent use. All of them operate on runes rather than bytes, so multi-byte characters are handled correctly; invalid UTF-8 simply matches no rule and is dropped by the filters.