Interactive Demo

Interactive Demo: How Laren Transliterates

Let's follow the journey of typing 7abibi (meaning "my love" / حبيبي) character by character, and see exactly what happens at every step inside Laren.

---

🎬 The Big Picture

graph LR
    A["🧑 You type:<br/><b>7 a b i b i</b>"]
    A --> B["🔄 Rule Engine<br/><i>What could each<br/>letter mean?</i>"]
    B --> C["🌳 DFS Explorer<br/><i>Try every possible<br/>combination</i>"]
    C --> D["📖 Dictionary<br/><i>Which combos are<br/>real Arabic words?</i>"]
    D --> E["🏆 Ranker<br/><i>Sort by how<br/>common they are</i>"]
    E --> F["📋 You see:<br/><b>حبيبي</b>"]

    style A fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
    style F fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px

Loading

---

Step 1: You Start Typing

Every time you press a key, Laren processes the entire buffer so far. Let's watch:

Keystroke 1: 7

Buffer: [ 7 ]

graph TD
    subgraph "What does '7' mean in Arabic?"
        K["⌨️ You typed: <b>7</b>"]
        K --> M1["ح<br/>(Haa)<br/><i>the deep H sound</i>"]
    end

    style K fill:#e1f5fe
    style M1 fill:#fff9c4

Loading

Only one possibility! The number 7 always means ح (Haa) — that guttural H sound that doesn't exist in English. This is one of the most iconic Arabizi conventions.

Keystroke 2: 7a

Buffer: [ 7 a ]

graph TD
    subgraph "What does 'a' mean?"
        K["⌨️ You typed: <b>7a</b>"]
        K --> A1["Option 1:<br/><b>∅ (nothing)</b><br/><i>'a' is a short vowel,<br/>Arabic doesn't write these!</i>"]
        K --> A2["Option 2:<br/><b>ا</b> (Alef)<br/><i>'a' is a long vowel</i>"]
        K --> A3["Option 3:<br/><b>أ</b> (Alef + Hamza)<br/><i>glottal stop + a</i>"]
        K --> A4["Option 4:<br/><b>ى</b> (Alef Maqsura)<br/><i>word-final 'a' sound</i>"]
        K --> A5["Option 5:<br/><b>ه</b> (Ha)<br/><i>light h at end of words</i>"]
        K --> A6["Option 6:<br/><b>ة</b> (Ta Marbuta)<br/><i>feminine ending</i>"]
    end

    style K fill:#e1f5fe
    style A1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
    style A2 fill:#fff9c4
    style A3 fill:#fff9c4
    style A4 fill:#fff9c4
    style A5 fill:#fff9c4
    style A6 fill:#fff9c4

Loading

6 possibilities for just the letter 'a'! This is where the magic happens. The most likely option (highlighted in green) is SKIP — meaning this 'a' is a short vowel that Arabic simply doesn't write. Think of it like silent letters in English.

So after just 2 keystrokes, we already have 6 different paths to explore:

Path 1: ح      (7=ح, a=skip)     ← most likely
Path 2: حا     (7=ح, a=ا)
Path 3: حأ     (7=ح, a=أ)
Path 4: حى     (7=ح, a=ى)
Path 5: حه     (7=ح, a=ه)
Path 6: حة     (7=ح, a=ة)

Keystroke 3: 7ab

Buffer: [ 7 a b ]

graph TD
    subgraph "The tree branches further"
        R["⌨️ <b>7ab</b>"] --> P1["ح"]
        R --> P2["حا"]
        R --> P3["حأ"]
        R --> P4["حى"]
        R --> P5["حه"]
        R --> P6["حة"]

        P1 -->|"b = ب"| R1["حب"]
        P2 -->|"b = ب"| R2["حاب"]
        P3 -->|"b = ب"| R3["حأب"]
        P4 -->|"b = ب"| R4["حىب"]
        P5 -->|"b = ب"| R5["حهب"]
        P6 -->|"b = ب"| R6["حةب"]
    end

    style R fill:#e1f5fe
    style R1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
    style R2 fill:#fff9c4
    style R3 fill:#ffecb3
    style R4 fill:#ffecb3
    style R5 fill:#ffecb3
    style R6 fill:#ffecb3

Loading

b is simple — it always means ب (Ba). But because 'a' had 6 options, we now have 6 branches, each getting a ب appended.

Keystroke 4: 7abi

Buffer: [ 7 a b i ]

Now i has 5 possibilities (just like a), so each of our 6 branches splits into 5:

6 branches × 5 options = 30 paths!

graph TD
    subgraph "The explosion of possibilities"
        R["From 'b' we had<br/>6 branches"]
        R -->|"× 5 options for 'i'"| E["<b>30 paths</b><br/>after just 4 characters!"]

        E --> EX1["حبب (7=ح a=skip b=ب i=skip)"]
        E --> EX2["حبي (7=ح a=skip b=ب i=ي)"]
        E --> EX3["حبا (7=ح a=skip b=ب i=ا)"]
        E --> EX4["حابب (7=ح a=ا b=ب i=skip)"]
        E --> EX5["... 26 more paths"]
    end

    style R fill:#e1f5fe
    style E fill:#fff3e0,stroke:#e65100,stroke-width:2px
    style EX1 fill:#c8e6c9
    style EX2 fill:#c8e6c9

Loading

Keystroke 5: 7abib

Buffer: [ 7 a b i b ]

30 branches × 1 option (b=ب) = 30 paths

Keystroke 6 (final): 7abibi

Buffer: [ 7 a b i b i ]

30 branches × 5 options for final 'i' = 150 total paths!

---

Step 2: The Explosion — All Possible Arabic Forms

From just 6 characters, Laren generated ~150 possible Arabic strings. Here are some of them:

graph TD
    subgraph "🌳 150 Possible Arabic Forms from '7abibi'"
        T["7abibi"] --> G1["حبيبي"]
        T --> G2["حبيبى"]
        T --> G3["حابيبي"]
        T --> G4["حبببي"]
        T --> G5["حبيبا"]
        T --> G6["حأبيبي"]
        T --> G7["حبيبأ"]
        T --> G8["حابيبى"]
        T --> G9["... ~142 more"]
    end

    style T fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
    style G1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:3px
    style G2 fill:#fff9c4
    style G3 fill:#fff9c4
    style G4 fill:#ffecb3
    style G5 fill:#ffecb3
    style G6 fill:#ffecb3
    style G7 fill:#ffecb3
    style G8 fill:#ffecb3
    style G9 fill:#eeeeee

Loading

Most of these are nonsense! That's where the dictionary comes in.

---

Step 3: Dictionary Lookup — Finding Real Words

Now Laren checks each of the ~150 forms against its 375,000 word Arabic dictionary. The dictionary is stored as a trie (prefix tree) for lightning-fast lookups.

How the Trie Works

Think of the trie as a tree where each branch is a letter. To look up a word, you follow the branches:

graph TD
    subgraph "📖 Inside the Dictionary Trie"
        ROOT["🌱 Root"] --> ح["ح"]
        ROOT --> س["س"]
        ROOT --> ع["ع"]

        ح --> حب["حب"]
        ح --> حا["حا"]

        حب --> حبي["حبي"]
        حب --> حبل["حبل<br/><i>rope (freq: 4200)</i>"]

        حبي --> حبيب["حبيب<br/><i>beloved (freq: 5800)</i>"]

        حبيب --> حبيبي["🎯 حبيبي<br/><i>my love (freq: 5600)</i>"]
        حبيب --> حبيبة["حبيبة<br/><i>beloved f. (freq: 5400)</i>"]
        حبيب --> حبيبه["حبيبه<br/><i>beloved f. (freq: 5200)</i>"]

        حا --> حال["حال<br/><i>condition (freq: 5100)</i>"]
    end

    style حبيبي fill:#c8e6c9,stroke:#2e7d32,stroke-width:3px
    style ROOT fill:#e1f5fe

Loading

The Lookup Process

flowchart TD
    subgraph "Checking each candidate against the dictionary"
        A["حبيبي"] -->|"Follow: ح → حب → حبي → حبيب → حبيبي"| R1["✅ FOUND!<br/>freq: 5600"]
        B["حابيبي"] -->|"Follow: ح → حا → حاب..."| R2["❌ Not found"]
        C["حبيبى"] -->|"Follow: ح → حب → حبي → حبيب → حبيبى"| R3["✅ FOUND!<br/>freq: 4100"]
        D["حبببي"] -->|"Follow: ح → حب → حبب..."| R4["❌ Not found"]
        E["حأبيبي"] -->|"Follow: ح → حأ..."| R5["❌ Not found"]
        F["... 145 more"] -->|"Most fail"| R6["❌ Not found"]
    end

    style R1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
    style R2 fill:#ffcdd2
    style R3 fill:#c8e6c9
    style R4 fill:#ffcdd2
    style R5 fill:#ffcdd2
    style R6 fill:#ffcdd2

Loading

Out of ~150 candidates, only a handful are real Arabic words. The dictionary acts as a reality filter.

---

Step 4: Ranking — Best Match First

The surviving candidates are sorted by frequency — how commonly used they are in Arabic:

graph TD
    subgraph "🏆 Ranking by Frequency"
        direction TB
        R1["🥇 حبيبي<br/><b>freq: 5600</b><br/><i>my love — extremely common</i>"]
        R2["🥈 حبيبى<br/><b>freq: 4100</b><br/><i>my love (alt spelling)</i>"]
        R3["🥉 حبيبه<br/><b>freq: 3800</b><br/><i>beloved (informal f.)</i>"]
    end

    R1 ~~~ R2 ~~~ R3

    style R1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:3px
    style R2 fill:#fff9c4
    style R3 fill:#fff9c4

Loading

What are frequency scores?

Think of it as a popularity contest:

┌─────────────────────────────────────────────────────────┐
│ Frequency Scale                                         │
│                                                         │
│ 7+ ████████████████████████████████  في، من، على       │
│    "the", "from", "on" — used in almost every sentence  │
│                                                         │
│ 5-7 ████████████████████            حبيبي، كتاب        │
│     Common everyday words                               │
│                                                         │
│ 3-5 ████████████                    ديمقراطية           │
│     Less common, formal words                           │
│                                                         │
│ 1-3 ██████                          Rare / archaic      │
│     Specialized terms                                   │
└─────────────────────────────────────────────────────────┘

---

Step 5: Fallback — When the Dictionary Has No Match

What if you type something that's not in the dictionary? Like a name or slang?

flowchart TD
    A["You type: <b>shfeeq</b>"] --> B["DFS generates ~200 forms"]
    B --> C{"Dictionary check"}
    C -->|"0 matches found"| D["😬 Empty results?"]
    D --> E["NO! Laren shows<br/><b>skeletal forms</b> as fallback"]

    E --> F["شفيق<br/><i>most likely form<br/>(short vowels skipped)</i>"]
    E --> G["شفيق<br/><i>alternative voweling</i>"]
    E --> H["شافيق<br/><i>with explicit alef</i>"]

    style A fill:#e1f5fe
    style D fill:#ffcdd2
    style F fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
    style G fill:#fff9c4
    style H fill:#fff9c4

Loading

Laren never shows empty results. If the dictionary doesn't have the word, the raw transliteration forms are shown instead.

---

Step 6: History — Laren Remembers You

The first time you type 7abibi and select حبيبي, Laren saves your choice:

sequenceDiagram
    participant You
    participant Laren
    participant History as 💾 History File

    Note over You,History: First time
    You->>Laren: Type "7abibi"
    Laren-->>You: 1. حبيبي  2. حبيبى  3. حبيبه
    You->>Laren: Select #1 (حبيبي)
    Laren->>History: Save: 7abibi → حبيبي

    Note over You,History: Next time (even after reboot!)
    You->>Laren: Type "7abibi"
    Laren->>History: Lookup "7abibi"
    History-->>Laren: Found: حبيبي
    Laren-->>You: ⭐ حبيبي (your choice)<br/>────────────<br/>1. حبيبى<br/>2. حبيبه

Loading

Your preferred word now always appears at the top with a star, separated from the other candidates.

---

The Complete Journey

Here's everything that happens in the ~1 millisecond between your keystroke and seeing results:

flowchart TD
    A["⌨️ <b>You type: 7abibi</b>"] --> B["📝 Buffer updated"]

    B --> C["1️⃣ <b>NORMALIZE</b><br/>Lowercase, collapse repeats<br/><i>'7aBiBi' → '7abibi'</i>"]

    C --> D["2️⃣ <b>EXPAND (DFS)</b><br/>Try every combination<br/><i>~150 Arabic forms generated</i>"]

    D --> E["3️⃣ <b>DICTIONARY FILTER</b><br/>Check each form against 375k words<br/><i>~5 real words survive</i>"]

    E --> F["4️⃣ <b>FALLBACK</b><br/>Fill remaining slots with<br/>skeletal forms if needed"]

    F --> G["5️⃣ <b>RANK</b><br/>Sort by word frequency<br/><i>حبيبي (5600) → #1</i>"]

    G --> H["6️⃣ <b>HISTORY CHECK</b><br/>User picked this before?<br/>Move to top with ⭐"]

    H --> I["📋 <b>CANDIDATE LIST</b><br/>Up to 50 results, paginated"]

    I --> J["👁️ <b>YOU SEE:</b><br/>⭐ حبيبي<br/>─────────<br/>1. حبيبى<br/>2. حبيبه<br/>..."]

    J --> K["✅ <b>Press Space</b><br/>'حبيبي ' inserted into your app"]

    style A fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
    style J fill:#fff9c4,stroke:#f9a825,stroke-width:2px
    style K fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
    style D fill:#fff3e0,stroke:#e65100
    style E fill:#e8f5e9,stroke:#2e7d32

Loading

---

Try It Yourself!

Example 1: salam → سلام (peace)

s → [س, ص]           2 options
a → [∅, ا, أ, ى, ه, ة]  6 options
l → [ل]              1 option
a → [∅, ا, أ, ى, ه, ة]  6 options
m → [م]              1 option
                      ─────────
Total:                2 × 6 × 1 × 6 × 1 = 72 paths
Dictionary matches:   سلام ✅  سلم ✅
Winner:              سلام (freq: 5230)

Example 2: 3arab → عرب (Arab)

3 → [ع]              1 option
a → [∅, ا, أ, ى, ه, ة]  6 options
r → [ر]              1 option
a → [∅, ا, أ, ى, ه, ة]  6 options
b → [ب]              1 option
                      ─────────
Total:                1 × 6 × 1 × 6 × 1 = 36 paths
Dictionary matches:   عرب ✅  عراب ✅  عارب ✅
Winner:              عرب (freq: 5100)

Example 3: el7amdulellah → الحمدلله (praise God)

This one is more complex:
  e → 5 options
  l → 1 option
  7 → 1 option
  a → 6 options
  m → 1 option
  d → 2 options (د or ض)
  u → 3 options
  l → 1 option
  e → 5 options
  l → 1 option
  l → 1 option  (wait — "ll" is a digraph → ل)
  a → 6 options
  h → 2 options (ه or ح)
                      ─────────
Total:               5×1×1×6×1×2×3×1×5×1×1×6×2 = 10,800 paths!

But max_expansions = 4096, so Laren stops after 4096 paths.
Still finds: الحمدلله ✅
Winner:     الحمدلله (freq: 6200)

---

Why This Approach Works

graph TD
    subgraph "The Arabizi Problem"
        P1["Arabic doesn't write<br/>short vowels"]
        P2["Multiple Arabic letters<br/>share the same Latin letter"]
        P3["Every person spells<br/>Arabizi differently"]
    end

    subgraph "Laren's Solution"
        S1["SKIP sentinel handles<br/>missing vowels"]
        S2["DFS explores ALL<br/>possible interpretations"]
        S3["Dictionary filters out<br/>nonsense combinations"]
    end

    P1 --> S1
    P2 --> S2
    P3 --> S3

    S1 --> R["Result: type however you want,<br/>Laren figures out what you meant"]
    S2 --> R
    S3 --> R

    style R fill:#c8e6c9,stroke:#2e7d32,stroke-width:3px
    style P1 fill:#ffcdd2
    style P2 fill:#ffcdd2
    style P3 fill:#ffcdd2
    style S1 fill:#e1f5fe
    style S2 fill:#e1f5fe
    style S3 fill:#e1f5fe

Loading

The key insight: Instead of trying to guess what the user meant, Laren tries everything and lets the dictionary decide. This is why it works even when people spell Arabizi differently — 7abibi, habibi, 7abeby all lead to حبيبي.