RFC 0001: Research program, roadmap and contribution model

[mbaneshi]

Status: Open for comment · Author: @mbaneshi · Created: 2026-10-08
Supersedes nothing. Builds on the kickoff discussion and the north-star README.

Summary

This RFC proposes the research program, roadmap and contribution model for persian-cli: an open lab whose goal is for a Persian speaker (and anyone writing an Arabic-script or other right-to-left script) to do everything in a terminal an English speaker can (read, write, edit, search, select, copy and paste) in any terminal, multiplexer, editor or CLI app.

It asks for input on five open questions (§6). Comment inline, or open an issue for anything concrete.

1. Motivation

The terminal is a grid of fixed-width cells, filled left to right, one character per cell. Arabic script breaks three of its assumptions:

The grid assumes…	Arabic script needs…	What breaks
Each cell is independent	Shaping: letter forms depend on neighbours	Disconnected letters
Text runs left to right	Bidi: visual order depends on the paragraph (UAX #9)	Reversed words, misplaced numbers
One character = one cell	Variable width: لا, ZWNJ, diacritics	App and terminal disagree on the cursor

Under all three is an unresolved ownership question: is bidi the terminal's job or the application's? If both do it, text is reversed twice. If neither does, it stays backwards.

2. What we already know

We read the code of ten existing projects (prior work) and mapped each onto a nine-layer model. The headline results:

1. Every tool that rewrote the text (presentation forms, pre-reversed strings) broke copy, search or escape sequences, and gave up on full-screen apps. The renderers that kept a logical buffer got copy right.
2. Width (layer 3) is unsolved everywhere. No project fits shaped text to the cell grid.
3. Multiplexers (layer 8): 0 of 10. Nobody has looked at tmux or Zellij.
4. Nobody negotiates bidi ownership. It is applied unconditionally or by guesswork, including inside full-screen apps. None of them use the BiDi in Terminal Emulators draft (2019).
5. Input (layer 2) is ignored. No project handles IME composition or ZWNJ entry.

3. Principles (normative)

1. Store and transmit text in logical order; reorder only at display time. No presentation-form substitution, no pre-reversed strings.
2. One owner per concern, with an explicit handshake between terminal, multiplexer and app.
3. Standards over hacks: Unicode (UAX docs: prior-art catalog for Persian/RTL in the terminal #9, feat(site): first-class Prior work section — on-site catalog + Persian notes #11, #29), ECMA-48, the terminal BiDi draft. Where they are silent or wrong, we document it and propose a change.
4. Measure before fixing. Every claim is backed by a reproducible test case; a fix is something that changes a score.
5. Upstream first. Fix the tools people already use; build our own only where upstream can't or won't move.
6. Script-general. Persian drives the work, but nothing may work only for Persian.
7. Never make it worse. Bidi-unaware apps must not degrade in a bidi-aware terminal.

4. Proposal: roadmap

Each phase is a milestone and is tracked on the roadmap project.

Phase	Output	Exit criterion
0. Root-cause map	Primary-source notes on who owns what, layer by layer	Every layer has a written root-cause note with sources
1. Test corpus & scorecard	Hard test strings plus a harness that scores a terminal + multiplexer + app stack	v0 scorecard published for at least 3 terminals
2. Fonts & terminals	Font recommendations, configs, upstream rendering fixes	Measurable score gains landed upstream
3. Multiplexers	Shaping and bidi passed through tmux and Zellij	Corpus passes inside a multiplexer as it does outside
4. Editors & TUI frameworks	Fixes or plugins for Neovim, readline/zle, Ink, ratatui…	At least one editor and one TUI framework pass the corpus
5. Ownership standard	Work with maintainers to finish and adopt a bidi negotiation standard	Implemented by at least two terminals and one app

Immediate focus (Phase 0 → 1): the three gaps nobody has touched: width (layer 3), multiplexers (layer 8) and bidi ownership (layer 5), plus a v0 test corpus.

5. Contribution model

• Issues carry a layer/* and a type/* label and belong to a phase milestone. Good entry points are tagged good first issue and help wanted.
• Breakage reports are the most valuable contribution right now: terminal, multiplexer, app and font versions, the exact input, and expected vs. actual output.
• RFCs like this one live in Discussions with an RFC NNNN: prefix. Accepted RFCs are summarised into docs/ and linked from the issues that implement them.
• Research notes cite primary sources and keep verified separate from claimed.
• Licensing: code under Apache-2.0; research docs and the test corpus under CC BY 4.0.
• Languages: English is the source language; Persian is a first-class locale on the site, and more RTL locales are welcome (Arabic first).

6. Open questions (please weigh in)

1. Conformance levels. Should the scorecard be a flat per-layer pass/fail, or graded levels (e.g. L0 Unicode → L1 glyph → L2 bidi → L3 layout → L4 interaction → L5 apps)?
2. First terminals to score. Proposal: WezTerm, kitty, Ghostty, VTE (GNOME Terminal), iTerm2, Windows Terminal. What's missing?
3. Harness approach. Screenshot comparison, terminal-state introspection (e.g. reading the grid back), or both? Screenshots are universal but fragile; introspection is precise but not every terminal exposes it.
4. Bidi ownership. Adopt the 2019 BiDi in Terminal Emulators draft as-is, extend it, or propose something new? Who should we talk to first (VTE, WezTerm, kitty maintainers)?
5. Scope of scripts. Persian first, but which others belong in the v0 corpus: Arabic, Urdu, Hebrew (RTL, non-joining), Syriac, N'Ko?

7. Next steps

Comments stay open for two weeks. Afterwards the outcome is summarised in docs/rfcs/0001-roadmap.md and the open questions are turned into issues.

[mbaneshi]

Tracking. The roadmap in this RFC is now set up on GitHub:

• Milestones: Phase 0 → Phase 5
• Board: persian-cli roadmap, with Phase, Layer and Work type fields
• Labels: layer/1-encoding … layer/9-apps, type/* and rfc

Issues opened from this RFC:

Issue	Phase	Open question
#13 Root cause: width	0	n/a
#14 Root cause: bidi ownership	0	Q4
#15 Root cause: multiplexers	0	n/a
#16 Reproduce: where Persian breaks (good first issue)	0	n/a
#17 Test corpus v0 (good first issue)	1	Q5
#18 Scorecard harness design	1	Q1, Q3
#19 Site: Arabic locale	0	n/a

Q2 (which terminals to score first) stays here in the thread. Please reply with your terminal of choice and why.