When Unicode bidi is not enough

[jbezos]

When a document contains both left-to-right (LTR) and right-to-left (RTL) text, lualatex together with bidi=basic can normally handle the text without any explicit directional markup. However, there are special cases where the desired interpretation cannot be obtained from the characters themselves and therefore require intervention.

The basic and recommented mechanism is using \foreignlanguage{...}{...} to explicitly establish the language and directional context of a piece of text. An alternative approach is inserting the control characters LRM and RLM, i.e., invisible left-to-right or right-to-left mark at a particular boundary. Note babel doesn’t understand other control characters on purpose (Unicode UAX#9: “Where available, markup should be used instead of the explicit formatting characters”). See the babel manual for details.

LRM and RLM control characters can be inserted in a visible way with ^^^^200e and ^^^^200f. Sadly, these combinations can be displayed by editors in a somewhat weird way. On the other hand, some editors can also make the characters visible (for example, Notepad++). You may also define your own macros to insert them, according to your needs (e.g., you may prefer an active character).

This article follows the examples in the W3C's Inline bidi markup examples, but expresses the solutions using babel. The examples from the W3C are useful because they illustrate some situations in which the Unicode bidi may need additional information.

The difficulty arises when characters do not themselves have a strong direction. Characters such as: . , : ; ! ? ( ) + - are generally neutral and dependent on their surrounding context for bidi resolution.

\documentclass{article}

\usepackage[english, bidi=basic]{babel}
\babelprovide[onchar=ids fonts]{arabic}
\babelprovide[onchar=ids fonts]{hebrew}

% FreeSerif is not a great font, but it contains Latin, Arabic and
% Hebrew characters, so just a declaration is enough. Remember babel
% fills Language and Script for each language.
\babelfont{rm}{FreeSerif}

\begin{document}

In all example groups, the first one is the default Unicode result.

The title is “مفتاح معايير الويب!” in Arabic.

The title is “\foreignlanguage{arabic}{مفتاح معايير الويب!}” in Arabic.

The title is “مفتاح معايير الويب!^^^^200f” in Arabic.

The title is “مدخل إلى C++” in Arabic.

The title is “\foreignlanguage{arabic}{مدخل إلى \foreignlanguage{english}{C++}}” in Arabic.

The title is “\foreignlanguage{arabic}{مدخل إلى C++^^^^200e}” in Arabic.

We find the phrase ‘פעילות הבינאום’ 5 times on the page.

We find the phrase ‘\foreignlanguage{hebrew}{פעילות הבינאום}’ 5 times on the page.

We find the phrase ‘פעילות הבינאום’^^^^200e 5 times on the page.

The names of these states in arabic are مصر, البحرين and الكويت, respectively.

The names of these states in arabic are \foreignlanguage{arabic}{مصر}, البحرين and الكويت, respectively.

The names of these states in arabic are مصر,^^^^200e البحرين and الكويت, respectively.

\end{document}