Unexpected \babeltags behaviour with RTL languages when starting a new paragraph

[dcpurton]

Brief outline of the bug

When using \babeltags with RTL languages, the begin{tag}…\end{tag} environment gives unexpected behaviour when it starts a new paragraph. Under lualatex the paragraph direction remains LTR, but under xelatex it is implicitly set to RTL, hence the output is different. The xelatex output is still unexpected though, since the paragraph indent occurs on the left rather than the right.

This is because \babeltags uses \begin{otherlanguage*}{language}…\end{otherlanguage*} internally which is basically the same as \foreignlanguage{language}{…}.

I think it would make more sense to force \begin{language}…\end{language} to always behave like a paragraph. You could use \begin{language*}…\end{language*} for the current behaviour which seems more consistent with \begin{otherlanguage}{language} and \begin{otherlanguage*}{language}. This would match what polyglossia does and mean that document syntax could be the same for both babel and polyglosssia.

I realise this breaks backwards compatibility so may not be an acceptable solution.

Minimal example showing the bug

This MWE requires fontspec and the Noto Serif Hebrew font. (Sorry about the Hebrew diacritics not showing correctly in GitHub.)

\RequirePackage{latexbug}
\documentclass{article}
\usepackage[nil,bidi=default]{babel}
\babelprovide[import=en-AU,main]{australian}
\babelprovide[import=he]{hebrew}
\babelfont[australian]{rm}{Latin Modern Roman}
\babelfont[hebrew]{rm}{Noto Serif Hebrew}
\babeltags{english=australian, hebrew=hebrew}
\begin{document}
\begin{hebrew}
  \textsuperscript{\textenglish{1}}בְּרֵאשִׁ֖ית בָּרָ֣א אֱלֹהִ֑ים אֵ֥ת הַשָּׁמַ֖יִם וְאֵ֥ת הָאָֽרֶץ׃
  \textsuperscript{\textenglish{2}}וְהָאָ֗רֶץ הָיְתָ֥ה תֹ֨הוּ֙ וָבֹ֔הוּ וְחֹ֖שֶׁךְ עַל־פְּנֵ֣י תְהֹ֑ום וְר֣וּחַ
  אֱלֹהִ֔ים מְרַחֶ֖פֶת עַל־פְּנֵ֥י הַמָּֽיִם׃ \textenglish{(Genesis 1:1–2)}
\end{hebrew}

\begin{otherlanguage}{hebrew}
  \textsuperscript{\foreignlanguage{australian}{1}}בְּרֵאשִׁ֖ית בָּרָ֣א אֱלֹהִ֑ים אֵ֥ת הַשָּׁמַ֖יִם
  וְאֵ֥ת הָאָֽרֶץ׃ \textsuperscript{\foreignlanguage{australian}{2}}וְהָאָ֗רֶץ הָיְתָ֥ה תֹ֨הוּ֙
  וָבֹ֔הוּ וְחֹ֖שֶׁךְ עַל־פְּנֵ֣י תְהֹ֑ום וְר֣וּחַ אֱלֹהִ֔ים מְרַחֶ֖פֶת עַל־פְּנֵ֥י הַמָּֽיִם׃
  \foreignlanguage{australian}{(Genesis 1:1–2)}
\end{otherlanguage}
\end{document}

lualatex output:

xelatex output:

Log file (required) and possibly PDF file

babeltags.log (lualatex)

babeltags.log (xelatex)

[josephwright]

Successful migration: #12