Have emoji and U+28FF working in sphinx lualatex generated for titles and code

[LuisBL]

Describe the bug

with latex_elements = {"preamble": r"\setmainfont{Symbola}"} we have beautiful B&W emoji in LuaLaTeX generated PDF content but not on titles and not on code.

How to Reproduce

Below a sphinx to illustrate emoji in latexpdf:

$ sphinx-quickstart --no-batchfile --no-sep -p test_emoj -l en -a me -v 1.0 -q
$ cd test_emoj; tree
├── _build
├── conf.py
├── index.rst
├── Makefile
├── NotoColorEmoji.ttf
├── _static
└── _templates

add some content:

$ vim index.rst
$ cat index.rst 

=========
test_emoj
=========

Hello space 🚀 and ⣿
=====================

Space 🚀, hammer 🔨 and ⣿

.. code::

  ✔ app 8 layers [⣿⣿⣿]  Pulled
  Space 🚀, hammer 🔨 and ⣿

Set latex_engine and article lualatex:

echo "latex_engine ='lualatex'"  >> conf.py
echo "latex_theme='howto'"  >> conf.py

HTML emoji are ok

$ make html 
...
$ 

HTML emoji are ok

latexpdf with lualatex engine have no emoji:

$ make latexpdf 
...
$ evince _build/latex/test_emoj.pdf 

with setmainfont get get some emoji (B&W and only in content):

$ echo 'latex_elements = {"preamble": r"\setmainfont{Symbola}"}' >> conf.py
$ make latexpdf 
...
$ evince _build/latex/test_emoj.pdf 

with Symbola B&W emoji but not on title and code

The latex log file expose that in code:: latex look for emoji in FreeMono.otf and in title latex look for emoji in FreeSansBold.otf:

$ grep '^Missi' _build/latex/test_emoj.log 
$ grep '^Missi' _build/latex/test_emoj.log |sort | uniq
Missing character: There is no 🔨 (U+1F528) in font [FreeMono.otf]:mode=harf;s
Missing character: There is no 🚀 (U+1F680) in font [FreeMono.otf]:mode=harf;s
Missing character: There is no 🚀 (U+1F680) in font [FreeSansBold.otf]:mode=ha
Missing character: There is no ⣿ (U+28FF) in font [FreeMono.otf]:mode=harf;scr
Missing character: There is no ⣿ (U+28FF) in font [FreeSansBold.otf]:mode=harf
$

To see which one of my fonts has the ⣿.

$ java -jar /usr/share/texlive/texmf-dist/scripts/albatross/albatross.jar --detailed ⣿

On my unbuntu 22.04 the below fonts have the ⣿ character:

/usr/share/fonts/truetype/ancient-scripts/Symbola_hint.ttf
/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf
/usr/share/fonts/truetype/dejavu/DejaVuSans-BoldOblique.ttf
/usr/share/fonts/truetype/dejavu/DejaVuSans-Oblique.ttf
/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf
/usr/share/fonts/truetype/dejavu/DejaVuSansCondensed-Bold.ttf
/usr/share/fonts/truetype/dejavu/DejaVuSansCondensed.ttf
/usr/share/fonts/truetype/dejavu/DejaVuSerif
/usr/share/fonts/truetype/noto/NotoSansSymbols2-Regular.ttf

Environment Information

$ cat /etc/issue
Ubuntu 22.04.4 LTS \n \l
$

$ sphinx-build --bug-report
Please paste all output below into the bug report template



Platform:              linux; (Linux-6.5.0-26-generic-x86_64-with-glibc2.35)
Python version:        3.11.8 (main, Feb 25 2024, 21:35:23) [GCC 11.4.0])
Python implementation: CPython
Sphinx version:        7.2.6
Docutils version:      0.20.1
Jinja2 version:        3.1.3
Pygments version:      2.17.2
$

Sphinx extensions

extensions = []

Additional context

No response

[LuisBL]

Using #12195 n-peugnet solution I go it (almost) working:

Change latex_elements in conf.py:

latex_elements = {"preamble": r"""
\directlua{ luaotfload.add_fallback("emoji", {"[TwemojiMozilla.ttf]:mode=harf"}) }
\setmainfont{LatinModernRoman}[RawFeature={fallback=emoji},SmallCapsFont={* Caps}]
\setsansfont{LatinModernSans}[RawFeature={fallback=emoji}]
\setmonofont{DejaVuSansMono}[RawFeature={fallback=emoji},Scale=0.8]
""" }

content, titles and code Colors emoji in PDF !

But ⣿ character missing, in content, code and titles ;(

$ grep Missi _build/latex/test_emoj.log 
Missing character: There is no ⣿ (U+28FF) in font LatinModernRoman/B:mode=harf
Missing character: There is no ⣿ (U+28FF) in font LatinModernSans/B:mode=harf;
Missing character: There is no ⣿ (U+28FF) in font LatinModernRoman:mode=harf;s
Missing character: There is no ⣿ (U+28FF) in font DejaVuSansMono:mode=harf;scr
Missing character: There is no ⣿ (U+28FF) in font DejaVuSansMono:mode=harf;scr
Missing character: There is no ⣿ (U+28FF) in font DejaVuSansMono:mode=harf;scr
Missing character: There is no ⣿ (U+28FF) in font DejaVuSansMono:mode=harf;scr
$ 

How to have both hammer 🔨 and ⣿ (U+28FF) in the final PDF ?

[n-peugnet]

You can provide a chain of fallback fonts in add_fallback. I didn't test myself but check the description of the function in luaotfload's documentation.
I guess you could add some Noto symbolic font (or Symbola) to the chain.

[jfbu]

I applied @n-peugnet advice and this works at my locale

latex_engine = 'lualatex'
latex_theme = 'howto'

latex_elements = {
    "preamble": r"""
\directlua {
  luaotfload.add_fallback("emoji",
  {
     "[TwemojiMozilla.ttf]:mode=harf;",
     "[DejaVuSans.ttf]:mode=harf;",
  } 
  )
}
\setmainfont{LatinModernRoman}[RawFeature={fallback=emoji},SmallCapsFont={* Caps}]
\setsansfont{LatinModernSans}[RawFeature={fallback=emoji}]
\setmonofont{DejaVuSansMono}[RawFeature={fallback=emoji},Scale=0.8]
    """,
}

produces

[jfbu]

I changed the type of this ticket to question because it does not seem to be related to a bug in Sphinx LaTeX support, except if one considers support for color emojis should be built-in, on which topic I don't have myself any strong opinion!

[LuisBL]

Yes working like a charm !

[maurerle]

As I stumbled across this issue:
For other unicode characters like greek letters, it is enough to add \usepackage{commonunicode} to the preamble in the latex_elements config in the sphinx conf.py:

latex_elements = {
    'preamble': r"""
\usepackage{commonunicode}
""",
}