Hyphen character missing when using luatex with DVI output

[jeremy-engel]

When using lualatex --output-format=dvi, if the hyphen character is followed by a non-space character, then the hyphen is missing from the output. Consider the following example:

\documentclass{article}

\begin{document}

A-B

-

A-\hspace{0pt}B

A-

-B

\end{document}

Compiling with latex and converting the DVI file with dvisvgm results in

With lualuatex --output-format=dvi, the resulting SVG file after conversion is

This is with the following versions:

This is LuaTeX, Version 1.17.0 (TeX Live 2023)
Development id: 7581

dvisvgm 3.0.3 (x86_64-pc-linux-gnu)
-----------------------------------
brotli:      1.0.9
clipper:     6.2.1
freetype:    2.13.0
Ghostscript: 9.55.0
kpathsea:    6.3.5
potrace:     1.16
xxhash:      0.8.1
zlib:        1.2.13

[muzimuzhi]

Looks like \automatichyphenmode=2 (defaults 0) works, but I don't know why.

The LuaTeX dvi outputs with \automatichyphenmode set to 0, 1, and 2 are already different, so the culprit might not be dvisvgm.

\documentclass{article}
\pagestyle{empty}
\def\testhyphens{%
  A-B\par
  -\par
  A-\hspace{0pt}B\par
  A-\par
  -B\par\bigskip}

\begin{document}
\automatichyphenmode=0
\testhyphens

\automatichyphenmode=1
\testhyphens

\automatichyphenmode=2
\testhyphens
\end{document}

Running

lualatex --output-format=dvi dvisvgm-gh255
dviasm dviasm dvisvgm-gh255.dvi > dvisvgm-gh255.dump

then in dvisvgm-gh255.dump

\automatichyphenmode=0	\automatichyphenmode=1	\automatichyphenmode=2
push: right: 77pt down: 63pt fnt: [...] at 10pt set: '\x1b"' pop: push: right: 77pt down: 75pt set: '@' pop: push: right: 77pt down: 87pt set: '\x1b@"' pop: push: right: 77pt down: 99pt set: '\x1b@' pop: push: right: 77pt down: 111pt set: '"' pop:	push: right: 77pt down: 135pt set: '\x1b"' pop: push: right: 77pt down: 147pt set: '@' pop: push: right: 77pt down: 159pt set: '\x1b@"' pop: push: right: 77pt down: 171pt set: '\x1b@' pop: push: right: 77pt down: 183pt set: '@"' pop:	push: right: 77pt down: 207pt set: '\x1b@"' pop: push: right: 77pt down: 219pt set: '@' pop: push: right: 77pt down: 231pt set: '\x1b@"' pop: push: right: 77pt down: 243pt set: '\x1b@' pop: push: right: 77pt down: 255pt set: '@"' pop:

You can see the first and last line of each test group with \automatichyphenmode=2 are different from when \automatichyphenmode is set to other values.

[mgieseki]

@muzimuzhi Awesome, thank you for the detailed analysis!

dvisvgm indeed converts the DVI files correctly. The initial one created by LuaTeX doesn't contain the missing hyphens so that dvisvgm can't process them.

[muzimuzhi]

With lualuatex --output-format=dvi and \automatichyphenmode=(0|1), one also got

Missing character: There is no @ (U+0040) in font nullfont!
Missing character: There is no @ (U+0040) in font nullfont!

(\tracinglostchars=3 can be used to convert them into errors.)

[muzimuzhi]

Turns out it's a luaotfload problem, see latex3/luaotfload#270. luaotfload maintainer Marcel Krüger has already kindly pushed a fix to dev branch, which you can test with locally by executing

git clone --branch=dev --depth=1 git@github.com:latex3/luaotfload.git
cd luaotfload
# needs l3build package
l3build install

cd /path/to/tex/files
# needs latex-base-dev and latex-bin-dev packages perhaps
# should be equivalent to "lualatex-dev --output-format=dvi"
dvilualatex-dev dvisvgm-gh255
dvisvgm --font-format=woff2 dvisvgm-gh255

[jeremy-engel]

Great, thanks for the detailed explanation and for hunting down that luaotfload issue. I just confirmed that the dev branch for luaotfload does resolve this issue (and the problem I originally had that evolved into the MWE) as you said. I'll close this as the issue is in luaotfload and already fixed pending release. Thanks again.