lang=ja + font-fallback + Inconsolata + typesetting the string ".ph" = crash

[ctrlcctrlv]

MWE:

\begin{document}
\script[src=packages/font-fallback]
\script[src=packages/tate]
%\language[main=und] -- uncommenting me fixes everything
\font[family=Inconsolata]
.ph
\end{document}

Crash:

This is SILE v0.10.5
<issue932.tex>

! Some glyph(s) not available in any fallback font, run with '-d font-fallback' for more detail at:
	/usr/share/sile/core/baseclass.lua:259: in \vfill
stack traceback:
	/usr/share/sile/packages/font-fallback.lua:109: in function 'shapeToken'
	/usr/share/sile/core/base-shaper.lua:62: in function 'measureChar'
	/usr/share/sile/languages/ja.lua:11: in function 'converter'
	/usr/share/sile/core/measurement.lua:82: in function 'tonumber'
	/usr/share/sile/core/measurement.lua:30: in function '_pt_amount'
	/usr/share/sile/core/measurement.lua:108: in function '___add'
	/usr/share/sile/core/length.lua:107: in function '___add'
	/usr/share/sile/core/break.lua:459: in function 'checkForLegalBreak'
	/usr/share/sile/core/break.lua:545: in function 'doBreak'
	...
	/usr/share/sile/packages/bidi.lua:239: in function 'boxUpNodes'
	/usr/share/sile/core/typesetter.lua:566: in function 'leaveHmode'
	/usr/share/sile/classes/plain.lua:77: in function '?'
	/usr/share/sile/core/sile.lua:284: in function 'call'
	/usr/share/sile/core/baseclass.lua:259: in function 'finish'
	/usr/share/sile/core/inputs-texlike.lua:169: in function 'process'
	/usr/share/sile/core/sile.lua:237: in function 'readFile'
	/usr/bin/sile:62: in function </usr/bin/sile:62>
	[C]: in function 'xpcall'
	/usr/bin/sile:62: in main chunk
	[C]: in ?

Error detected:
	/usr/share/sile/core/base-shaper.lua:63: attempt to index field '?' (a nil value)
stack traceback:
	/usr/share/sile/core/base-shaper.lua:63: in function 'measureChar'
	/usr/share/sile/languages/ja.lua:11: in function 'converter'
	/usr/share/sile/core/measurement.lua:82: in function 'tonumber'
	/usr/share/sile/core/measurement.lua:30: in function '_pt_amount'
	/usr/share/sile/core/measurement.lua:108: in function '___add'
	/usr/share/sile/core/length.lua:107: in function '___add'
	/usr/share/sile/core/break.lua:459: in function 'checkForLegalBreak'
	/usr/share/sile/core/break.lua:545: in function 'doBreak'
	/usr/share/sile/core/typesetter.lua:275: in function 'breakIntoLines'
	/usr/share/sile/core/typesetter.lua:317: in function 'boxUpNodes'
	/usr/share/sile/packages/bidi.lua:239: in function 'boxUpNodes'
	/usr/share/sile/core/typesetter.lua:566: in function 'leaveHmode'
	/usr/share/sile/classes/plain.lua:77: in function '?'
	/usr/share/sile/core/sile.lua:284: in function 'call'
	/usr/share/sile/core/baseclass.lua:259: in function 'finish'
	/usr/share/sile/core/inputs-texlike.lua:169: in function 'process'
	/usr/share/sile/core/sile.lua:237: in function 'readFile'
	/usr/bin/sile:62: in function </usr/bin/sile:62>
	[C]: in function 'xpcall'
	/usr/bin/sile:62: in main chunk
	[C]: in ?

[simoncozens]

There are two bugs here; one is that tate changed the document language, which the PR above fixes. This makes it harder to see the other bug, which I need to work on more...

[simoncozens]

Even more minimal example:

\begin{document}
\script[src=packages/font-fallback]
\language[main=ja]
\font[family=Inconsolata]
.ph
\end{document}

[simoncozens]

Oh ho ho. Loading Japanese support defines the zenkaku width character, which is defined by measuring a sample Japanese character (we measure あ). But at this point font-fallback goes mad because we don't have any fonts defined which support that character. This is thoroughly understandable, although it's obviously annoying that it crashes - I can't really see what the correct behavior is here, other than raise a warning.

[alerque]

Would it help if we made defined the zenkaku width unit with a callback that would essentially delay initialization until first use? By the tme it is first used we should have a font for for the language that's going to be relevant, no?

[ctrlcctrlv]

If no fonts have あ for 全角, then we should assume that we are being run by a Japanese user who is making a very short document, perhaps even a business card. We should use the width of "M" instead, and add a hook checking for if a Japanese font is eventually loaded, to get a real 全角. This would be my fix.