Format expandable errors like other errors

[blefloch]

Instead of

<argument> \::error
                                 ! mykpg: explanation text

we should have (using \cs_neq_eq:cN { ~ ^^I } \or: or similar)

<argument \  
                        
! Package mypkg Error: explanation text

following the formatting of normal errors. In LuaTeX we could then simply use Lua code to raise the error expandably (there should be some code for that in l3trial/l3kernel-extras/).

[FrankMittelbach]

my latest last night attempt at ascii art:

<argument> \

  O>   arrg
 \| 
                              
mypkg: Some long explanation about 'arg1'

we could use that for all error if date = 10/31 :-)

---

More seriously I think the suggested approach is fine and we should make it simple as outlined there.

Using a non-printing active char to avoid the \ is just asking for trouble (and the <argument> \ isn't that bad). But I would drop the ! at the start of the line --- other LaTeX ...Error: messages don't have that either, so why make it look half like a Tex half like a LaTeX error?

[blefloch]

On 5/18/21 12:36 PM, Frank Mittelbach wrote: my latest last night attempt at ascii art: |<argument> \ O> arrg \| mypkg: Some long explanation about 'arg1' | we could use that for all error if date = 10/31 :-)

Or National Hummingbird Day (in the US), which is the first Saturday in September. This requires us to have resolved issue #547 (Date and time tools). https://www.checkiday.com/7e10a9aff26fe865aa2f21c165504cd9/national-hummingbird-day

-------------------------------------------------------------------------------- More seriously I think the suggested approach is fine and we should make it simple as outlined there. Using a non-printing active char to avoid the |\| is just asking for trouble (and the |<argument> \| isn't that bad). But I would drop the ! at the start of the line --- other LaTeX ...Error: messages don't have that either, so why make it look half like a Tex half like a LaTeX error?

Ok, I hadn't re-checked what other LaTeX errors do. Bruno

[FrankMittelbach]

Or National Hummingbird Day (in the US), which is the first Saturday in September. This requires us to have resolved issue #547 (Date and time tools). https://www.checkiday.com/7e10a9aff26fe865aa2f21c165504cd9/national-hummingbird-day

nice one ...we can see about that when #547 is resolved then :-)

Ok, I hadn't re-checked what other LaTeX errors do.

I forgot you do not get LaTeX errors with your documents :-) --- it is really just the ! that I'm sort of objecting too

[blefloch]

I'm having problems: things like \use:n { \ERROR a ^^J b } will correctly give a and b on different lines using the system's pdfTeX (because it is built with a suitable tcx character translation file), but in our test suite it gives a^^Jb on the same line.

I'm worried that this character translation may be subtly different in different engines. What to do?

[FrankMittelbach]

Isn't that because we have explicitly turned those off in the test suite to see the chars as ^^J etc?
I think this isn't something we have to worry about in reality, we use ^^J all over the place in 2e to denote a line breaks as in

\def\MessageBreak{^^J}%

so I think we can assume this works in real life

[Skillmon]

It doesn't work in TeX Lives XeLaTeX.

[FrankMittelbach]

it does with the option -8bit, but not sure what side effects that has (or whether that could or should be set as a default.

[blefloch]

My problem is not with ^^J in messages but in the context shown by TeX when producing an error. These don't seem to be treated quite the same way. Maybe in XeTeX we could use one of the unicode vertical space characters other than ^^J?

[FrankMittelbach]

My problem is not with ^^J in messages but in the context shown by TeX when producing an error. These don't seem to be treated quite the same way. Maybe in XeTeX we could use one of the unicode vertical space characters other than ^^J?

certainly something we could try. As I said there is an option call "8bit" if you use that then the ^^J gets interpreted correctly, but I guess a real unicode character will also work. It just seems that the hat-hat notation is turned off by XeTeX by default.

[blefloch]

Using a real unicode char doesn't work because it just displays in my terminal as a square with hex digits inside it.

This is in fact already present in the TeX engine (TeXlive's version): when showing the context of an error message, the \newlinechar seems to be taken into account for printable chars but not for some others. For instance, with \test{Z} the error shows the two X on different lines (and the message then also does), while with \test{^^0a} the error shows X^^JX while the message shows two X on different lines (correctly).

\def\use#1{#1}
\def\test#1{%
  \newlinechar=`#1\relax
  \message{\use{\ERROR X#1X}}
}
\test{Z}
\test{^^0a}

I think that this comes from the code around {temporarily disable new-line character} in the following procedure in tex.web.

procedure print(@!s:integer); {prints string |s|}
label exit;
var j:pool_pointer; {current character code position}
@!nl:integer; {new-line character to restore}
begin if s>=str_ptr then s:="???" {this can't happen}
@.???@>
else if s<256 then
  if s<0 then s:="???" {can't happen}
  else begin if selector>pseudo then
      begin print_char(s); return; {internal strings are not expanded}
      end;
    if (@<Character |s| is the current new-line character@>) then
      if selector<pseudo then
        begin print_ln; return;
        end;
    nl:=new_line_char; new_line_char:=-1;
      {temporarily disable new-line character}
    j:=str_start[s];
    while j<str_start[s+1] do
      begin print_char(so(str_pool[j])); incr(j);
      end;
    new_line_char:=nl; return;
    end;
j:=str_start[s];
while j<str_start[s+1] do
  begin print_char(so(str_pool[j])); incr(j);
  end;
exit:end;

[FrankMittelbach]

ok so that is the behavior in Knuth's TeX and XeTeX but not in pdftex or luatex. So by disabling new_line_char the print_char routine is displaying ^^J by default, whereas pdftex and luatex have some change that displays the ascii character (and you get that in xetex with the option -8bit (and strangely enough also with Knuth TeX :-))

So what now, ask for xetex to sync with pdftex and luatex in this regard? Tell users to use -8bit if they don't like the error displays?

[blefloch]

I've pushed a branch gh931-expandable-errors in which \msg_new:nnn { mypkg } { foo } { some~text } and \msg_expandable_error:nn { mypkg } { foo } produces the expandable error

! Extra \or.
<argument> \   
                Package mypkg Error: some text

In contrast the non-expandable errors are formatted as follows (not sure why we have ... there).

! Package mypkg Error: some text

Type <return> to continue.
 ...                                              

This works in all engines, without any funny characters, but the error is shifted right by a lot. We could ask on the TeXlive list (and MikTeX) whether they are open to making -8bit the default in XeTeX. Same problem in epTeX and eupTeX, by the way.

[FrankMittelbach]

Maybe a solution for XeTeX, could be done along those lines

\newlinechar=`\^^^^0370
\showthe\newlinechar
  
\catcode`\^^J=13
\def^^J{^^^^0370}


\def\use#1{#1}
\def\test#1{%
  \message{\use{\ERROR X#1Y}}
}

\typeout{Test:^^JTwo^^JLines}

\test{^^^^0370}

Maybe U+0370 is a slot taken by now (in my Unicode book it is one of the first unused ones) but then one could select some other unused unicode point.

[Skillmon]

Is using \or instead of an undefined control sequence a good idea? Do we have control that an expandable error message is never thrown inside of an \ifcase construct?

[FrankMittelbach]

I asked that too and @blefloch showed me how to ensure that: \iftrue\expandafter\fi\or

[jfbu]

I have read this with interest but as the total length of the error context does not seem extendible, I am not sure using ^^J makes the short expandable error message more legible to user.

Mentioning in passing that as alternative to Extra \or you are now experimenting with, or the previous Undefined control sequence way, which I had plagiarized over to my xint context, I started experimenting with this:

! Use of \xint/ doesn't match its definition.
<argument> \xint/  
                    Division by 0 of 1 (hit RET) 

(where \xint/ with catcode letter / was defined as \def\xint/.{} and used as \xint/<space>). Wondering if there is some other error message which could be used, but I did not see anything obvious after quick scan of pdftex web.

[blefloch]

On 5/19/21 6:49 PM, Skillmon wrote: Is using |\or| instead of an undefined control sequence a good idea? Do we have control that an expandable error message is never thrown inside of an |\ifcase| construct?

That's the purpose of doing \iftrue\expandafter\fi\or, namely TeX is inside an \iftrue construction (at the deepest level) when expanding \or.

[FrankMittelbach]

! Use of \xint/ doesn't match its definition.
\xint/
Division by 0 of 1 (hit RET)

nice idea too. I rather like the sound of the low-level error ... so one could have

Use of \LaTeX  doesn't match its definition.
<argument> \LaTeX   
                    hooks Error: Some explanation

Maybe better than Extra \or or Undefined .... On the other hand it is not LaTeX that is not matching but the user document ... so it not quite right either. Requires \LaTeX<space> to be misused here which probably too dangerous as such command names are normally used for the payload of robust commands. And if you change the above to \LaTeX! or \LaTeX/ then I think it gets less attractive.

A variation using the module or package name as @jfbu did, e.g.

! Use of \hooks/ doesn't match its definition.
<argument> \hooks/   
                   LaTeX Error: some text

Dunno, it is really pity to be so constraint by the number of available characters.

[jfbu]

Meanwhile I realized that the superior aspect of @blefloch Extra \or is how brief the TeX explanations are. When running say pdflatex with -interaction=nonstopmode the log file will contain the extra help with in "doesn't match its definition" looks:

If you say, e.g., `\def\a1{...}', then you must always
put `1' after `\a', since control sequence names are
made up of letters only. The macro here has not been
followed by the required stuff, so I'm ignoring it.

This is comparable to current situation with an undefined control sequence, and more verbose than the I'm ignoring this; it doesn't match any \if. of #934. An additional twist is that some LaTeX IDE may intercept error and display them, for example Emacs/AUCTeX will display something about picture commands being mis-used in the case of the "doesn't match its definition".

[blefloch]

Frank suggested a XeTeX solution making the newlinechar some high unused codepoint and making ^^J active and expanding to that new line character. That's quite clever, and it should be possible to make it work in eupTeX as well, but if I understand correctly epTeX only deals with bytes (0-255) like pdfTeX so that we don't have such an unused codepoint. Maybe I'm wrong?

@jfbu is right about why I preferred Extra \or to Use of \foo doesn't match its definition, but I think I was wrong because \foo can be changed to a control sequence of an arbitrary length, and newlines are correctly dealt with in all engines. This is not perfect because \foo must still be a predefined control sequence and cannot be defined on the fly. Still, we can make

! Use of \
! Package mypkg Error:
Something doesn't match its definition.
<argument> ...First piece of this explanation text 
                                                  and the rest of the story....

I see the following options.

1. Current Format expandable errors closer to other errors (see #931) #934, which uses Extra \or (for its short explanation text) but no newlines, with the error message being shifted towards the right. Total length of Package mypkg Error etc: 63 or 64 chars max.

2. Same but with some spaces replaced by ^^J or by a higher code point in XeTeX. Nicer output, same constraints, but seems to require epTeX to be called with the -8bit option.

3. Based on Use of ... doesn't match with control sequence (space)(space). Total length of Package mypkg Error etc: 62 or 63 chars max.

4. Variant with control sequence \Package mypkg Error gives the strange ! Use of \Package mypkg Error doesn't match its definition. but then the error message ends up at 65 or 66 chars max.

5. A new \msg_new_expandable:nnn { mypkg } { msgname } { arbitrarily~long~message~with~no~arguments } which predefines a control sequence. Then we can have

! Use of \

! LaTeX filehook Error: Filename fallback loop.
(filehook)              The file #1 (see below) was defined to fall
(filehook)              back to #2, which constitutes a loop when
(filehook)              trying to resolve the filename.

Something doesn't match its definition.
<argument> ...               #1:mypkg.sty,
                                                  #2:mypkg-fallback.sty

EDIT: changed the example message and formatting here.
Notice that the error text itself cannot replace #1 etc by their values. I also completed the sentence "doesn't match its definition" with something reasonable. The context is unavoidably split at a very specific point: 36 chars before and 29 after.

[FrankMittelbach]

Frank suggested a XeTeX solution making the newlinechar some high unused codepoint and making ^^J active and expanding to that new line character. That's quite clever, and it should be possible to make it work in eupTeX as well, but if I understand correctly epTeX only deals with bytes (0-255) like pdfTeX so that we don't have such an unused codepoint. Maybe I'm wrong?

There is one char we could use perhaps ,which is <delete> or ^^? because that has a catcode of 15 in all engines and formats (God knows why :-) ). So I can't really see a reason why we couldn't misuse that char as the newlinechar with ^^J expanding to it. That would then work everywhere including XeTeX. And I can't see this effecting any existing documents as right now the use of slot 127 in the input gives you an "invalid char" error. With that change it would give you a newline, but so there ...

Having said that, my preference goes for option 2, I think. It gives in my opinion the nicest output without much fuss and it is losing only 2 chars or so over the other solutions (except for 5., but I don't like it so much because that really makes things look totally different).

[blefloch]

@FrankMittelbach You're right about ^^? being perfect. To gain 2-3 more chars, we could even make the active ^^? be equal to \or, instead of making the (space)(space) control sequence equal to \or.

I've changed the formatting of option 5 in my previous message to make it look closer to normal messages. What do you think now?

[car222222]

@FrankMittelbach Don't look only at the (emailed) unedited version of comments.

Of course we cannot have so many !s in the expandable messages only.

[FrankMittelbach]

@jfbu you make a good point and I overlooked your \unexpanded so yes, that would be a case where XeTeX would then differ and would need to change the coding to 2 calls to \unexpanded with `^^J in the middle, which of course only works if you control the content. If you don't I don't think that getting nl chars in there from random input is likely, is it?

[jfbu]

@FrankMittelbach I looked at old respectable package hyperref.sty and I find this

    \immediate\write\@auxout{%
      \string\providecommand\string\HyperFirstAtBeginDocument{%
        \string\AtBeginDocument}^^J%
      \string\HyperFirstAtBeginDocument{%
        \string\ifx\string\hyper@anchor\string\@undefined^^J%
          \string\global\string\let\string\oldcontentsline\string\contentsline^^J%
          \string\gdef\string\contentsline%
            \string#1\string#2\string#3\string#4{%
            \string\oldcontentsline%
              {\string#1}{\string#2}{\string#3}}^^J%
          \string\global\string\let\string\oldnewlabel\string\newlabel^^J%
          \string\gdef\string\newlabel\string#1\string#2{%
             \string\newlabelxx{\string#1}\string#2}^^J%
          \string\gdef\string\newlabelxx%
             \string#1\string#2\string#3\string#4\string#5\string#6{%
             \string\oldnewlabel{\string#1}{{\string#2}{\string#3}}}^^J%
          \string\AtEndDocument{%
            \string\ifx\string\hyper@anchor\string\@undefined^^J%
              \string\let\string\contentsline\string\oldcontentsline^^J%
              \string\let\string\newlabel\string\oldnewlabel^^J%
            \string\fi%
          }^^J%
        \string\fi%
      }^^J%
      \string\global\string\let\string\hyper@last\relax^^J%
      \string\gdef\string\HyperFirstAtBeginDocument\string#1{\string#1}%
    }%

A more modern package might not have had the patience of all those \string and perhaps would have used a \detokenize wrapper, including the ^^J. But I have no concrete example.

EDIT: sorry the # mean the author would have used \string.... or not, as it could make the definition with # of catcode 12 and then use \detokenize wrapping all.

[blefloch]

For the exclamation mark: ok, I will have to add it, and a space, so we lose two characters, but I think it's more important to have exactly the right formatting.

Frank, have you managed to make your ^^? suggestion work? That character works fine in messages/typeout, so \catcode127=12\newlinechar=127\message{X^^?Y} puts X and Y on different lines, but then \def\use#1{#1}\use{\boom X^^?Y} shows the five characters X^^?Y on the same line, even with TeXlive's pdftex. With the -8bit option it works with all engines, but of course we could then simply use ^^J.

On making ^^J active (e.g. if we decide to use the higher unicode code-point solution in XeTeX and eupTeX), one worry is that many packages use explicit ^^J. I haven't found any case where it's not in an expansion context, so Frank's suggestion seems workable, but as jfbu says it is not quite safe. We should provide a way to get back the old behaviour, at least through latexrelease. I guess I need to make l3msg clever enough to deal with a change of \newlinechar if latexrelease goes back in time, probably through some explicit function \iow_gset_newlinechar:n that redefines \iow_newline: and the active ^^J (because it could be in some code already).

Maybe the two important questions are:

• How hard would it be to get -8bit to be the default in platex at least?
• How hard would it be to get an \expandableerrmessage primitive added to all engines?

[jfbu]

I have found the following "real-life" example

\begin{filecontents*}{\jobname-in.tex}
%<*tag>
bla bla
%</tag>
\end{filecontents*}

\documentclass{article}
% \catcode127 12
% \catcode10 \active
% \def^^J{^^?}
% \newlinechar127
\usepackage{catchfilebetweentags}
\begin{document}

\CatchFileBetweenTags{\foo}{\jobname-in.tex}{tag}

\show\foo
\end{document}

This will contain

> \foo=macro:
->bla bla.
l.17 \show\foo

(running with pdflatex -file-line-error -interaction=nonstopmode.

But if I uncomment the ^^J/^^? stuff I get this

> \foo=macro:
->
bla bla
.
l.17 \show\foo

showing something happened.

I grepped for *.sty containing both usage of ^^J, and of \detokenize (should I have done with \unexpanded) to get some manageable intersection list and it provides this in latex/ repertory of TeXLive

amsmath/amsmath.sty
asyfig/asyprocess.sty
asymptote/asymptote.sty
base/latexrelease.sty
bxpdfver/bxpdfver.sty
catchfilebetweentags/catchfilebetweentags.sty
catoptions/catoptions.sty
csquotes/csquotes.sty
eledmac/eledmac.sty
enumitem-zref/enumitem-zref.sty
footnotehyper/footnotehyper.sty
fvextra/fvextra.sty
glossaries-extra/glossaries-extra.sty
gmdoc/gmdoc.sty
gmutils/gmampulex.sty
gmutils/gmbase.sty
gmutils/gmcommand.sty
gmutils/gmtypos.sty
gmverb/gmverb.sty
gnuplottex/gnuplottex.sty
graphics/graphics.sty
graphicx-psmin/graphicx-psmin.sty
hardwrap/hardwrap.sty
koma-script/scrlayer-notecolumn.sty
loops/loops.sty
ltxkeys/ltxkeys.sty
ltxtools/ltxtools-base.sty
ltxtools/ltxtools-incluput.sty
ltxtools/ltxtools-trace.sty
mgltex/mgltex.sty
minted/minted.sty
minted/minted1.sty
moodle/moodle.sty
newtx/newtxmath.sty
phffullpagefigure/phffullpagefigure.sty
phfnote/phfnote.sty
proof-at-the-end/proof-at-the-end.sty
pstool/pstool.sty
pythontex/pythontex.sty
qstest/qstest.sty
reledmac/reledmac.sty
skeyval/skeyval-bc.sty
spreadtab/spreadtab.sty
svg/svg-extract.sty
tabu/tabu.sty
xwatermark/xwatermark.sty

In the case of catchfilebetweentags, it sets the \endlinechar to ^^J which is now active, and uses then \scantokens to retokenize however as far as I understand from looking fast, the catched contents in a toks register which is expanded via \the

      \scantokens\expandafter{%
         \expandafter\CatchFBT@tok\expandafter{\the\CatchFBT@tok}}%

so the file end of lines converted to active ^^J remain active rather than acting as newlinechar for scantokens proceedings and this explains why the are there in final result.

[jfbu]

I have found an alternative methods which allows to display

• messages up to 68 characters
• starting cleanly at start of line, separated by blank line from before and after
• with little TeX extraneous explanations ending up in log file
• message may contain linebreaks
• it works with both pdflatex and xelatex without -8bit option

\documentclass{article}
\begin{document}

\long\def\use#1{#1}

\def\boom#1\boom{}

\def\expandableerrormessage#1{%
   \expandafter\Clean\boom^^J#1^^J^^J\par
}%

% edited: removed \long, which was left-over from prior trials
\def\Clean#1\par{#1}

\def\userather#1#2#3{#1}

\def\divide#1#2{#1/\ifnum #2=0
    \expandableerrormessage{%
LaTeX3 error: \divide^^Jused with 0 as 2nd argument.^^JI will use 17299.%
}%
    \use{17299}\else #2\fi}%


\message{This is the result: \the\numexpr \divide{1000000000}{0}\relax}

\end{document}

$ xelatex expandableerror.tex 
This is XeTeX, Version 3.141592653-2.6-0.999993 (TeX Live 2021) (preloaded format=xelatex)
 restricted \write18 enabled.
entering extended mode
(./expandableerror.tex
LaTeX2e <2020-10-01> patch level 4
L3 programming layer <2021-05-11>
(/usr/local/texlive/2021/texmf-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/usr/local/texlive/2021/texmf-dist/tex/latex/base/size10.clo))
(/usr/local/texlive/2021/texmf-dist/tex/latex/l3backend/l3backend-xetex.def
(|extractbb --version)) (./expandableerror.aux)
(/usr/local/texlive/2021/texmf-dist/tex/latex/base/ts1cmr.fd)
Runaway argument?

LaTeX3 error: \divide 
used with 0 as 2nd argument.
I will use 17299.

! Paragraph ended before \boom was complete.
<to be read again> 
                   \par 
l.23 ...esult: \the\numexpr \divide{1000000000}{0}
                                                  \relax}
? 
This is the result: 57807 (./expandableerror.aux) )
No pages of output.
Transcript written on expandableerror.log.

The same with Inserting some tokens

$ xelatex expandableerror.tex 
This is XeTeX, Version 3.141592653-2.6-0.999993 (TeX Live 2021) (preloaded format=xelatex)
 restricted \write18 enabled.
entering extended mode
(./expandableerror.tex
LaTeX2e <2020-10-01> patch level 4
L3 programming layer <2021-05-11>
(/usr/local/texlive/2021/texmf-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/usr/local/texlive/2021/texmf-dist/tex/latex/base/size10.clo))
(/usr/local/texlive/2021/texmf-dist/tex/latex/l3backend/l3backend-xetex.def
(|extractbb --version)) (./expandableerror.aux)
(/usr/local/texlive/2021/texmf-dist/tex/latex/base/ts1cmr.fd)
Runaway argument?

LaTeX3 error: \divide 
used with 0 as 2nd argument.
I will use 17299.

! Paragraph ended before \boom was complete.
<to be read again> 
                   \par 
l.23 ...esult: \the\numexpr \divide{1000000000}{0}
                                                  \relax}
? I\userather{100}
This is the result: 10000000 (./expandableerror.aux) )
No pages of output.
Transcript written on expandableerror.log.

There may be a problem if one wants \par token to appear in the message, but the idea can perhaps be refined.
It has a limited message length, which goes a bit beyond what the other methods (5. being left aside) allow and one can use ^^J in there, but of course it has limited interest. Apart from allowing a two line messages perhaps.

V2:

\documentclass{article}
\begin{document}

\long\def\use#1{#1}

\expandafter\def
\csname/^^J%
expansion completed, due to an exception (see the above^^J%
explanation about the specifics). If in interactive^^J%
mode, simply hit RET and LaTeX3 will try to recover.^^J%
Hopefully, the above short explanation\endcsname
#1\boom{}

\def\expandableerrormessage#1{%
\def\expandableerrormessage##1##2{%
   \expandafter\expandafter\expandafter
   \Clean
   \expandafter
   #1\ErrorStart{##1}##2^^J^^J\par
}}\expandafter\expandableerrormessage
\csname/^^J%
expansion completed, due to an exception (see the above^^J%
explanation about the specifics). If in interactive^^J%
mode, simply hit RET and LaTeX3 will try to recover.^^J%
Hopefully, the above short explanation\endcsname

\def\Clean#1\par{#1}

\def\ErrorStart#1{! #1 error:^^J}%
 
\def\userather#1#2#3{#1}

\def\divide#1#2{#1/\ifnum #2=0
    \expandableerrormessage
    {l3foo}%
    {\divide used with 0 as 2nd argument. I will use 17299.}%
    \use{17299}\else #2\fi}%


\message{This is the result: \the\numexpr \divide{1000000000}{0}\relax}

\end{document}

Runaway argument?
! l3foo error:
\divide used with 0 as 2nd argument. I will use 17299.

./expandableerror.tex:40: Paragraph ended before \/
expansion completed, due to an exception (see the above
explanation about the specifics). If in interactive
mode, simply hit RET and LaTeX3 will try to recover.
Hopefully, the above short explanation was complete.
<to be read again> 
                   \par 
l.40 ...esult: \the\numexpr \divide{1000000000}{0}
                                                  \relax}
? 
This is the result: 57807 (./expandableerror.aux) )

Unfortunately with the -file-line-error we can not control the position of the backslash hence doing ascii art is not feasible. So I simply completed it with a /.

Starting the error with !<space> as above lets the Emacs/auctex error overview show something reasonable:

ERROR: l3foo error:

--- TeX said ---
\divide used with 0 as 2nd argument. I will use 17299.

./expandableerror.tex:40: Paragraph ended before \/
expansion completed, due to an exception (see the above
explanation about the specifics). If in interactive
mode, simply hit RET and LaTeX3 will try to recover.
Hopefully, the above short explanation was complete.
<to be read again> 
                   \par 
l.40 ...esult: \the\numexpr \divide{1000000000}{0}
                                                  \relax}
--- HELP ---
From the .log file...

I suspect you've forgotten a `}', causing me to apply this
control sequence to too much text. How can we recover?
My plan is to forget the whole thing and hope for the best.

[blefloch]

@jfbu's V2 proposal is really great, so I'll implement that in a few days (I need my full focus not to mess up all of the test file updates). The only drawback is that we have to fill in the dots in one of the sentences (depending on whether we use \par or other things to end the scanning):

• «Paragraph ended before \… was complete.»
• «File ended while scanning use of \….» (or "text of" if we use \unexpanded)
• «Forbidden control sequence found while scanning use of \….» (or "text of" if we use \unexpanded)

[FrankMittelbach]

before you do that @blefloch, can we first check how well that works with editors that evaluate error messages? I also think @jbu's idea is a neat, but my fear is that scanning editors will pick on the wrong part and never present the right information and if that's the case, then it may not be so good. Sorry for the late reply but it got my first shot today and not feeling so well (which is perhaps a good sign as people claim)

[Skillmon]

The \par limitation is no problem. You can simply \detokenize the second argument to make this always not contain a \par (or \boom for that matter).

[car222222]

@FrankMittelbach Yes, reaction to jab is good! Can you send some jabs here for us? :-) -- There are almost none available here, not even for big money!

[car222222]

Re @blefloch 's ... (fill in the dots) question.

Several types of possibility:

• something short, probably meaningless, maybe invisible,
  but not typable;
• something meaningful but clearly a 'dummy name';
• something that helps to clarify the overall
  message (in all cases).

Or maybe we can magic them away?

[Skillmon]

I think something like in @jfbu's code is quite a good idea. Some general help text explaining that this is an expandable error that has limitations by nature. And that the user is expected to just hit <return> for recovery. For one of my packages I'll go for:

! Paragraph ended before \<some-command> completed
due to above exception.   The number of characters
in that error message is limited, so it is concise
yet hopefully comprehensible.  Else take a look in
the package documentation.
If you're in interactive mode hit <return> now and
I'll try to recover.
I hope the above short explanation was complete.

[jfbu]

somewhat imitating Skillmon's formulation I have the following local version for geeks:

./temptest.tex:35: Paragraph ended before \romannumeral (or
\expanded, \numexpr, ...) triggered expansion could terminate.
See the above short explanation about the exception specifics.
xint will now try to recover (hit <return> at the ? prompt if in
interactive mode) and will go ahead hoping repair was complete.
<to be read again> 
                   \par 
l.35 \xinteval{1/0}

Regarding the actual error I am keeping it on one line

Runaway argument?
! xint error: Division by 0 of 1

because Emacs/AucTeX then displays in its error buffer

ERROR: xint error: Division by 0 of 1

--- TeX said ---
./temptest.tex:35: Paragraph ended before \romannumeral (or
\expanded, \numexpr, ...) triggered expansion could terminate.
See the above short explanation about the exception specifics.
xint will now try to recover (hit <return> at the ? prompt if in
interactive mode) and will go ahead hoping repair was complete.
<to be read again> 
                   \par 
l.35 \xinteval{1/0}
                   
--- HELP ---
From the .log file...

I suspect you've forgotten a `}', causing me to apply this
control sequence to too much text. How can we recover?
My plan is to forget the whole thing and hope for the best.

and its "ERROR" line seems to have to be that: one line only (the next one ends up below ---TeX said---)

I don't have to maintain style guidelines in my case.

[blefloch]

Compiling the attached TeX file gives 9 errors, corresponding to different ways of producing an error stating

! LaTeX Error: Something's wrong--perhaps a missing \item.

The first three are non-expandable versions (LaTeX2e, LaTeX3 with no help text, LaTeX3 with help text). The next three are recognized in my emacs/auctex tests as some other error (Extra \or, Undefined control sequence, some issue with picture commands). The last three look good for me, with emacs/auctex recognizing the well-known "missing \item" error and telling me about lists.

Could some of compile the attached TeX file in your editor of choice and let me know which of the errors look good (including help text etc)? In emacs, 1-3 are nice, 4-6 are pretty bad, 7-9 are ok.

1. LaTeX2e
2. LaTeX3 short
3. LaTeX3 long
4. Extra or
5. Undefined cs
6. Use doesn't match def
7. Paragraph ended before
8. File ended while scanning
9. Forbidden control sequence

test-expandable-errors.tex.txt
(This file may need renaming to satisfy editors.)

[jfbu]

I use emacs/auctex as well so can only confirm your findings with it. Here is error overview provided by auctex:

 test-expandable-errors.tex     8 Error   LaTeX Error: Something's wrong--perhaps a missing \item.
 test-expandable-errors.tex    16 Error   LaTeX  Error: Something's wrong--perhaps a missing \item.
 test-expandable-errors.tex    26 Error   LaTeX  Error: Something's wrong--perhaps a missing \item.
 test-expandable-errors.tex    44 Error   Extra \or.
 test-expandable-errors.tex    49 Error   Undefined control sequence.
 test-expandable-errors.tex    65 Error   Use of \?? doesn't match its definition.
 test-expandable-errors.tex    86 Error   LaTeX Error: Something's wrong--perhaps a missing \item.
 test-expandable-errors.tex   116 Error   LaTeX Error: Something's wrong--perhaps a missing \item.
 test-expandable-errors.tex   140 Error   LaTeX Error: Something's wrong--perhaps a missing \item.

(is there some reason for the two spaces between LaTeX and Error in cases 2 and 3 which use the current non-expandable LaTeX3 \msg_error:nn?)

I tested also with an old TeXShop (2017 or so): it opens up a window during compilation with console output where one can interact as if on command line, plus a button to jump back to source where error is located. So in this interface the "perhaps a missing \item" does not trigger some information about lists being displayed as error help, one only gets the raw extra help from entering h<return> (as if executing from command line). But this is old TeXShop.

In definition of \Cerror:n for para-ended case is there some reason for \use:n { #1 ##1 \par } and not directly #1 ##1 \par (and only one expandafter)? (I suspect this is left-over from when you constructed test file by copying pasting template from one to next case)

In final version, will inserting tokens via I<tokens> be possible or will they be discarded as in the test file?

[Skillmon]

In every expandable error solution VIM-LaTeX (aka latex-suite) will not show the missing item error but the underlying mechanism to throw the error. (I never compile with it though, so that doesn't bother me)

The quickfix-window contents (the lines not started with || are the errors VIM-LaTeX detected):

|| This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021) (preloaded format=pdflatex)
||  restricted \write18 enabled.
|| entering extended mode
|| (./test-expandable-errors.tex
|| LaTeX2e <2020-10-01> patch level 4
|| L3 programming layer <2021-05-11>
|| 
|| ========== @noitemerr ==================================================
|| 
|| 
test-expandable-errors.tex|8| LaTeX Error: Something's wrong--perhaps a missi
|| ng \item.
|| 
|| See the LaTeX manual or LaTeX Companion for explanation.
|| Type  H <return>  for immediate help.
||  ...                                              
||                                                   
|| l.8 }
||      
|| 
|| ========== short-error =================================================
|| 
|| 
test-expandable-errors.tex|16| LaTeX  Error: Something's wrong--perhaps a mis
|| sing \item.
|| 
|| Type <return> to continue.
||  ...                                              
||                                                   
|| l.16 }
||       
|| 
|| ========== error-with-help =============================================
|| 
|| 
test-expandable-errors.tex|26| LaTeX  Error: Something's wrong--perhaps a mis
|| sing \item.
|| 
|| For immediate help type H <return>.
||  ...                                              
||                                                   
|| l.26 }
||       
|| 
|| ========== extra-or-error ==============================================
|| 
test-expandable-errors.tex|44| Extra \or.
|| <argument> \   
||                ! LaTeX Error: Something's wrong--perhaps a missing \item.
|| l.44 }
||       
|| 
|| ========== undefined-error =============================================
|| 
test-expandable-errors.tex|49| Undefined control sequence.
|| <argument> \   
||                ! LaTeX Error: Something's wrong--perhaps a missing \item.
|| l.49 }
||       
|| 
|| ========== use-doesnt-match ============================================
|| 
test-expandable-errors.tex|65| Use of \?? doesn't match its definition.
|| <argument> \??  
||                 ! LaTeX Error: Something's wrong--perhaps a missing \item.
|| l.65 }
||       
|| 
|| ========== para-ended ==================================================
|| 
|| Runaway argument?
|| 
|| ! LaTeX Error: Something's wrong--perhaps a missing \item.
|| 
test-expandable-errors.tex|86| Paragraph ended before \ after
|| some expandable code finished evaluating (see error message above).
|| 
|| See the LaTeX manual or LaTeX Companion for explanation.
|| Type <return> to continue.
||  ...
|| Error before the document was complete.
|| <to be read again> 
||                    \par 
|| l.86 }
||       
|| 
|| ========== file-ended ==================================================
|| 
|| Runaway argument?
|| 
|| ! LaTeX Error: Something's wrong--perhaps a missing \item.
|| 
test-expandable-errors.tex|116| File ended while scanning use of \
|| an expandable error message given above.
|| 
|| See the LaTeX manual or LaTeX Companion for explanation.
|| Type <return> to continue.
||  ...
|| <inserted text> 
||                 \par 
|| l.116 }
||        
|| 
|| ========== outer-def ===================================================
|| 
|| Runaway argument?
|| 
|| ! LaTeX Error: Something's wrong--perhaps a missing \item.
|| 
test-expandable-errors.tex|140| Forbidden control sequence found while scanni
|| ng use of \
|| the expandable error message given above.
|| 
|| See the LaTeX manual or LaTeX Companion for explanation.
|| Type <return> to continue.
||  ...
|| <inserted text> 
||                 \par 
|| l.140 }
||        
||  )
|| (see the transcript file for additional information)
|| No pages of output.
|| Transcript written on test-expandable-errors.log.

[Skillmon]

@jfbu I don't know how useful I<tokens> is here. In general the macros using the expandable error mechanism are at a level where the average user has no clue how to fix this with a single token injection, and the error recovery should be handled by the code itself. The only sensible user interaction is either to cancel the TeX run, or to hit <return>. Even worse, what if the user decides to do gobble following tokens with 1-9? This could make the recovery trip even more.

[jfbu]

@Skillmon agreed. Only mentioning this because current code allows it and even went to some lengths using safe delimiter to implement it. This being said TeXShop strangely reminds the user of the I<tokens> but not that H gives possibly further help.

[blefloch]

In retrospect I agree supporting I is not useful in these expandable error messages. Occasionally I've used it to check up on what was happening, but that only works when you know exactly what's going on in the code.

@jfbu The double space in LaTeX error has been corrected a few days ago in the main branch, maybe on CTAN this time or at the next upload, it doesn't matter much.

Here are some more tests. I'll get back with some more later.

• No parsing.
  • TeXworks: stops at every error, showing the log as is with no parsing as far as I can tell.
  • Gummi: I couldn't get it to do anything else than fatally stop at an error, no attempt at parsing the log file.
• emacs-like
  • emacs/auctex: 7-9 show up super nicely with appropriate explanations about lists, 4-6 show up as Extra \or and similar, with the ! LaTeX Error: as part of the help text.
  • TeXstudio, TeXmaker, Kile: recognize errors exactly like emacs (so 1-3 and 7-9 are ok), but highlight in red the sentences ! Paragraph ended ... or similar in versions 7-9, and no helpful explanation about lists.
  • LaTeXila: 7-9 are displayed roughly as Something is wrong, perhaps..., while 4-6 are displayed as Extra \or and similar.
• vim-like
  • Vim-latex (latex-suite) per Skillmon's description above: detects the lines starting with the filename with -file-line-error, which means that they don't see the ! LaTeX Error lines in the expandable errors as errors. Thus, 4-6 are slightly better since the explanation text is after the actual TeX error so presumably jumping in the log to the given error the user would see what's below the error more than what's above.
  • Winefish is similar to Vim but detects ! LaTeX Error lines as separate errors, but with no line number nor file. In options 4-6 they show up after the TeX error in the error list, which means going through errors one by one at least goes at the right place in the file. In options 7-9, the ! LaTeX Error lines show up before other errors so one doesn't get to the right place in the file until clicking on the next error.
• Overleaf: mostly awful display of these errors: 1 has nice explanation, 2-3 don't but that's probably because of an extra space in LaTeX Error, 4 shows Extra \or and the error we want, 5 shows "helpful" text about undefined cs then our error, 6 shows Use of ... then our error; 7 shows two errors: Runaway arg... LaTeX Error: Something is wrong and Paragraph ended before ..., 8-9 don't show our message, only show File ended and Forbidden cs. Overall, 4,6,7 are mostly reasonable but the others are not.

[Rmano]

I use vimtex and this is what I have:

test-expandable-errors.tex|8 error| LaTeX Error: Something's wrong--perhaps a missing \item.
test-expandable-errors.tex|16 error| LaTeX  Error: Something's wrong--perhaps a missing \item.
test-expandable-errors.tex|26 error| LaTeX  Error: Something's wrong--perhaps a missing \item.
test-expandable-errors.tex|44 error| Extra \or. \   
test-expandable-errors.tex|49 error| Undefined control sequence. \   
test-expandable-errors.tex|65 error| Use of \?? doesn't match its definition. \??  
test-expandable-errors.tex|| Something's wrong--perhaps a missing \item.
test-expandable-errors.tex|86 error| Paragraph ended before \ after
test-expandable-errors.tex|| Something's wrong--perhaps a missing \item.
test-expandable-errors.tex|116 error| File ended while scanning use of \
test-expandable-errors.tex|| Something's wrong--perhaps a missing \item.
test-expandable-errors.tex|140 error| Forbidden control sequence found while scanning use of \

I hope this is somewhat useful --- I was getting pinged on chat but I have no time now to look at it more in depth --- just loaded the file and tried compiling.

[FrankMittelbach]

@blefloch in case you have a new free cycle to do a little TeX work at some point, could you perhaps implement a new expandable error message that fits into the 2e world? The decision in the team meeting was to use \??? not matching it's definition a while ago.