\importmodule does not tolerate _ in module names

[kohlhase]

\importmodule{test_test} gives me the error.

Error:unexpected:_ Script _ can only appear in math mode
    /Users/kohlhase/localmh/MathHub/MiKoMH/GenCS/source/dmath/en/test.tex#textrange(from=2;0,to=2;26)
    In Core::Definition::Primitive[Subscript] from TeX.pool.ltxml line 3488
     <= Core::Stomach[@0x7fac4db5c5b8] <= Core::Gullet[@0x7fac4a8505e0] <= Core::Stomach[@0x7fac4db5c5b8] <= ...

Just run sTeX on

\begin{module}[id=test]
  \importmodule{test_test}
\end{module}

I am not sure where this comes from, I had suspected that we have to give the argument of \importmodule the type Semivarbatim or even TeXFileName, but that did not really help.

[kohlhase]

@dginev, could you have a look at this?

[kohlhase]

this is important (priority) to me, since we need if of the OpenDreamKit flexiformalization of elliptic curves.

[dginev]

Can you easily use underscores in the TeX implementation? I was left with the impression they remain active unless explicitly neutralized. Active underscores are easily one of the most common problems in Authorea as well btw.

[kohlhase]

There does not seem to be a problem in sTeX here.

[kohlhase]

But the only thing that is really happening is that the arguments are passed around to an \input eventually, and (in a way that I do not really understand) into macro names.

[angerhang]

https://github.com/KWARC/LaTeXML/blob/master/lib/LaTeXML/Package/TeX.pool.ltxml#L3404-L3407
From this block, we can see that LaTeXML doesn't allow us to have ^ or _ in a non-math block for the argument.

I can think of two possible workarounds:

I. a sketchy approach: explicitly silencing this error for sTeX, which however, might introduce other errors and make KWARC's LaTeXML deviate from Bruce's branch.

II. a might work approach: without modifying any LaTeXML code, instead of passing the module name by argument, we can pass it by keyval e.g. \importmodule[modules = test_test] as LaTeXML doesn't check the scripts, _, ^ when they are passed.

Can @dginev and @kohlhase comment on this?

[dginev]

Right, you are pointing to the exact bit of code that triggers the error, when the underscore is expanded outside of math mode.

So the idea with the Semiverbatim parameter type is exactly to avoid that. It is a different reader than the Plain parameter one, which will slurp in characters a lot more leniently, and it would never treat the underscore as a primitive.

https://github.com/KWARC/LaTeXML/blob/master/lib/LaTeXML/Package/TeX.pool.ltxml#L237

If you could first change the plain argument {} to Semiverbatim in the DefConstructor here:
https://github.com/KWARC/sTeX/blob/master/sty/modules/modules.sty.ltxml#L123

and try using an underscore name, I would be quite curious to read more about what the error is and how it's triggered. It should "just work". I am equally surprised Michael has made this work in TeX so easily, since the catcode must have been neutralized there as well.

[dginev]

Btw, @kohlhase just a reminder that if you switch the \importmodule argument to Semiverbatim, you will no longer be able to expand macros in it, and I remember there used to be importmodule arguments that had macros (e.g. for instantiating the right relative path).

It may be better to use the dedicated TeXFileName parameter type here, but I would ask @brucemiller if that fits the intended use of that reader.

[angerhang]

Right,

DefConstructor('\importmodule OptionalKeyVals:importmodule Semiverbatim',

works on on

  \importmodule{test_test}

with no problem.

[dginev]

Perfect, then there is probably 0 dev work to do here. I am still wondering if using TeXFileName isn't better than Semiverbatim in this case.

[dginev]

Btw, @angerhang here is the intro-level documentation of binding "prototypes" where semiverbatim is briefly introduced: http://math.nist.gov/~BMiller/LaTeXML/manual/modules/LaTeXML_Package.html#SSx2.SSSx1.Px1

TeXFileName is not present in these docs, so @brucemiller could tell us if that is an intended or unintended omission :> But it's in TeX.pool so we can certainly use it.

[angerhang]

Yap thanks for the tip : )

but I don't know why Miko said neither Semivarbatim nor TeXFileName helped for him. Perhaps @kohlhase can also elaborate on this.

[kohlhase]

I probably had some problems testing, I was wondering myself. Did you commit?

[angerhang]

I haven't because Deyan and I were not sure if Semivarbatim or TeXFileName is better.

[kohlhase]

I am not sure either, so let's just start with Semiverbatim and then escalate to TeXFileName when necessary.

[kohlhase]

Note that we also need to do the same for \usemodule.

[brucemiller]

TeXFileName is probably not what you want: it reads the filename the TeX way: w/o braces, up to a space, cr or comment and doesn't accept macros. Semiverbatim is more likely what you want: it neutralizes ^,_,@,~,&,$,' and accepts control sequences and (depending on the caller) does accept expand them.

Note that up until a recent patch, % was also accepted, but that turns out to have been a mistake. (there are ways to get % accepted to, if you need).

[kohlhase]

Thanks to Bruce for the clarification. I had suspected something like this (you might still update the LaTeXML documentation though). And we definitely do not want TeXFileName it seems like.

[dginev]

All clear, thanks for the details!

[kohlhase]

@angerhang I have tested this with the modules at https://gl.mathhub.info/ODK/lmfdb/. And I am still getting script warnings. Could you please try as well? Just install them via lmh and that should be sufficient.

[angerhang]

I tried to install the modules by lmh install ODK but I get parsing failure:

lmh install ODK
'NoneType' object has no attribute 'find'
Parsing failure (make sure gl::projects_url is correct)

[kohlhase]

this is a known issue of lmh. Just use git clone directly. BUT I have renamed all files away from _ (to make progress), so you will have to make your own test cases I fear.

[kohlhase]

This issue was moved to slatex/LaTeXML-Plugin-sTeX#16