Add environment/macros to simplify testing fragile commands

[benzea]

I am working on a LaTeX 3 package (https://github.com/sdaps/sdaps-class) and would like to do regression testing using the proper LaTeX infrastructure. However, a number of commands I would like to test are fragile which means I cannot test certain corner cases inside the TEST macro definition.

It would be nice to have a TEST environment or e.g. BEGINTEST(/ENDTEST) macros to simplify testing fragile commands.

[josephwright]

The LaTeX3 approach is that everything should be fully-expandable or \protected, so 'm not sure how you are ending up with fragile commands. Can you give an example of what you are doing?

BTW, classes are a bit of an oddity at the moment: they are not 'really' right.

[josephwright]

BTW, I've literally just moved the test system to it's own repo: https://github.com/latex3/l3build

[benzea]

The main reason is so that fragile content works (e.g. using \verbatim inside different environments). This is something where I actually have users who currently use savebox and similar tricks to be able to create the contents they want and use it later. In particular I have:

• One environment which can duplicate its contents (like a chain letter)
• Multiple environments that users want to insert e.g. images and code listings into

A secondary reason is that I created my own "tabular" like environment which basically aligns multiple tables to each other (and allows transposing rows and columns, the code is somewhat nuts to be honest). In that case it is just about added convenience to allow the user to use & and \ as separators.

[benzea]

See http://sdaps.org/class-doc/sdapsarray.html for the extreme example (with an example of nested content in the last example of the page).

[benzea]

I think a set of macros/environments like the following would be useful for me:

https://gist.github.com/benzea/bd9aa91f827943f886ec51a2d2520c28

Then I could have multiple tests per file (one on each page) and only the page number should change if a new test is added to a file with multiple layout regression tests.

But then, maybe I am doing everything wrong ;)

[josephwright]

Oh, forgot to mention earlier that l3build is used for the LaTeX2e kernel tests, and they do of course include lots of fragile commands :)

I'll take a look over your links shortly (yes, I know my comment is out-of-logical-order as a result).

[josephwright]

I'll see what the rest of the team think: for the 2e tests we mainly just do more or less

\begin{document}
\showoutput
<various fragile things>
\end{document}

at least in part as the approach there pre-dates some of the structures in l3build.

[benzea]

Yep, that works quite well.

But I don't think it scales nicely if one tries to put multiple tests into a single file. In that case ideally the diff of the .tlg should make it obvious that none of the other tests were affected when a single one is updated or added.
For this each test would need to be self contained (i.e. exercise the page builder afterwards).

[blefloch]

Since the build system can be (and is) used for plain TeX packages, we're not keen on using \begin{...}...\end{...} environments. However, what about

• \BEGINTEST...\ENDTEST doing your MYTEST environment
• \BEGINPAGETEST...\ENDPAGETEST doing your MYTESTPAGESHOW environment

(Actually I'm not too sure about the pagetest thing, why not just include \showoutput and \vfil\break in the test file?)

\logpos and \lognamedpos seem like a different question.

I haven't thought enough about OMITPAGE.

[benzea]

Ignore the \logpos stuff. It might be useful, but really this is a special case as I actually care about \pdfsavepos giving consistent results in these tests.

How about:

• Add \BEGINTEST...\ENDTEST (which does \begingroup/\endgroup)
• Use \showoutput and \vfil\break in an example documenting the commands

OMITPAGE had two reasons, but I think it is pointless with the above suggestion:

• A \begin{MYTESTPAGE} needed to be at the start of a page, so anything earlier may need to flush out the page (including e.g. code to load fonts).
• API consistency

I have been playing with it and the environments work well for me and could be easily implemented with the macros as suggested by @blefloch (see here for a commit).

[josephwright]

I've set up an issue in the new l3build repo: latex3/l3build#15. I'll close here and we can continue there :)