Adjustments to \verb and +v (ltcmd) #1087

josephwright · 2023-06-14T18:42:55Z

Internal housekeeping

Status of pull request

Feedback wanted
Under development
Ready to merge

Checklist of required changes before merge will be approved

Test file(s) added
Version and date string updated in changed source files
Relevant \changes entries in source included
Relevant changes.txt updated
ltnewsX.tex (and/or latexchanges.tex) updated

Includes a temporary fix to l3doc that will need to be moved back to expl3 in due course.

FrankMittelbach

I'm not sure that it is good to provide @vobeytabs. It might complicate the situation in the field --- needs some thought

base/doc/ltnews38.tex

FrankMittelbach · 2023-06-14T19:22:08Z

base/ltmiscen.dtx

+{\catcode`\^^I=\active
+\gdef\@vobeytabs{\catcode`\^^I\active\let^^I\@xobeytab}}


are we sure that it is better to have a separate command or to stick the TAB handling into @vobeyspaces? I'm just wondering, but there are several packages out there that mimic verbatim setup and maybe they would not have to change and nevertheless pick up the ^Î handling this way. Just as thought ... not thought through

I can see this both ways. I agree that if you want to say 'basically tabs are spaces' then yes, it makes sense not to have a separate macro. But the moment you handle tabs there's the possibility of diverging from space handling, and it was that I was thinking of: in particular, people could allow a tab to convert to multiple spaces in verb material.

It could still be moved (by default) into @vobeyspaces (using your @vobeytabs macro) this way probably avoid all the redefs and still gains the flexibility.

Ah, you mean include \@vobeytabs in the def. for \@vobeyspaces?

I didn't mean this originally, but yes, your argument about being able to adjust it is valid, so I now thought moving it in there gives you the best of both worlds

See latex3/latex3#756.

[ci skip]

FrankMittelbach · 2023-06-15T06:13:32Z

base/ltmiscen.dtx

+%<latexrelease>\IncludeInRelease{2023/11/01}%
+%<latexrelease>                 {\verbatim}{Obeyed tabs}%
+%<*2ekernel|latexrelease>
+\def\verbatim{\@verbatim \frenchspacing\@vobeyspaces\@vobeytabs \@xverbatim}
+%</2ekernel|latexrelease>
+%<latexrelease>\EndIncludeInRelease
+%<latexrelease>\IncludeInRelease{0000/00/00}%
+%<latexrelease>                 {\verbatim}{Obeyed tabs}%
+%<latexrelease>\def\verbatim{\@verbatim \frenchspacing\@vobeyspaces \@xverbatim}
+%<latexrelease>\EndIncludeInRelease
+%<*2ekernel>


if tabs are handled inside @vobeyspaces then this hsouldn't be needed, should it?

I'd wondered about that. However, that leaves you vulnerable - to a re-definition of \@vobeyspaces, as that would also kill tab support. That might of course be the position we want to take. So I've backed out my change and will look again at this - there's also something not quite right and I want to understand that too.

the way I see it is this: we interpret \@vobeyspaces as "handle all space like chars (space TAB, maybe others one day) for verbatim" and \@vobeytabs as an indirection to do something special to tabs alone on top of it. To my knowledge there is no redefinition of \vobeyspaces right now and if there is one in the future it has to take care of the assumption.

"all space-like chars"!
This could be a large number, and it can be somewhat controversial which ones are "truly space-like".
It is not even clear that TAB is always and everywhere considered space-like.

See if you like the new version

I am not sure how to find the latest version.

FrankMittelbach · 2023-06-15T06:14:13Z

base/ltmiscen.dtx

 %     {Provide visible space in \cs{verb*} also for XeTeX and LuaTeX (github/69)}
+% \changes{v1.2b}{2023/06/14}
+%     {Support visible tabs in \cs{verb*}}
 %    \begin{macrocode}
 \@namedef{verbatim*}{\@verbatim
  \@setupverbvisiblespace
-  \frenchspacing\@vobeyspaces\@sxverbatim}
+  \@setupverbvisibletab
+  \frenchspacing\@vobeyspaces\@vobeytabs\@sxverbatim}
 \expandafter\let\csname endverbatim*\endcsname =\endverbatim


FrankMittelbach · 2023-06-15T06:14:59Z

base/ltmiscen.dtx

+%<latexrelease>\IncludeInRelease{2018/12/01}%
+%<latexrelease>                 {\verbatim*}{Setup visible space for \verb}%
+%<latexrelease>\@namedef{verbatim*}{\@verbatim
+%<latexrelease>  \@setupverbvisiblespace
+%<latexrelease>  \frenchspacing\@vobeyspaces\@sxverbatim}
+%<latexrelease>\EndIncludeInRelease


FrankMittelbach · 2023-06-15T06:16:23Z

base/ltmiscen.dtx

  \ifnum\catcode`\ =\active
  \else  \@setupverbvisiblespace \@vobeyspaces \fi
+  \ifnum\catcode`\^^I=\active
+  \else  \@setupverbvisibletab \@vobeytabs \fi


however in that case I'm not sure this logic is still correct.

Indeed: take a look at the modified one

Where is this?

This reverts commit 61049bf.

car222222 · 2023-06-16T09:18:43Z

There seem to be a few comments etc that are still pending.

car222222

A typo and some replies/comments.

car222222 · 2023-06-15T07:35:24Z

base/doc/usrguide.tex

@@ -794,6 +794,42 @@ \subsection{Commands at the start of tabular cells}
 \end{tabular}
 \end{verbatim}

+\subsection{Using the verbatim argument types}
+
+As described about, the \texttt{v}-type argument may be viewed as similar to


Maybe this already got changed? Presumably: about ==> above ??

Are you sure: the diff suggests that it now reads: aboveß ??

car222222 · 2023-06-15T07:48:52Z

base/ltmiscen.dtx

+%<latexrelease>\IncludeInRelease{2023/11/01}%
+%<latexrelease>                 {\verbatim}{Obeyed tabs}%
+%<*2ekernel|latexrelease>
+\def\verbatim{\@verbatim \frenchspacing\@vobeyspaces\@vobeytabs \@xverbatim}
+%</2ekernel|latexrelease>
+%<latexrelease>\EndIncludeInRelease
+%<latexrelease>\IncludeInRelease{0000/00/00}%
+%<latexrelease>                 {\verbatim}{Obeyed tabs}%
+%<latexrelease>\def\verbatim{\@verbatim \frenchspacing\@vobeyspaces \@xverbatim}
+%<latexrelease>\EndIncludeInRelease
+%<*2ekernel>


"all space-like chars"!
This could be a large number, and it can be somewhat controversial which ones are "truly space-like".
It is not even clear that TAB is always and everywhere considered space-like.

car222222 · 2023-06-16T09:15:17Z

base/ltmiscen.dtx

+%<latexrelease>\IncludeInRelease{2023/11/01}%
+%<latexrelease>                 {\verbatim}{Obeyed tabs}%
+%<*2ekernel|latexrelease>
+\def\verbatim{\@verbatim \frenchspacing\@vobeyspaces\@vobeytabs \@xverbatim}
+%</2ekernel|latexrelease>
+%<latexrelease>\EndIncludeInRelease
+%<latexrelease>\IncludeInRelease{0000/00/00}%
+%<latexrelease>                 {\verbatim}{Obeyed tabs}%
+%<latexrelease>\def\verbatim{\@verbatim \frenchspacing\@vobeyspaces \@xverbatim}
+%<latexrelease>\EndIncludeInRelease
+%<*2ekernel>


I am not sure how to find the latest version.

car222222 · 2023-06-16T09:17:34Z

base/ltmiscen.dtx

  \ifnum\catcode`\ =\active
  \else  \@setupverbvisiblespace \@vobeyspaces \fi
+  \ifnum\catcode`\^^I=\active
+  \else  \@setupverbvisibletab \@vobeytabs \fi


Where is this?

car222222 · 2023-06-17T09:54:03Z

When the code is finalised I shall review the text in usrguide:
I cannot follow some parts of the current version.

josephwright · 2023-06-18T08:40:28Z

When the code is finalised I shall review the text in usrguide: I cannot follow some parts of the current version.

Removed: Frank suggested activating tabs was done as part of activating spaces, so it's all in one place.

car222222 · 2023-06-18T09:21:08Z

What was "Removed:"?

I assume that it was not all the text in usrguide about verbatim and friends!

josephwright · 2023-06-18T21:23:31Z

What was "Removed:"?

I assume that it was not all the text in usrguide about verbatim and friends!

I meant the lines you'd been talking about: Frank had various suggestions which I addressed.

car222222 · 2023-06-19T04:39:49Z

Aha! But I am now, at least, referring to the whole of the entry in usrguide.

josephwright · 2023-06-19T05:04:19Z

Aha! But I am now, at least, referring to the whole of the entry in usrguide.

That's all still there :)

mrpiggi · 2023-06-20T09:30:39Z

base/doc/usrguide.tex

@@ -796,7 +796,7 @@ \subsection{Commands at the start of tabular cells}

 \subsection{Using the verbatim argument types}

-As described about, the \texttt{v}-type argument may be viewed as similar to
+As described aboveß, the \texttt{v}-type argument may be viewed as similar to


that's a typo as well ;)

car222222 · 2023-07-03T09:33:08Z

Is all this code now finalised? So that I can attempt to sort out the documentation.

josephwright · 2023-07-03T19:33:09Z

Is all this code now finalised? So that I can attempt to sort out the documentation.

Yes no changes planned: awaiting thoughts from review

tobiasBora · 2023-09-01T08:14:53Z

@josephwright @FrankMittelbach Ok now it is clearer, thanks a lot. I’m curious though, how would you make the difference between \par (the token) and \par (the string), for instance in the string replacement command?

josephwright · 2023-09-01T08:21:43Z

@josephwright @FrankMittelbach Ok now it is clearer, thanks a lot. I’m curious though, how would you make the difference between \par (the token) and \par (the string), for instance in the string replacement command?

(Aside: TeX strings are not like strings in other languages: you mean a token list.)

Using expl3, is we do something like

\tl_set:Nn \l_tmpa_tl {#1} % The grabbed tokens
\tl_replace_all:Nnn \l_tmpa_tl { \par } { [PAR] }

we would replace the \par tokens as TeX will tokenize \par as normal. However, if we switch to

\tl_set:Nn \l_tmpa_tl {#1} % The grabbed tokens
\tl_replace_all:Nxn \l_tmpa_tl { \token_to_str:N \par } { [PAR] }

then the \par is converted by \token_to_str:N to four 'other' tokens, \, p, a and r, and we would replace that string.

tobiasBora · 2023-09-01T08:24:12Z

Oh, so \tl_replace would be usable on strings as well. Thanks!

car222222 · 2023-09-01T09:57:32Z

One further typo, in ltnews38:
obeydline ==> obeyedline

I shall deal with the (slightly improved, hooray!) documentation later.

Skillmon · 2023-09-04T19:14:07Z

One could further generalise and leave one end-of-line control sequence and one begin-of-line control sequence. Would make processing line wise easier and not hurt if the default would expand to empty. If one has to insert control sequences one could as well make processing really easier and not just hard in another way (and it wouldn't hurt typesetting with an empty control sequence -- though I doubt that direct typesetting is really the most used functionality for +v).

FrankMittelbach · 2023-09-04T20:25:01Z

One could further generalise and leave one end-of-line control sequence and one begin-of-line control sequence. Would make processing line wise easier and not hurt if the default would expand to empty. If one has to insert control sequences one could as well make processing really easier and not just hard in another way (and it wouldn't hurt typesetting with an empty control sequence -- though I doubt that direct typesetting is really the most used functionality for +v).

quite an interesting thought, worth exploring.

car222222 · 2023-09-05T01:49:18Z

+1 (re +v)from me on both:

@FrankMittelbach certainly any such is worth investigating.

@Skillmon on the usage of +v
But with
Questions/Requests

Please can you provide some examples of real-world usages of +v, with associated necessary/possible settings for the two commands.

These examples will be useful both for the consideration stage and as a basis for some badly needed documentation of such uses.

TIA !

josephwright · 2023-09-05T16:34:43Z

Packages/classes I can find using +v

cprotectinside
ducksay
l3doc
letgut
nodetree-embed
scontents

Most of the use cases here seem to be typesetting the content rather than writing to files, etc.

Skillmon · 2023-09-06T20:12:17Z

I use +v in some documentations of mine to typeset examples and show the results next to each other. An example for this is the macro \codeexample in pxpic.dtx (using \scantokens, but writing to a temporary file would be essentially the same).

Other than that ducksay uses +v for its \AddAnimal (directly typeset with EOL turned into \tabularnewline\null via \tl_replace_all:Nnn) and for one of its argument-reading modes (which uses \scantokens as well, and replaces two consecutive EOLs to \par and a single EOL to a single space).

daleif · 2023-09-07T07:21:34Z

I use +v in some documentations of mine to typeset examples and show the results next to each other. An example for this is the macro \codeexample in pxpic.dtx (using \scantokens, but writing to a temporary file would be essentially the same).

Other than that ducksay uses +v for its \AddAnimal (directly typeset with EOL turned into \tabularnewline\null via \tl_replace_all:Nnn) and for one of its argument-reading modes (which uses \scantokens as well, and replaces two consecutive EOLs to \par and a single EOL to a single space).

Nice example (the \codeexample). I'm wondering, can this sort of thing also be done via envs? I have a latex book with many examples and there I use the method of writing it to disk and doing a dual input (input + lstlisting) and I've always wondered if it can be done via code alone (aka no writing to disk)

car222222 · 2023-09-07T12:45:45Z

It may be time to separate the changes to \verb, and friends from the ongoing v-type considerations and possibilities.

The former, where all is resolved, can thus be executed very soon now.

The latter clearly need further ideas and discussion, followed by extensions of the documentation.

josephwright · 2023-09-12T13:32:22Z

@car222222 I've pulled out the parts of the PR that only impacted on \verb, and merged those - so we now cover tabs. That will be in the pre-release, but the adjustments to v-type (both tab support and ^^M tokens) are left here to be resolved.

josephwright · 2023-09-12T13:42:47Z

Marking beginning and end of each line is easy enough when grabbing, but does it really offer a significant step up over simply marking boundaries? I'm not sure I see a use case where there is a non-trivial impact.

Skillmon · 2023-09-14T18:55:16Z

If you want to process the contents line wise you can simply do \def\myBOL#1\myEOL{<stuff>} and are done if you place a BOL and EOL marker. If you only place an EOL marker you're back to checking whether your loop is done after each line, even though the reading process already knew that.

car222222 · 2023-09-15T02:13:57Z

@Skillmon Understood, thanks!

github-actions · 2023-11-14T08:23:28Z

This PR has been automatically marked as stale because it has not had recent activity.

josephwright added 3 commits June 14, 2023 07:19

Support tabs in \verb*

15bfa28

Collect tabs and spaces as actives in ltcmd v-type

75f6369

Collect \endlinechar as \par in ltcmd +v-type

a64deae

Includes a temporary fix to l3doc that will need to be moved back to expl3 in due course.

josephwright linked an issue Jun 14, 2023 that may be closed by this pull request

Support tabs in \verb* #1085

Closed

josephwright requested review from PhelypeOleinik and FrankMittelbach and removed request for PhelypeOleinik June 14, 2023 18:42

FrankMittelbach reviewed Jun 14, 2023

View reviewed changes

josephwright and others added 4 commits June 14, 2023 21:20

Document details of v-type ltcmd argument

8d9bc2a

See latex3/latex3#756.

Adust ltnews to clarify 1 tab -> 1 space

1f74563

Activate tabs in \@vobeyspaces

61049bf

typo

d4cd2a9

[ci skip]

PhelypeOleinik approved these changes Jun 15, 2023

View reviewed changes

FrankMittelbach reviewed Jun 15, 2023

View reviewed changes

josephwright added 2 commits June 15, 2023 07:32

Revert "Activate tabs in \@vobeyspaces"

0276243

This reverts commit 61049bf.

Activate tabs as part of activating spaces

556cc35

car222222 reviewed Jun 17, 2023

View reviewed changes

Typo

fb9ae32

mrpiggi reviewed Jun 20, 2023

View reviewed changes

Grrr

5e67fdd

josephwright added 6 commits September 1, 2023 09:41

Update a test file

49829db

Add a missing "}"

19f8add

Better grammar

b500b91

Update the l3doc hack

3afde70

Provide global definition for active tab

b37bab3

Merge branch 'develop' into ltcmd-verb

9c4eb08

Correct typo

2a88d2b

josephwright mentioned this pull request Sep 12, 2023

Verb adjustments #1136

Merged

6 tasks

github-actions bot added the stale label Nov 14, 2023

josephwright added category base (latex) and removed stale labels Nov 14, 2023

muzimuzhi mentioned this pull request Mar 7, 2024

Document catcodes for xparse's "verbatim" argument type, document how to reproduce \verb latex3/latex3#756

Open

josephwright closed this Mar 21, 2024

josephwright deleted the ltcmd-verb branch March 21, 2024 11:20

		{\catcode`\^^I=\active
		\gdef\@vobeytabs{\catcode`\^^I\active\let^^I\@xobeytab}}

Adjustments to \verb and +v (ltcmd) #1087

Adjustments to \verb and +v (ltcmd) #1087

Conversation

josephwright commented Jun 14, 2023

Internal housekeeping

Status of pull request

Checklist of required changes before merge will be approved

FrankMittelbach left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

car222222 commented Jun 16, 2023

car222222 left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

car222222 commented Jun 17, 2023

josephwright commented Jun 18, 2023

car222222 commented Jun 18, 2023

josephwright commented Jun 18, 2023

car222222 commented Jun 19, 2023

josephwright commented Jun 19, 2023

Choose a reason for hiding this comment

car222222 commented Jul 3, 2023

josephwright commented Jul 3, 2023

tobiasBora commented Sep 1, 2023

josephwright commented Sep 1, 2023

tobiasBora commented Sep 1, 2023

car222222 commented Sep 1, 2023

Skillmon commented Sep 4, 2023

FrankMittelbach commented Sep 4, 2023 • edited

car222222 commented Sep 5, 2023

josephwright commented Sep 5, 2023

Skillmon commented Sep 6, 2023

daleif commented Sep 7, 2023

car222222 commented Sep 7, 2023

josephwright commented Sep 12, 2023

josephwright commented Sep 12, 2023

Skillmon commented Sep 14, 2023

car222222 commented Sep 15, 2023

github-actions bot commented Nov 14, 2023

car222222 left a comment •

edited

FrankMittelbach commented Sep 4, 2023 •

edited