ltnews35.tex

% \iffalse meta-comment
%
% Copyright 2021-2022
% The LaTeX Project and any individual authors listed elsewhere
% in this file.
%
% This file is part of the LaTeX base system.
% -——————————————
%
% It may be distributed and/or modified under the
% conditions of the LaTeX Project Public License, either version 1.3c
% of this license or (at your option) any later version.
% The latest version of this license is in
%    https://www.latex-project.org/lppl.txt
% and version 1.3c or later is part of all distributions of LaTeX
% version 2008 or later.
%
% This file has the LPPL maintenance status "maintained".
%
% The list of all files belonging to the LaTeX base distribution is
% given in the file `manifest.txt'. See also `legal.txt' for additional
% information.
%
% The list of derived (unpacked) files belonging to the distribution
% and covered by LPPL is defined by the unpacking scripts (with
% extension .ins) which are part of the distribution.
%
% \fi
% Filename: ltnews35.tex
%
% This is issue 35 of LaTeX News.

\NeedsTeXFormat{LaTeX2e}[2020-02-02]

\documentclass{ltnews}

%%  Maybe needed only for Chris' inadequate system:
\providecommand\Dash {\unskip \textemdash}

%% NOTE:  Chris' preferred hyphens!
%%\showhyphens{parameters}
%%  \hyphenation{because parameters parameter}

\usepackage[T1]{fontenc}

\usepackage{lmodern,url,hologo}

\usepackage{csquotes}
\usepackage{multicol}
\usepackage{color}

\providecommand\hook[1]{\texttt{#1}}

\providecommand\meta[1]{$\langle$\textrm{\itshape#1}$\rangle$}
\providecommand\option[1]{\texttt{#1}}
\providecommand\env[1]{\texttt{#1}}
\providecommand\Arg[1]{\texttt\{\meta{#1}\texttt\}}


\providecommand\eTeX{\hologo{eTeX}}
\providecommand\XeTeX{\hologo{XeTeX}}
\providecommand\LuaTeX{\hologo{LuaTeX}}
\providecommand\pdfTeX{\hologo{pdfTeX}}
\providecommand\MiKTeX{\hologo{MiKTeX}}
\providecommand\CTAN{\textsc{ctan}}
\providecommand\TL{\TeX\,Live}
\providecommand\githubissue[2][]{\ifhmode\unskip\fi
     \quad\penalty500\strut\nobreak\hfill
     \mbox{\small\slshape(%
       \href{https://github.com/latex3/latex2e/issues/\getfirstgithubissue#2 \relax}%
          	    {github issue#1 #2}%
           )}%
     \par\smallskip}
%% But Chris has to mostly disable \href for his TEXPAD app:
%%  \def\href #1{}  % Only For Chris' deficient TeX engine

% simple solution right now (just link to the first issue if there are more)
\def\getfirstgithubissue#1 #2\relax{#1}

\providecommand\sxissue[1]{\ifhmode\unskip\fi
     \quad\penalty500\strut\nobreak\hfill
     \mbox{\small\slshape(\url{https://tex.stackexchange.com/#1})}\par}

\providecommand\gnatsissue[2]{\ifhmode\unskip\fi
     \quad\penalty500\strut\nobreak\hfill
     \mbox{\small\slshape(%
       \href{https://www.latex-project.org/cgi-bin/ltxbugs2html?pr=#1\%2F\getfirstgithubissue#2 \relax}%
          	    {gnats issue #1/#2}%
           )}%
     \par}

\let\cls\pkg
\providecommand\env[1]{\texttt{#1}}
\providecommand\acro[1]{\textsc{#1}}

\vbadness=1400  % accept slightly empty columns


\makeatletter
% maybe not the greatest design but normally we wouldn't have subsubsections
\renewcommand{\subsubsection}{%
   \@startsection      {subsubsection}{2}{0pt}{1.5ex \@plus 1ex \@minus .2ex}%
      {-1em}{\@subheadingfont\colonize}%
}
\providecommand\colonize[1]{#1:}
\makeatother

\let\finalvspace\vspace          % for document layout fixes

% Undo ltnews's \verbatim@font with active < and >
\makeatletter
\def\verbatim@font{%
  \normalsize\ttfamily}
\makeatletter

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\providecommand\tubcommand[1]{}
\tubcommand{\input{tubltmac}}

\publicationmonth{June}
\publicationyear{2022}

\publicationissue{35}

\begin{document}

\tubcommand{\addtolength\textheight{4.2pc}}   % only for TUB

\maketitle
{\hyphenpenalty=10000 \spaceskip=3.33pt \hbadness=10000 \tableofcontents}

\setlength\rightskip{0pt plus 3em}


\medskip


\section{Introduction}

The 2022 June release of \LaTeX{} is again focussing on improvements
made for our multi-year project to automatically offer tagged PDF
output~\cite{35:blueprint}. These are the new document metadata
interface, the new mark mechanism for \LaTeX{}, a standard key/value
approach for options, and the introduction of the \texttt{latex-lab}
area for temporary code that can be optionally loaded by a document
(when \cs{DocumentMetadata} is used with certain test keys). These
additions are described in the first sections.  Related to this effort
there are updates to \pkg{hyperref} and \pkg{tagpdf} both of which
have their own distributions.

As usual, we also added a number of smaller improvements and bug
fixes in various components of core \LaTeX{}. Perhaps the most
interesting ones (for some users) are direct support for floating
point arithmetic (via \cs{fpeval} see below) and the ability to
properly color parts of math formulas without introducing spacing
problems. For this we now offer the command \cs{mathcolor}; see the
description near the end of the newsletter.  There is also a new major
release of the \pkg{doc} package that supports a more fine-grained
classification of code elements and properly supports \pkg{hyperref}.


\section{Document metadata interface}

Until recently there was no dedicated location to declare
settings that affect a document as a whole. Settings had to be placed
somewhere in the preamble or as class options or sometimes even as
package options. For some such settings this may cause issues, e.g.,
setting the PDF version is only possible as long as the PDF output
file has not yet been opened which can be caused by loading one or the
other package.

For the \enquote{\LaTeX{} Tagged PDF project}~\cite{35:blueprint}
further metadata about the whole document (and its processing) need to
be specified and again this data should be all placed in a single
well-defined place.

For this reason we introduce the new command \cs{DocumentMetadata} to
unify all such settings in one place.  The command expects a key/value
list that describes all document metadata for the current document. It
is only allowed to be used at the very beginning of the document,
i.e., the declaration has to be placed \emph{before}
\cs{documentclass} and will issue an error if found later.


At this point in time we only provide the bare command in the format,
the actual processing of the key/value is defined externally and the
necessary code will be loaded if the command is used. This scheme is
chosen for two reasons: by adding the command in the kernel it is
available to everybody without the need to load a special package
using \cs{RequirePackage}. The actual processing, though, is external
so that we can easily extend the code (e.g., offering additional keys
or changing the internal processing) while the above mentioned project
is progressing. Both together allows users to immediately benefit from
intermediate results produced as part of the project, as well as
offering the \LaTeX{} Project Team the flexibility to enable such
intermediate results (for test purposes or even production use)
in-between and independently of regular \LaTeX{} releases. Over time,
tested and approved functionality can then seamlessly move into the
kernel at a later stage without any alterations to documents already
using it. At the same time, not using the new consolidated interface
means that existing documents are in no way affected by the work that
is carried out and is in a wider alpha or beta test phase.

Documentation about the new command and
already existing keys are in \file{l3meta.pdf} and \file{documentmetadata-support.pdf}
and also in the documentation of the \pkg{pdfmanagement-testphase} package.

Package and class authors can test if a user has used \cs{DocumentMetadata}
with \cs{IfDocumentMetadataTF}.


\section{The \pkg{latex-lab} bundle}

We added a new \pkg{latex-lab}oratory bundle in which we place new
code that is going to be available only through a
\cs{DocumentMetadata} declaration and that is\Dash most
importantly\Dash work under development and subject to
change without further notice. This means, that commands and interfaces provided there may
get altered or removed again after some public testing.  The code can
be accessed through the \cs{DocumentMetadata} key \texttt{testphase}.
Currently supported values are \texttt{phase-I} and \texttt{phase-II}
that enable code of the tagged PDF project (phase-I is frozen and
phase-II the phase we are currently working on).  With
\begin{verbatim}
  \DocumentMetadata{testphase=phase-II}
\end{verbatim}
you currently enable tagging for paragraphs and footnotes, more
document elements will follow soon.

For more detailed testing it is also possible to pass other values to
\texttt{testphase}, for example, the first incarnation of a template
design interface based on \pkg{l3keys} can be accessed through the
value \texttt{prototype}, thus
\begin{verbatim}
  \DocumentMetadata
             {testphase={phase-II,prototype}}
\end{verbatim}
will enable all of phase-II plus the draft template interface (which
is not yet integrated in phase-II).

Eventually, code will move (once considered stable) from the
testphase into the \LaTeX{} kernel itself. Tagging will continue to
require a \cs{DocumentMetadata} declaration, but you will then be able
to drop the \texttt{testphase} key setting.


\section{A new mark mechanism for \LaTeX{}}

The mark mechanism is \TeX{}'s way to pass information to the
page-building process which happens asynchronously in order to
communicate relevant data for running headers and footers to the
latter, e.g., what is the first section on the page or the last
subsection, etc. However, marks may also be
used for other purposes. The new kernel module provides a generalized
mechanism for marks of independent classes.

The \TeX{} engines offer a low-level mark mechanism to
communicate information about the content of the current page to
the asynchronous operating output routine. It works by placing
\cs{mark} commands into the source document.

This mechanism works well for simple formats (such as plain \TeX)
whose output routines are only called to generate pages. It
fails, however, in \LaTeX{} (and other more complex formats),
because here the output routine is sometimes called without
producing a page, e.g., when encountering a float and placing it
into one of the float regions.
%
When that happens \TeX{}'s  \cs{topmark} no
longer reflects the situation at the top of the next page when that
page is finally boxed.

Furthermore, \TeX{} only offered a single mark while \LaTeX{} wanted to
keep track of more than one piece of information.  For that reason,
\LaTeX{} implemented its own mark mechanism where the marks always
contained two parts with their own interfaces: \cs{markboth} and
\cs{markright} to set marks and \cs{leftmark} and \cs{rightmark} to
retrieve them.

Unfortunately, this extended mechanism (while supporting scenarios such
as chapter/section marks) was far from general. The mark
situation at the top of a page (i.e., \cs{topmark}) remained
unusable and the two marks offered were not really independent of
each other because \cs{markboth} (as the name indicates) was
always setting both.


The new mechanism now available in \LaTeX{} starting with the 2022
release overcomes both issues:
\begin{itemize}
\item
  It provides arbitrary many, fully independent named marks, that
  can be allocated and from that point onwards used.
\item
  It offers access for each such marks to retrieve its top,
  first, and bottom value separately.
\item
  Furthermore, the mechanism is augmented to give access to marks
  in different ``regions'' which may not be just full pages.
\end{itemize}
The legacy interfaces, e.g., \cs{markboth}, are kept. Thus classes and
packages making use of them continue to work flawlessly. To make use
of the extended possibility a new set of commands for declaration of
mark classes, setting their values and querying their state (in the
output routine) is now available in addition.
%
You find the documentation for the new interfaces together with
examples and further notes on the mechanism in the file
\texttt{ltmarks-doc.pdf}. Just call \texttt{texdoc}
\texttt{ltmarks-doc} to display it on your computer.


\section{A key/value approach to option handling}

The classical \LaTeXe{} method for handling options, using \cs{ProcessOptions},
treats each entry in the list as a string. Many package authors have sought to
extend this handling by treating each entry as a key--value pair (keyval)
instead. To-date, this has required the use of additional packages, for example
\pkg{kvoptions}.

The \LaTeX{} team have for some time offered the package \pkg{l3keys2e} to
allow keyvals defined using the L3 programming layer module \pkg{l3keys} to act
as package options. This ability has now been integrated directly into the
kernel. As part of this integration, the syntax for processing keyval options
has been refined, such that
\begin{verbatim}
\ProcessKeyOptions
\end{verbatim}
will now automatically pick up the package name as the key \emph{family},
unless explicitly given as an optional argument.
\begin{verbatim}
\ProcessKeyOptions[family]
\end{verbatim}

To support creating key options for this mechanism, the new command
\cs{DeclareKeys} has been added. This works using the same general
approach as \pkg{l3keys} or \pkg{pgfkeys}: each key has one or more
\emph{properties} which define its behavior.

Options for packages which use this new approach will not be checked for
clashes by the kernel. Instead, each time a \cs{usepackage} or
\cs{RequirePackage} line is encountered, the list of options given will be
passed to \cs{ProcessKeyOptions}. Options which can only be given
the first time a package is loaded can be marked using the property
\texttt{.usage = load}, and will result in a warning if used in a subsequent
package loading line.

Package options defined in this way can also be set within a package using
the new command \cs{SetKeys}, which again takes an optional argument
to specify the \emph{family}, plus a mandatory one for the options themselves.



\section{New or improved commands}


\subsection{Floating point and integer calculations}

The L3 programming layer offers expandable commands for calculating
floating point and integer values, but so far these functions have
only been available to programmers, because they require
\cs{ExplSyntaxOn} to be in force. To make them easily available at the
document-level, the small package \pkg{xfp} defined \cs{fpeval} and
\cs{inteval}.


An example of use could be the following:
\begin{verbatim}
  \LaTeX{} can now compute:
  \[ \frac{\sin (3.5)}{2} + 2\cdot 10^{-3}
      = \fpeval{sin(3.5)/2 + 2e-3}         \]
\end{verbatim}
which produces the following output:
\begin{quote}
\LaTeX{} can now compute:
\[ \frac{\sin (3.5)}{2} + 2\cdot 10^{-3}
   = \fpeval{sin(3.5)/2 + 2e-3}         \]
\end{quote}
These two commands have now been moved into the kernel and in addition
we also provide \cs{dimeval} and \cs{skipeval}. The details of their
syntax are described in \file{usrguide3.pdf}.  The command \cs{fpeval}
offers a rich syntax allows for extensive calculations whereas the
other three commands are essentially thin wrappers for \cs{numexpr},
\cs{dimexpr}, and \cs{glueexpr} \Dash therefore inheriting some syntax
peculiars and limitations in expressiveness.
\begin{verbatim}
  \newcommand\calulateheight[1]{%
    \setlength\textheight{\dimeval{\topskip
          + \baselineskip * \inteval{#1-1}}}}
\end{verbatim}
The above, for example, calculates the appropriate \cs{textheight} for
a given number of text lines.
%
\githubissue{711}


\subsection{CamelCase commands for changing arguments to csnames}

It is sometimes helpful to \enquote{construct} a command name on the
fly rather than providing it as a single \cs{...} token. For these
kind of tasks the \LaTeX3 programming layer offers a general mechanism
(in form of \cs{exp\_args:N...} and
\cs{cs\_generate\_variant:Nn}). However, when declaring new
document-level commands with \cs{NewDocumentCommand} or
\cs{NewCommandCopy}, etc.\ the L3 programming layer may not be active,
and even if it is, mixing CamelCase syntax with L3 programming syntax
is not really a good approach. We have therefore added the commands
\cs{UseName} and \cs{ExpandArgs} to assist in such
situations, e.g.,
\begin{verbatim}
\NewDocumentCommand\newcopyedit{mO{red}}
  {\newcounter{todo#1}%
   \ExpandArgs{c}\NewDocumentCommand{#1}{s m}%
     {\stepcounter{todo#1}%
      \IfBooleanTF {##1}%
         {\todo[color=#2!10]%
            {\UseName{thetodo#1}: ##2}}%
         {\todo[inline,color=#2!10]%
            {\UseName{thetodo#1}: ##2}}%
     }%
  }
\end{verbatim}
which provides a declaration mechanism for copyedit commands, so that
\verb=\newcopyedit{FMi}[blue]= then defines \cs{FMi} (and the
necessary counter).

The command \cs{ExpandArgs} can be useful with the argument \texttt{cc} or
\texttt{Nc} in
combination with \cs{NewCommandCopy} if the old or new command name
or both need constructing. Finally, there is \cs{UseName} which
takes its argument and turns it into a command (i.e., a CamelCase
version of \cs{@nameuse} (\LaTeXe) or \cs{use:c} (L3 programming
layer)) which was also used in the example above.
%
\githubissue{735}


\subsection{Testing for (nearly) empty arguments}
%
In addition to \cs{IfNoValueTF} to test if an optional argument was
provided or not, there is now also \cs{IfBlankTF}, which tests if the
argument is empty or contains only blanks. Based on the result it
selects a true or false code branch. As usual, the variants
\cs{IfBlankT} and \cs{IfBlankF} are also provided for use when only one
branch leads to some action.  Further details and examples are given
in \file{usrguide3.pdf}.

This test can also be useful if you set up key/value options and want
to test if a key was specified without giving a value or through
specifying \enquote{\textit{key}\texttt{ = ,}}.


\subsection{Better allocator for Lua command ids}

In \LuaTeX\ we already had the \cs{newluafunction} macro which allocates
a Lua function identifier which can be used to define commands
with \cs{luadef}. But this always required two steps: \cs{newluafunction}
defines the passed control sequence as an integer, which then has to be used
to define the actual Lua command with \cs{luadef}. After that, the integer is
no longer needed. This was inconsistent with other allocators. Therefore we
added two new allocators \cs{newluacmd} and \cs{newexpandableluacmd} which
directly define a control sequences invoking the allocated Lua function.
The first one defines a non-expandable Lua command, the second one an expandable
one. Of course, the associated Lua function still has to be defined by assigning
a function to the \verb|lua.get_functions_table()| table. The required index is
available in \cs{allocationnumber}.

An example could be
\begin{verbatim}
\newluacmd \greeting
\directlua {
lua.get_functions_table()
   [tex.count.allocationnumber]
   = function()
    local name = token.scan_argument()
    tex.sprint('Hello ', name, '!')
  end
}

\greeting{world}
\end{verbatim}
%
\githubissue{536}

\subsection{Starred command version for \cs{ref}, \cs{Ref} and \cs{pageref}}

For a long time \pkg{hyperref} provides starred versions for the reference commands
that don't create active links. This syntax extension required users and
package authors to check if \pkg{hyperref} was loaded and adjust the coding
accordingly or take the starred forms out if text was copied to a document
without \pkg{hyperref}. The commands have now been aligned with
the \pkg{hyperref} usage and always allow an optional star. The \pkg{showkeys} package
has been updated to handle the starred versions too, both with \pkg{hyperref} or \pkg{nameref}
and without. The commands are defined with \cs{NewDocumentCommand} and so no longer expand when
written to auxiliary files. This reduces the number of compilations needed to resolve references
in captions and sectioning commands. The package \pkg{ifthen} has been updated to ensure that
\cs{pageref} can still be used inside tests like \cs{isodd}.


\subsection{Preparation for supporting PDF in backends}

At the current point in time, basic support for PDF in
backends is not part of \LaTeX{} core; it is provided by external
package like \pkg{hyperref}.
At some time in the future that work will be placed
into the kernel but for now it is separate and has to be
explicitly loaded in the document. To enable class and package authors
to support PDF specific task like the creation of link targets without
having to test first if \pkg{hyperref} has been loaded, dummy versions of
the commands \cs{MakeLinkTarget}, \cs{LinkTargetOn}, \cs{LinkTargetOff} and
\cs{NextLinkTarget} are provided.

\section{Code improvements}

\subsection{\cs{protected} UTF-8 character definitions}
The characters defined via \file{utf8.def} are now defined as \cs{protected}
macros. This makes them safe to use in expansion contexts where the
classic \cs{protect} mechanism is not enabled, notably L3 programming
layer \texttt{e} and \texttt{x} arguments.

Related to this change \cs{MakeUppercase} and \cs{MakeLowercase} have
been updated to use the Unicode-aware case changing functions
\cs{text\string_lowercase:n} in place of the \TeX-primitive \cs{lowercase}.
A similar change will be made in the \pkg{textcase} package.

Note for technical reasons these low level character handling changes
will not be rolled back if the format version is rolled back using the
\pkg{latexrelease} package rollback mechanism.
%
\githubissue{780}

\subsection{A small update to \cs{obeylines} and \cs{obeyspaces}}

The plain \TeX{} versions of \cs{obeylines} and \cs{obeyspaces} make
\verb=^^M= and \verb*= = active and force them to execute \cs{par} %*
and \cs{space}, respectively. Don Knuth makes a remark in the \TeX{}book
that one can then use a trick such as
\begin{verbatim}
  \let\par=\cr \obeylines \halign{...
\end{verbatim}
However, redefining \cs{par} like this is not really a great idea in \LaTeX{},
because it may lead to all kind of problems. We have therefore changed
the commands to use an indirection: the active characters now execute
\cs{obeyedline} and \cs{obeyedspace}, which in turn do what the
hardwired solution did before.

\begin{quote}
  \renewcommand\obeyedspace{\ \textbullet\ }
  \footnotesize\obeyspaces%
But this means that it is now possible to %
achieve special effects in a safe way. %
This paragraph, for example, was produced by %
making \cs{obeyedspace} generate %
\texttt{\{\cs{\verbvisiblespace}\cs{textbullet}\cs{\verbvisiblespace}\}} and %
enabling \cs{obeyspaces} within a %
quote environment.
\end{quote}
Thus, if you are keen to use the plain \TeX{} trick, you need to say
\cs{let}\cs{obeyedlines}\texttt{=}\cs{cr} now.
%
\githubissue{367}



\subsection{\class{ltxdoc} gets a \option{nocfg} option}

The \LaTeX{} sources are formatted with the \class{ltxdoc} class,
which supports loading a local config file \file{ltxdoc.cfg}. In the
past the \LaTeX{} sources used such a file but it was not distributed.
As a result reprocessing the \LaTeX{} sources elsewhere showed
formatting changes.  We now distribute this file which means that it
is loaded by default. With the option \option{nocfg} this can be
prevented.


\subsection{\pkg{doc} upgraded to version~3}

After roughly three decades the \pkg{doc} package gets a cautious
uplift, as already announced at the TUG conference 2019\Dash changes
to \pkg{doc} are obviously always done in a leisurely manner.

Given that most documentation is nowadays viewed on screen,
\pkg{hyperref} support is added and by default enabled (suppress it
with option \option{nohyperref} or alternatively with
\option{hyperref}\texttt{=false}) so the internal cross-reference are
properly resolved including those from the index back into the
document.

Furthermore, \pkg{doc} has now a general mechanism to define
additional \enquote{doc} elements besides the two \texttt{Macro} and
\texttt{Env} it did know in the past. This enables better
documentation because you can now clearly mark different types of
objects instead of simply calling them all \enquote{macros}.
If desired, they can be collected together under a heading
in the index so that you have a section just with your document
interface commands, or with all parameters, or \ldots

The code borrows ideas from Didier Verna's \pkg{dox} package (although
the document level interface is different) and it makes use of Heiko
Oberdiek's \pkg{hypdoc} package, which at some point in the future
will be completely integrated, given that its whole purpose it to
patch \pkg{doc}'s internal commands to make them \pkg{hyperref}-aware.

All changes are expected to be upward compatible, but if you run into
issues with older documentation using \pkg{doc} a simple and quick
solution is to load the package as follows:
\verb/\usepackage{doc}[=v2]/

\subsection{\pkg{doc} can now show dates in change log}

Up to now the change log was always sorted by version numbers
(ignoring the date that was given in the \cs{changes} command).  It
can now be sorted by both version and date if you specify the option
\option{reportchangedates} on package level and in that case the
changes are displayed with
\begin{quote}
  \meta{version} -- \meta{date}
\end{quote}
as the heading (instead of just \meta{version}), when using
\cs{PrintChanges}.
%
\githubissue{531}



\subsection{Lua\TeX\ callback improvements}

The Lua\TeX\ callbacks \texttt{hpack\_quality} and \texttt{vpack\_quality} are
now \texttt{exclusive} and therefore only allow a single handler.
The previous type \texttt{list} resulted in incorrect parameters when multiple
handlers were set, therefore this only makes an existing restriction more
explicit.

Additionally the return value \texttt{true} for \texttt{list}
callbacks is now handled internally and no longer passed on to the
engine. This simplifies the handling of these callbacks and makes it
easier to provide consistent interfaces for user defined \texttt{list}
callbacks.


\subsection{Class \class{proc} supports \option{twoside}}

The document class \class{proc}, which is a small variation on the
\class{article} class, now supports the \option{twoside} option
displaying different data in the footer line on recto and verso pages.
%
\githubissue{704}


\subsection{Croatian character support}

The default \pkg{inputenc} support has been extended to support the 9 characters
D\v Z, D\v z, d\v z, LJ, Lj, lj, NJ, Nj, nj, input as single UTF-8 code points
in the range U+01C4 to U+01CC.
%
\githubissue{723}


\subsection{Cleanup of the Unicode declaration interface}

When declaring encoding specific commands for the Unicode (TU)
encoding some declarations (e.g., \cs{DeclareUnicodeComposite}) do not
have an explicit argument for the encoding name, but instead use
the command \cs{UnicodeEncodingName} internally. There was one
exception though: \cs{DeclareUnicodeAccent} required an explicit
encoding argument.  This inconsistency has now been removed and the
encoding name is always implicit. To avoid a breaking change for a few
packages on CTAN \cs{DeclareUnicodeAccent} still accepts three
arguments if the second argument is \texttt{TU} or
\cs{UnicodeEncodingName}. Once all packages have been updated this
code branch will get removed.

At the same time we added \cs{DeclareUnicodeCommand} and
\cs{DeclareUnicodeSymbol} for consistency. They also use
\cs{UnincodeEncodingName} internally, instead of requiring an encoding
argument as their general purpose counterparts do.
%
\githubissue{253}


\subsection{New hook: \hook{include/excluded}}

A few releases ago we introduced a number of file hooks for different
types of files, see~\cite{35:ltnews32} and in
particular~\cite{35:ltfilehook-doc}.
%
The hooks for \cs{include} files now got an addition: if such a file
is not included (because \cs{includeonly} is used and its \meta{name}
is not listed in the argument) then the hooks \hook{include/excluded}
and \hook{include/\meta{name}/excluded} are executed in that
order\Dash of course, only if they contain code.  This happens after
\LaTeX{} has loaded the \texttt{.aux} file for this include file,
i.e., after \LaTeX{} has updated its counters to pretend that the file
was seen.


\subsection{Input support for normalized angle brackets}

Source files containing \textlangle\ or \textrangle\ directly written
as Unicode codepoints U+2329 and U+232A no longer break when the
source file gets normalized under Unicode normalization rules.
%
\githubissue{gh/714}




\section{Bug fixes}

\subsection{Using \cs{DeclareUnicodeCharacter} with C1 control points}
An error in the UTF-8 handling for non-Unicode \TeX, has prevented
\cs{DeclareUnicodeCharacter} being used with characters in the range
hex 80 to 9F, this has been corrected in this release.
%
\githubissue{730}



\subsection{Fix \cs{ShowCommand} when used with \pkg{ltcmd}}

When \cs{ShowCommand} support was added for \pkg{ltcmd} in the previous
release~\cite{35:ltnews34}, a blunder in the code made it so that
when \cs{ShowCommand} was used on a command defined with \pkg{ltcmd}, it
only printed the meaning of the command in the terminal, but didn't stop
for interaction as it does elsewhere (mimicking \cs{show}).  The issue
is now fixed.
%
\githubissue{739}



\subsection{Make \cs{cite}\texttt{\textbraceleft\textbraceright} produce a warning}

When the \cs{cite} command can't resolve a citation label it issue a
warning \enquote{Citation `\meta{label}' on page \meta{page}
  undefined}.  However, due to some implementation details a completely
empty argument was always silently accepted. Given that there are probably
people who write \verb=\cite{}= with the intention to fill in the
correct label later it is rather unfortunate if that is not generating
a warning that something in the document is still amiss.
This has finally been corrected and a warning is now generated also in this case.
%
\githubissue{790}



\subsection{Fix adding \hook{cmd} hooks to simple macros}

A bug in how \LaTeX{} would detect the type of a command was causing a
premature forced expansion of such commands, which, depending on their
definition, could be harmless or could cause severe trouble.  This has
been fixed in the latest release.
%
\githubissue{795}
\sxissue{q/637565}


\subsection{Warn if \hook{shipout/lastpage} hook is executed too early}

The hook \hook{shipout/lastpage} is intended to place \cs{special}s
into the last page shipped out. This is needed for some use cases,
e.g., tagging. If that hook is nonempty and the user has added additional
pages since the last run, then \LaTeX{} executes this hook too early,
but until now without giving any indication that the document needs
rerunning. This has now been corrected and an appropriate warning is
given.
%
\githubissue{813}


\subsection{More consistent use of cramped math styles in \LuaTeX}

Using \LuaTeX's \cs{Udelimiterover} to place a horizontally extensible glyph
on top of a mathematical expression now causes the expression to be set in cramped
style as used in similar situations by traditional \TeX\ math rendering.
Similarly cramped style is now used for expressions set under such a delimiter
using \cs{Uunderdelimiter} but no longer used when setting an expression on top
of such extensible glyphs using \cs{Uoverdelimiter}.
This new behavior follows \TeX's rule that cramped style is used whenever something
else appears above the expression.
Additionally the math style of these constructs can now be detected using \cs{mathstyle}.

The old behavior can be restored by adding
\begin{verbatim}
   \mathdefaultsmode=0
\end{verbatim}
to a document.


\subsection{Fixed bug when setting hook rules for one-time hooks}

If a \cs{DeclareHookRule} command is set for a one-time hook, it has to
come \emph{before} the hook gets used, because otherwise it never
applies\Dash after all, the hook is used only once.  There was a bug in
the implementation in that the sorting mechanism was still applied if
the \cs{DeclareHookRule} declaration appeared while the one-time hook was
executed, causing the spurious typesetting of the code labels and the
hook name.  This bug is now fixed and an error is raised when a new
sorting rule is added to an already used one-time hook.

A possible scenario in which this new error is raised is the following:
package \pkg{AAA} declares a hook rule for \hook{begindocument} (i.e.,
\cs{AtBeginDocument}) to sort out the behavior between itself and some
other package. Package \pkg{BBB} wants to load package \pkg{AAA} but
only if it hasn't been loaded in the preamble, so delays the loading to
\hook{begindocument}. In that case the hook rule declared by \pkg{AAA}
can no longer be applied and you get the error. If that happens the
solution is to load the package in \hook{begindocument/before}, which
is executed at the very end of the preamble but before
\hook{begindocument} is processed.
%
\githubissue{818}




\section{Changes to packages in the \pkg{amsmath} category}


\subsection{\pkg{amsopn}: Do not reset \cs{operator@font}}

The package \pkg{amsopn} used to define \cs{operator@font} but this
command is already provided by the \LaTeX{} format (for at least 14
years). As a result the definition in \pkg{amsopn} is equivalent to a
reset to the kernel definition, which is unnecessary and surprising if
you alter the math setup (e.g., by loading a package) and at a later
stage add \pkg{amsmath}, which then undoes part of your setup. For
this reason the definition was taken out and
\pkg{amsmath}/\pkg{amsopn} now relies on the format definition.

In the unlikely event that you want the resetting to happen, use
\begin{verbatim}
  \makeatletter
   \def\operator@font{\mathgroup\symoperators}
  \makeatother
\end{verbatim}
after loading the package.
%
\githubissue{734}

\subsection{\pkg{amsmath}: Error in \cs{shoveleft}}

If \cs{shoveleft} started out with the words \enquote{plus} or
\enquote{minus} it was misunderstood as part of a rubber length and
led either to an error or was swallowed without trace.  By adding a
\cs{relax} this erroneous scanning into the argument of \cs{shoveleft}
is now prevented.
%
\githubissue{714}


\subsection{\pkg{amsmath} and \pkg{amsopn}: Robustify user commands}

Most user-level commands have been made robust in the \LaTeX{} kernel
during the last years, but variant definitions in \pkg{amsmath} turned
them back into fragile beings. We have now made most commands in
\pkg{amsmath} and \pkg{amsopn} robust as well to match the kernel
behavior. This also resolves a bug recently discovered in the
\pkg{mathtools} package, which was due to \cs{big} not being robust after
\pkg{amsmath} was loaded.
%
\githubissue{123}



\section{Changes to packages in the \pkg{graphics} category}

\subsection{Color in formulas}

While it is possible to color parts of a formula using \cs{color}
commands the approach is fairly cumbersome. For example, to color an
summation sign, but not its limits, you need four \cs{color} commands and
some seemingly unnecessary set of braces to get coloring and spacing
right:
\begin{verbatim}
\[ X = \color{red} \sum
% without {{ the superscript below is misplaced
                    _{{\color{black} i=1}}
% without {{ the \sum is black
                    ^{{\color{black} n}}
   \color{black}    % without it the x_i is red
   x_i           \]
\end{verbatim}
Leave out any of the \cs{color} commands or any of the \verb={{...}}=
will give you a wrong result instead of the desired
\[ X = \color{red} \sum
    _{{\color{black} i=1}} % without {{ the superscript is misplaced
    ^{{\color{black} n}}   % without {{ the \sum is black
   \color{black}           % without it the x_i is red
   x_i
\]
So even if this is possible, it is not a very practical solution and
furthermore there are a number of cases where it is impossible to
color a certain part of a formula, for example, an opening symbol such
as \verb=\left(= but not the corresponding \verb=\right)=.

We have therefore added the command \cs{mathcolor} to the \pkg{color}
and \pkg{xcolor} package, which has the same syntax as \cs{textcolor},
but is specially designed for use in math and handles sub and
superscripts and other aspects correctly and preserves correct
spacing. Thus, the above example can now be written as
\begin{verbatim}
\[ X = \mathcolor{red}{\sum}_{i=1}^n x_i \]
\end{verbatim}
This command is \emph{only} allowed in formulas.
For details and further examples, see \file{mathcolor.pdf}.


\subsection{Fix locating files with \cs{graphicspath}}
%

If a call to \cs{includegraphics} asked for a file (say, \file{image})
without extension, and if both \file{A/image.pdf} and \file{B/image.tex}
existed (both \file{A/} and \file{B/} in \cs{graphicspath}, but neither
in a folder searched by kpse), then \file{A/image.pdf} would not be
found, and a \enquote{file not found} error would be incorrectly thrown.
The issue is now fixed and the graphics file is correctly found.
%
\githubissue{776}
\sxissue{q/630167}




\section{Changes to packages in the \pkg{tools} category}

\subsection{\pkg{multicol}: Fix \cs{newcolumn}}

The recently added \cs{newcolumn} didn't work properly if used in
vertical mode, where it behaved like \cs{columnbreak}, i.e., spreading
the column material out instead running the column short.
%
\sxissue{q/624940}


\subsection{\pkg{bm}: Fix for \pkg{amsmath} operators}

An internal command used in the definition of operator commands such
as \cs{sin} in \pkg{amsmath} has been guarded in \cs{bm} to prevent
internal syntax errors due to premature expansion.
%
\githubissue{744}



\medskip

\begin{thebibliography}{9}

\fontsize{9.3}{11.3}\selectfont

\bibitem{35:blueprint} Frank Mittelbach and Chris Rowley:
  \emph{\LaTeX{} Tagged PDF \Dash A blueprint for a large project}.\\
  \url{https://latex-project.org/publications/indexbyyear/2020/}

%\bibitem{35:source2e}
%  \emph{\LaTeX{} documentation on the \LaTeX{} Project Website}.\\
%  \url{https://latex-project.org/help/documentation/}

%\bibitem{35:ltnews31} \LaTeX{} Project Team:
%  \emph{\LaTeXe{} news 31}.\\
%  \url{https://latex-project.org/news/latex2e-news/ltnews31.pdf}

\bibitem{35:ltnews32} \LaTeX{} Project Team:
  \emph{\LaTeXe{} news 32}.\\
  \url{https://latex-project.org/news/latex2e-news/ltnews32.pdf}

%\bibitem{35:ltnews33} \LaTeX{} Project Team:
%  \emph{\LaTeXe{} news 33}.\\
%  \url{https://latex-project.org/news/latex2e-news/ltnews33.pdf}

\bibitem{35:ltnews34} \LaTeX{} Project Team:
  \emph{\LaTeXe{} news 34}.\\
  \url{https://latex-project.org/news/latex2e-news/ltnews34.pdf}

\bibitem{35:ltfilehook-doc} Frank Mittelbach, Phelype Oleinik, \LaTeX{} Project Team:
  \emph{The \texttt{\upshape ltfilehook} documentation}.\\
  Run \texttt{texdoc} \texttt{ltfilehook-doc} to view.
\end{thebibliography}



\end{document}