Incompatible with menukeys package

[alexSmercury]

Loading the menukeys package means hereapplies does not store anything in the .hax file. This then prevents whereapplies from working.

It does not matter whether menukeys is loaded before or after hereapplies.

[madmurphy]

Hi @alexSmercury

Thank you for reporting this. I have given a look at the menukeys package, but I have no idea what causes the problem. And, oddily enough, if I place the menukeys.sty file in the same directory in which the .tex file is located (the .tex file I am compiling) everything seems to be working fine (could you please confirm?).

It might not be a bad idea to ask the developer of the menukeys package for help too…

@tweh

Could you please have a look at this?

[alexSmercury]

And, oddily enough, if I place the menukeys.sty file in the same directory in which the .tex file is located (the .tex file I am compiling) everything seems to be working fine (could you please confirm?).

I couldn't replicate this. The hax file is never populated no matter whether it's menukeys.sty from the MikTeX distribution or one moved into the same directory as the document being compiled. I checked the .log to confirm the local one was being used.

I cleaned all the intermediate files including removing the .hax file for these tests. I also tried swapping the load order of menukeys and hereapplies.

Perhaps what you have discovered is that the ability to write to the .hax file is broken so an existing file will still work but won't be updated.

I'm using the latest menukeys and hereapplies that MikTeX update provides. The kernel and other things were updated last week.

This is LuaHBTeX, Version 1.15.1 (MiKTeX 22.11.25)
LaTeX2e <2022-11-01> patch level 1
 L3 programming layer <2022-11-02>
...
Package: menukeys 2022/07/01 v1.6.2 -- A package to format menus, paths and sho
rtcuts
...
Package: hereapplies 2022/09/13 Here Applies

[Skillmon]

This is not menukeys but you're actually incompatible with TikZ, which is one of menukeys dependencies.

See the following MWE which also produces an empty .hax:

\documentclass{article}

\RequirePackage{tikz}
\usepackage{hereapplies}

\begin{document}

\title{Some title}

\author{Some author}

\maketitle

This is concept one. To find this concept applied, please
see \whereapplies{conceptOne}.

This is concept two. To find this concept applied, please
see \whereapplies{conceptTwo}.\newpage

\hereapplies{conceptOne} This is page \thepage. As you can see,
``concept one'' applies here.\newpage

\hereapplies{conceptTwo} This is page \thepage. As you can see,
``concept two'' applies here.\newpage

\hereapplies{conceptOne, conceptTwo} This is page \thepage. As you
can see, both ``concept one'' and ``concept two'' apply here.\newpage

\hereapplies{conceptTwo} This is page \thepage. As you can see,
``concept two'' applies here.\newpage

\hereapplies[myref]{conceptOne} This is page \thepage. As you can
see, ``concept one'' applies here. This point in the document is
labeled \texttt{myref}.

\end{document}

I'm not entirely sure why this happens, but an easy fix is putting the following in the preamble.

\makeatletter
\AtEndDocument{%
    \addtocontents{hax}{\@ha@commons@@haxcontent}%
}%
\makeatother

[alexSmercury]

Based on @Skillmon's finding I drilled down. It seems to be line 93 of pgfutil-latex.def

\AtBeginDocument{\AtEndDocument{\clearpage\the\pgfutil@everybye}}

I do not know enough about TeX/LaTeX to know why this line is causing the problem. Nor do I really understand why @Skillmon's fix works.

Search method

It is possible my investigation method is flawed, so here is what I did.

Started with the example provided by @Skillmon and added \listfiles

Based on the file list produced, deduced it was not xcolor, graphicx, atveryend-ltx, atbegshi-ltx, epstopdf-base or trig by trying all those separately without tikz.

Substituted \usepackage{tikz} with usepackage{pgf} and started working through the list. I copied the pgf file I was trying into the same directory as the test file and checked the log to see that lualatex was loading the local copy not the distribution version. I narrowed it down to pgfrcs which loads three files:

\input pgfutil-common.tex
\input pgfutil-latex.def

\input{pgfrcs.code.tex}

Deduced that pgfutil-common.tex was not causing an empty .hax file but pgfutil-latex.def was.

Started commenting out code in the local copy of pgfutil-latex.def from the back until a non-empty .hax was produced.

Toggling line 93 as shown above seems to be the one that stops the .hax file being populated.

Failed attempts at fixing the code

In hereafter.sty version 1.0.0 2022/09/13 I tried changing line 312 from

\AtEndDocument{%

to \AddToHook{
using in turn:

1. enddocument/afterlastpage
2. enddocument/afteraux
3. enddocument/info
4. enddocument/end
   but none worked.

Keeping the original \AtEndDocument and doing what pgfutil-common.tex does by wrapping it in \AtBeginDocument causes an error can only be used in preamble on the first occurrence of \whereapplies. I tried using \AddToHook{begindocument/end}{\AtEndDocument....} but that doesn't work either.

Mainly this proves I don't know enough about TeX hooks.

[alexSmercury]

Possible fix

This seems to work.
Change line 312 of hereapplies.sty
from

\AtEndDocument{%

to

\AtEndEnvironment{document}{%

Needs \RequirePackage{etoolbox}

[Skillmon]

I'd argue that this is still wrong @alexSmercury. Imho, there is no good reason to only conditionally write the .hax file anyway as is done in the package currently. Instead I'd restructure the package to move the entire code block of lines 304 to 315 outside of the definition of \@ha@newidentifier. The following does that and works:

\documentclass{article}

\RequirePackage{tikz}
\usepackage{hereapplies}
\makeatletter
\gdef\@ha@newidentifier#1{%
    % Was this identifier already initialized?
    \ifcsname @ha@iter@@preamble@#1\endcsname\else%
        % The identifier was never initialized
        % Was a .hax file already exported during a previous run?
        \ifcsname @ha@prop@@labels@#1\endcsname\else%
            % This is the first run
            % Set the output to "??" - to be updated by the .hax file
            \expandafter\gdef\csname
                @ha@prop@@doutput@#1\endcsname{\textbf{??}}%
            % Set the starred output to "??" - to be updated by the .hax file
            \expandafter\gdef\csname
                @ha@prop@@soutput@#1\endcsname{\textbf{??}}%
            % Set the list of labels to an empty value
            \expandafter\gdef\csname @ha@prop@@labels@#1\endcsname{}%
        \fi%
        % Use "p." for the preamble when there is only one occurrence
        \global\expandafter\let\csname @ha@iter@@preamble@#1\endcsname\hapage%
        % Generate the output strings
        \g@addto@macro\@ha@commons@@haxcontent{%
            % Make sure that there are occurrences
            \ifcsname @ha@iter@@labels@#1\endcsname%
                % There are occurrences
                % Generate the output strings
                \protect\@ha@makeoutputstrings{#1}{\csname
                    @ha@iter@@preamble@#1\endcsname}{\csname
                    @ha@iter@@labels@#1\endcsname}%
            \fi%
        }%
    \fi%
}
% Was the .hax input already initialized during this run?
\ifdefined\@ha@commons@@haxcontent\else%
    % The .hax input was never initialized
    % Previous versions created unwanted whitespaces; I am thankful to
    % David Carlisle for suggesting `\endlinechar=\m@ne`
    {\endlinechar=\m@ne\@starttoc{hax}}%
    % Initialize the content to export to the .hax file
    \gdef\@ha@commons@@haxcontent{}%
    % Export the content when the document reaches the end
    \AtEndDocument{%
        \addtocontents{hax}{\@ha@commons@@haxcontent}%
    }%
\fi%
\makeatother

\begin{document}

\title{Some title}

\author{Some author}

\maketitle

This is concept one. To find this concept applied, please
see \whereapplies{conceptOne}.

This is concept two. To find this concept applied, please
see \whereapplies{conceptTwo}.\newpage

\hereapplies{conceptOne} This is page \thepage. As you can see,
``concept one'' applies here.\newpage

\hereapplies{conceptTwo} This is page \thepage. As you can see,
``concept two'' applies here.\newpage

\hereapplies{conceptOne, conceptTwo} This is page \thepage. As you
can see, both ``concept one'' and ``concept two'' apply here.\newpage

\hereapplies{conceptTwo} This is page \thepage. As you can see,
``concept two'' applies here.\newpage

\hereapplies[myref]{conceptOne} This is page \thepage. As you can
see, ``concept one'' applies here. This point in the document is
labeled \texttt{myref}.

\end{document}

[madmurphy]

@alexSmercury
@Skillmon

Thank you both for finding and studying the problem!

@Skillmon

Imho, there is no good reason to only conditionally write the .hax file anyway as is done in the package currently. Instead I'd restructure the package to move the entire code block of lines 304 to 315 outside of the definition of \@ha@newidentifier.

Writing the .hax file conditionally is simply for avoiding to export anything at all if a document for some reason includes my package but never uses it. I can adopt your solution, but one might argue that “there is no good reason” why the current code should not work…

--madmurphy

[alexSmercury]

@madmurphy You're welcome. I want to use hereapplies and I need it to work with tikz.

I'm not good enough at LaTeX to know what's the best solution. But I agree with @Skillmon that prevention is better than cure.

Writing the .hax file conditionally is simply for avoiding to export anything at all if a document for some reason includes my package but never uses it.

@madmurphy There might be good reasons for someone including hereapplies but not using it on that particular compilation run. I admit I can't of one right now. Compatibility testing maybe. Perhaps hereapplies could offer a setting for this or print a message to the job log.

Unrelated: @Skillmon I just realized where I "know" you from. I've been using duckuments as test text. Small world. With a lot of ducks.

[Skillmon]

@alexSmercury Well, I'm also one of the menukeys maintainers now (as Tobias doesn't have enough time to maintain it alone and I somehow slipped into being one of the maintainers when I fixed it for recent LaTeX kernels), and not only author of duckuments (but I'm glad that you like that little fun package).

[madmurphy]

Okay, this should fix the issue and still not create any .hax file if the package is included but not used.

@alexSmercury
@Skillmon

Could you please give it a try?

--madmurphy

[alexSmercury]

The patch works for me on my test files.

This is LuaHBTeX, Version 1.15.1 (MiKTeX 22.11.25)
 restricted system commands enabled.
(./testhereapplies.tex
LaTeX2e <2022-11-01> patch level 1
 L3 programming layer <2022-11-02> (C:/MikTeX/tex/latex/base/article.cls

Document Class: article 2022/07/02 v1.4n Standard LaTeX document class

[madmurphy]

The patch works for me on my test files.

Alright then. I have updated the package on CTAN as well. Thank you for helping improve this package!

--madmurphy

[Skillmon]

Just from the looks of it I'd say looks fine, no need to run it as well I guess.

[madmurphy]

@Skillmon

Thank you too for your help.

--madmurphy