Package specific code in luaotfload

[davidcarlisle]

Recent commits starting from

3b39119#diff-8a308e92f72a92c8e3b3e4c2ead6ee55

have introduced package specific code into luaotfload.

This seems a bit unfortunate. Conceptually luaotfload/luatexbase are providing format level access to the font loading and package-specific hooks here seem to be at the wrong level.

It wasn't entirely clear what the issue was looking at the code, but could it be addressed in beamer instead?

[dohyunkim]

I have committed the commit.

It's not a beamer's (or more precisely pgf's) fault. That kind of hack was necessary because we cannot access the current contents of a tex macro via lua code at the luatex's (prelinebreak, postlinebreak or whatever) callback stage.

The more fundamental issue here is that we have no common interface or no common practice for the implementation of transparent colors. For instance, upon compiling the following code we get no transparency in the resulting pdf.

\pdfcompresslevel=0
\documentclass{minimal}
\usepackage{pgf}
\usepackage{transparent}
\begin{document}
\leavevmode
\setbox0\hbox{ABC}%
\copy0
\kern-0.96\wd0
\texttransparent{0.5}{\box0}
\end{document}

The pdf file will have an object like this:

% 4 0 obj
<<
 /ColorSpace 3 0 R /Pattern 2 0 R /ExtGState 1 0 R /ExtGState<</TRP1<</ca 1/CA 1>>/TRP.5<</ca .5/CA .5>>>>

Here, /ExtGState 1 0 R is inserted by pgf package; /ExtGState<</TRP1<</ca 1/CA 1>>/TRP.5<</ca .5/CA .5>>>> by transparent package. They conflict each other, so we get no transparency! When you remove the pgf package from loading, everything is OK. The hack I've tackled was to deal with this kind of headache.

By the way, I have requested a new PR, which is more simplified and might be more clear to other people.

[davidcarlisle]

@dohyunkim hi yes I see the issue but still I think that it's the wrong place to hook the code, Currently luaotfload/luatexbase are not loaded into the format but they are conceptually at that level and the ltluatex repository shows one possible plan for moving luatexbase into the format.
package specific tests should not be here but in the package.

luaotfload should supply the necessary lua and tex hooks so that packages interface though some defined API, just as luatexbase disables the luatex primitive callback registration. and so forces packages to use its methods. If you add pgf specific code today then any other package doing similar or even identical things will expect similar support at this level.

if

 \newtoks\pgf@sys@pgf@resource@list@extgs@toks

was renamed as some sort of luaotfload defined hook then the redefinitions

\def\pgf@sys@pgf@resource@list@extgs{\the\pgf@sys@pgf@resource@list@extgs@toks}
    \def\pgf@sys@addpdfresource@extgs@plain#1{\global\pgf@sys@pgf@resource@list@extgs@toks
      \expandafter{\the\pgf@sys@pgf@resource@list@extgs@toks #1}}

could be in the pgf package when it detects it is running in luatex, that way the timing is much more easily controlled and you wouldn't need the comment

% under plain tex, tikz (pgf) should be loaded before luaotfload.

as pgf would just make the definitions at the right time. and other packages could (perhaps?)
add to that token register when they want to affect ExtGState

I'm sorry but I think the revised suggestion in the new PR conflicts the situation even more as it pushes the code into the lua so makes it impossible to load luaotfload (which is basically essential for any font support as you know:-) without injecting pgf specific tests and pending TeX code in \AtBeginDocument, and altering the value of count255.

But I don't intend to argue about it, sometimes you have to make things work even if architecturally you are working at the wrong level. When I saw this code had been added after I looked at the github sources rather than texlive 2015, while working on testing pull/281 I felt I should flag an issue. But that's done and I'd urge you to consider attacking the problem by just supplying a generic mechanism that pgf could use (then getting pgf to use it) but if that doesn't seem feasible to you then so be it, if you close the issue with no action I won't grumble too much:-)

[dohyunkim]

OK. I understand. Thanks for the point.
Pushed a new commit to the same PR.
I am just waiting for @phi-gamma to merge it.

[phi-gamma]

···<date: 2015-07-29, Wednesday>···<from: Dohyun Kim>···

OK. I understand. Thanks for the point.
Pushed a new commit to the same PR.
I am just waiting for @phi-gamma to merge it.

If you both agree that it’s the way to go, then it’ll be merged
in no time. Thanks very much to you both for caring about the
issue!

[phi-gamma]

@doyhunkim your latest pull request is part of master as
of 15d337d

@davidcarlisle

If you add pgf specific code today then any other package doing similar or even identical things will expect similar support at this level.

We have some code that used to be part of Fontspec
in the auxiliary library
already. Some assumptions of unicode math are taken
care of there as well (adding certain dimensions). If the
modification is only relevant for a single package we
might provide hooks to inject the code. I agree it’s not
clean to fuzz package dependencies like this but if it’s
more convenient to the packages, then I wouldn’t object
to further additions.

How is the issue going to be resolved eventually? On the
PGF end? (I’m out of the loop, Latex-wise, so sorry that
you have to spell everything out for me ;))

[davidcarlisle]

How is the issue going to be resolved eventually? On the
PGF end? (I’m out of the loop, Latex-wise, so sorry that
you have to spell everything out for me ;))

I'm happy to defer the actual code here to @dohyunkim as the details of PDF back end support for transparency are certainly not my area. I intentionally raised this as an issue to raise an architectural concern about package specific code at this level, rather than raising a pull request to suggest specific code changes.

If something does need to go into another (la)tex package then I'm happy to help apply pressure to the maintainers of that package. If in the end it does need to come here, then that's the way it is, but I think it would be better to be in an auxiliary package (or lua file) that is included rather than embedded into the main loading code. To make its status more clear and to allow alternative loading paths that do not use it.

I hope that helps (although I fear it doesn't much)

[josephwright]

The people behind pgf are pretty active so a change there should be possible. If something is needed in beamer I can make that change.

[phi-gamma]

Since @dohyunkim’s PR is in the master this should no longer be an issue.