Position tracking is broken in XeTeX

[pgf-tikz-bot]

Migrated from SourceForge
Author: st-mue
Timestamp: 2015-02-25 09:38:35.377000

I run into a problem with tikzmark and the professional help on the xetex mail list and on stackexchange (Ulrike Fischer) identified it as a pgf problem. Please find some description and minimal examples here:

http://tex.stackexchange.com/questions/229500/tikzmark-and-xelatex

[pgf-tikz-bot]

Migrated from SourceForge
Author: st-mue
Timestamp: 2015-03-04 11:36:41.868000

For convenience, I attach the code here:

The following code does not work when compiled with XeLaTeX but produces the expected result when compiled with PDFLaTeX:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{tikzmark}

\begin{document}

x\pgfmark{tA}some text \pgfmark{tB} some text
\begin{tikzpicture}[remember picture]
\draw (0,0)node (A){A} rectangle (1,1)node (B){B};
\end{tikzpicture}
\begin{tikzpicture}[remember picture]
\draw (0,0)node {\pgfmark{nA}} rectangle (1,1)node {\pgfmark{nB}};
\end{tikzpicture}

\vspace{3cm}\centering
\begin{tikzpicture}[overlay,remember picture]
\draw[red,->] (0,0)--(pic cs:tA) (0,0)--(pic cs:tB);
\draw\[blue,->](0,0)--(pic cs:nA) (0,0)--(pic cs:nB); %nB faulty
\draw\[green,->](0,0)--(A) (0,0)--(B);
\end{tikzpicture}

\end{document} 

[pgf-tikz-bot]

Migrated from SourceForge
Author: talopnahli
Timestamp: 2015-05-23 22:17:56.372000

Hi Stefan

tikzmark library is not actually an official library of TikZ but it is written by Andrew Stacey and he commented under your TeX.SX question under the nickname Loop Space what the problem is and how you can resolve.

Maybe you can contact him for further elaborations?

[pgf-tikz-bot]

Migrated from SourceForge
Author: st-mue
Timestamp: 2015-05-24 12:08:43.044000

Hi,

I checked the Tex.SX post but did not find anything. I can contact him directly. Thanks for the hint!

Best

Stefan

[pgf-tikz-bot]

Migrated from SourceForge
Author: cfr42
Timestamp: 2016-11-20 00:05:13.858000

The example given does not demonstrate the problem. Here's an example which uses tikzmark correctly (i.e. avoids nesting tikzpictures) by using \subnode within a node. With pdfTeX or LuaTeX, for example, this compiles correctly. With XeTeX, it does not.

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{tikzmark}
\begin{document}
\begin{tikzpicture}
  \node {An \subnode{p}{apple} a day makes all horses neigh.};
\end{tikzpicture}

\begin{tikzpicture}[remember picture, overlay]
    \draw [red] (p.north east) rectangle (p.south west);
\end{tikzpicture}
\end{document}

The problem appears to lie in the backend driver provided by PGF. The following code corrects the problem (from Jipí's answer at http://tex.stackexchange.com/a/339975/).

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{tikzmark}
% WORKAROUND:
% Definition copied from pgfsys-common-pdf-via-dvi.def
% http://tex.stackexchange.com/a/339975/
% Compare http://tex.stackexchange.com/q/229500 and comments!
\makeatletter
\def\pgfsys@hboxsynced#1{%
  {%
    \pgfsys@beginscope%
    \setbox\pgf@hbox=\hbox{%
      \hskip\pgf@pt@x%
      \raise\pgf@pt@y\hbox{%
        \pgf@pt@x=0pt%
        \pgf@pt@y=0pt%
        \special{pdf: content q}%
        \pgflowlevelsynccm%
        \pgfsys@invoke{q -1 0 0 -1 0 0 cm}%
        \special{pdf: content -1 0 0 -1 0 0 cm q}% translate to original coordinate system
        \pgfsys@invoke{0 J [] 0 d}% reset line cap and dash
        \wd#1=0pt%
        \ht#1=0pt%
        \dp#1=0pt%
        \box#1%
        \pgfsys@invoke{n Q Q Q}%
      }%
      \hss%
    }%
    \wd\pgf@hbox=0pt%
    \ht\pgf@hbox=0pt%
    \dp\pgf@hbox=0pt%
    \pgfsys@hbox\pgf@hbox%
    \pgfsys@endscope%
  }%
}
\makeatother
\begin{document}
\begin{tikzpicture}
  \node {An \subnode{p}{apple} a day makes all horses neigh.};
\end{tikzpicture}

\begin{tikzpicture}[remember picture, overlay]
    \draw [red] (p.north east) rectangle (p.south west);
\end{tikzpicture}
\end{document}

Examining the definition of this macro before it is redefined, it seems that the problem lies with its definition in pgfsys-dvipdfmx.def:

\def\pgfsys@hboxsynced#1{%
  \pgfsys@beginscope%
    \setbox#1=\hbox{\box#1}%
    \wd#1=0pt%
    \ht#1=0pt%
    \dp#1=0pt%
    \pgfsys@dvipdfmx@suspendcontent%
    \pgfsys@invoke{0 J [] 0 d}% reset line cap and dash
    \pgfsys@dvipdfmx@start@force@reset@color%
    \pgf@sys@bp@correct\pgf@pt@x%
    \pgf@sys@bp@correct\pgf@pt@y%
    \special{pdf:btrans matrix \pgf@pt@aa\space \pgf@pt@ab\space \pgf@pt@ba\space \pgf@pt@bb\space 
      \pgf@sys@tonumber{\pgf@pt@x} \pgf@sys@tonumber{\pgf@pt@y}}%
    \box#1%
    \special{pdf:etrans}%
    \pgfsys@dvipdfmx@stop@force@reset@color%
    \pgfsys@dvipdfmx@unsuspendcontent%
  \pgfsys@endscope%
}

I don't understand this code at all, but according to comments on TeX SE, I gather that this is preventing positions within a node from being saved -- the position is always saved as 0,0. However, I may have misunderstood the discussion there. Even so, overwriting this definition with the one from pgfsys-common-pdf-via-dvi.def certainly makes the code work with XeTeX as it does with other engines.

[pgf-tikz-bot]

Migrated from SourceForge
Author: mo-gul
Timestamp: 2018-12-23 18:26:21.001000

• labels: --> xelatex, tikzmark
• summary: tixmark has a problem with pgf --> [tixmark] has a problem with pgf

[kopeckyf]

Dear all,
I was wondering whether there was any progress on this issue. As of TikZ 3.14b, it seems to be unsolved. The issue is particularly relevant to users in linguistics, since they often use XeTeX for Unicode support, and sometimes need to reference parts of elements that use code which build on top of TikZ, such as forest trees.

[Rmano]

Hi, there has been another report today, it still seems broken... https://tex.stackexchange.com/questions/690413/using-subnode-in-a-tikzpicture-with-xelatex

[muzimuzhi]

The previous fix d0f6660 was reverted in 943a0a0 due to regression

• inclusion of scaled graphics in a tikzpicture with xelatex broken in the newest pgf #708.

I tried the \pgfsys@hboxsynced from pgfsys-dvipdfm.def (without the trailing x in backend name), surprisingly it ... works for both tikzmark examples and the example in #708.

Note I don't understand the two \pgfsys@hboxsynced implementations and their differences (yet).

Output comparison and full example

Engine and driver	Output
xelatex, \pgfsys@hboxsynced from pgfsys-dvipdfmx.def
xelatex, \pgfsys@hboxsynced from pgfsys-dvipdfm.def
pdflatex

% !TeX TS-program = xelatex
\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{tikzmark}

\makeatletter
\ifdefined\XeTeXrevision
  % pgfsys-dvipdfm.def
  \def\pgfsys@hboxsynced#1{%
    {%
      \pgfsys@begin@idscope%
      \pgfsys@beginscope%
      \setbox\pgf@hbox=\hbox{%
        \hskip\pgf@pt@x%
        \raise\pgf@pt@y\hbox{%
          \pgf@pt@x=0pt%
          \pgf@pt@y=0pt%
          \special{pdf: content q}%
          \pgflowlevelsynccm%
          \pgfsys@invoke{q -1 0 0 -1 0 0 cm}%
          \special{pdf: content -1 0 0 -1 0 0 cm q}% translate to original coordinate system
          \pgfsys@invoke{0 J [] 0 d}% reset line cap and dash
          \wd#1=0pt%
          \ht#1=0pt%
          \dp#1=0pt%
          \box#1%
          \pgfsys@invoke{n Q Q Q}%
        }%
        \hss%
      }%
      \wd\pgf@hbox=0pt%
      \ht\pgf@hbox=0pt%
      \dp\pgf@hbox=0pt%
      \pgfsys@hbox\pgf@hbox%
      \pgfsys@endscope%
      \pgfsys@end@idscope%
    }%
  }
\fi
\makeatother

\begin{document}
% https://tex.stackexchange.com/q/690413
\begin{tikzpicture}[remember picture]
  \node[draw] {A node with a \subnode{sn}{subnode}}; 
\end{tikzpicture} 
\begin{tikzpicture}[remember picture, overlay]   
  \draw [red,<-] (pic cs:sn) -- ++(0pt,25pt);
\end{tikzpicture}

% https://tex.stackexchange.com/q/229500
x\pgfmark{tA}some text \pgfmark{tB} some text
\begin{tikzpicture}[remember picture]
  \draw (0,0)node (A){A} rectangle (1,1)node (B){B};
\end{tikzpicture}
\begin{tikzpicture}[remember picture]
  \draw (0,0)node {\pgfmark{nA}} rectangle (1,1)node {\pgfmark{nB}};
\end{tikzpicture}

\vspace{3cm}\centering
\begin{tikzpicture}[overlay,remember picture]
  \draw[red,->] (0,0)--(pic cs:tA) (0,0)--(pic cs:tB);
  \draw[blue,->](0,0)--(pic cs:nA) (0,0)--(pic cs:nB); %nB faulty
  \draw[green,->](0,0)--(A) (0,0)--(B);
\end{tikzpicture}

% https://github.com/pgf-tikz/pgf/issues/708
\newsavebox\mybox
\sbox\mybox{\includegraphics[scale=0.5]{example-image.pdf}}
\fbox{\usebox\mybox}
%
\begin{tikzpicture}
 \node{\fbox{\usebox\mybox}};
\end{tikzpicture}

\end{document}

[hmenke]

dvips and XeTeX use a backend that is separate from the TeX engine, so when we move the point using PDF literals, the point where the engine inserts boxes is not moved. Some hacking with regards to transformations is also required. See how l3backend does it:

https://github.com/latex3/latex3/blob/170ab28ab51f2c908372682f15949f372dd82a08/l3backend/l3backend-draw.dtx#L349-L386

I'd really like to have another go at this, but currently I'm still blocked by the testsuite.

[muzimuzhi]

See how l3backend does it:

https://github.com/latex3/latex3/blob/170ab28ab51f2c908372682f15949f372dd82a08/l3backend/l3backend-draw.dtx#L349-L386

@hmenke What you linked is the l3backend implementation for dvips backend. For dvipdfmx and xetex backend, it's simpler

\cs_new_protected:Npn \__draw_backend_box_use:Nnnnn #1#2#3#4#5
  {
    \__kernel_backend_scope_begin:
    \__kernel_backend_literal:n
      { pdf:btrans~matrix~ #2 ~ #3 ~ #4 ~ #5 ~ 0 ~ 0 }
    \hbox_overlap_right:n { \box_use:N #1 }
    \__kernel_backend_literal:n { pdf:etrans }
    \__kernel_backend_scope_end:
  }

See relevant lines in l3backend-draw.dtx (or simply find \__draw_backend_box_use:Nnnnn in kpsewhich l3backend-xetex.def). Here \__draw_backend_box_use:Nnnnn is only used in \__draw_box_use:Nnnnn defined in l3draw-boxes.dtx.

From how l3draw and l3backend does it, it seems the \pgfsys@dvipdfmx@suspendcontent and \pgfsys@dvipdfmx@unsuspendcontent are unnecessary and actually when they are commented out in \pgfsys@hboxsynced from pgfsys-dvipdfmx.def, the example in #353 (comment) gives expected output.

PS: \pgfsys@dvipdfmx@suspendcontent and \pgfsys@dvipdfmx@unsuspendcontent were added in 529942e (fixed bug in dvisvgmx code and added pic path command, 2013-08-28), with changelog entry

2013-08-27 Till Tantau tantau@users.sourceforge.net

• Patched pgfsys-dvipdfmx.def to step around the bug in
  (x)dvipdfmx that caused scaled boxes (including scaled
  graphics) inside nodes to be displayed incorrectly.