Add `pyluatex-pysub` package for PythonTeX-like variable substitution

[kzhk75]

Hi! Thank you for developing PyLuaTeX. It is an incredibly fast and useful package.

I would like to propose an add-on package, pyluatex-pysub.sty, which introduces a pythontex-like syntax (!{...}) for inline variable substitution.

Some users migrating from pythontex to pyluatex miss the \begin{pysub} environment, which allows for clean and readable string interpolation without explicitly writing \py{...} every time.

I believe this would make PyLuaTeX even more accessible and user-friendly, especially for those creating dynamic educational materials or exams.

Please let me know if you have any suggestions for improvement. I'd be happy to make any changes or integrate it directly into the core code if you prefer!

Key Features

1. Familiar Syntax: Introduces the pysub environment and \pys{} command. Within these, users can simply write !{var} instead of \py{var}.
2. Built-in Auto-Escaping: Added a new !e{...} syntax that automatically escapes LaTeX special characters (like _, &, %) via a Python helper function. This prevents compilation errors when dealing with arbitrary strings.
3. Nested Braces Support: The Lua backend uses %b{} for pattern matching, fully supporting nested braces inside the tag. This allows for complex Python expressions like !{ data["scores"]["math"] } without breaking the parser.

Minimal Working Example

\documentclass{article}
\usepackage{pyluatex-pysub} % please refer to `pyluatex-pysub.sty` below

\begin{pythonq}
import math
username = "user_name & co"

# A dictionary with nested structures
data = {
    "scores": {
        "math": 95, 
        "physics": 88
    }
}
\end{pythonq}

\begin{document}

\begin{pysub}
% 1. Raw evaluation for numbers/math
Pi is approx !{round(math.pi, 2)}.

% 2. Nested braces support (Dictionary/List access)
My math score is !{ data["scores"]["math"] }.

% 3. Auto-escaped strings
Username is !e{username}.
\end{pysub}

\end{document}

pyluatex-pysub.sty

\ProvidesPackage{pyluatex-pysub}[2026/02/13 PyLuaTeX add-on for PythonTeX-like variable substitution]

\RequirePackage{pyluatex}
\RequirePackage{luacode}
\RequirePackage{environ}

% ============================================================
% 1. Python Side: Helper function for LaTeX escaping
% ============================================================
\begin{pythonq}
def latex_escape(s):
    """
    Escapes LaTeX special characters in a string to prevent compilation errors.
    Useful for safely printing arbitrary Python strings in LaTeX.
    """
    if s is None: 
        return ""
    s = str(s)
    # The order of replacement matters
    s = s.replace("\\", r"\textbackslash{}")
    s = s.replace("_", r"\_")
    s = s.replace("&", r"\&")
    s = s.replace("%", r"\%")
    s = s.replace("$", r"\$")
    s = s.replace("#", r"\#")
    s = s.replace("{", r"\{")
    s = s.replace("}", r"\}")
    s = s.replace("^", r"\textasciicircum{}")
    s = s.replace("~", r"\textasciitilde{}")
    return s
\end{pythonq}

% ============================================================
% 2. Lua Side: Substitution engine supporting nested braces
% ============================================================
\begin{luacode*}
function process_pysub(content)
    -- First, process !e{...} : Escaped output (safe for strings with special characters)
    -- Using %b{} ensures that nested braces (e.g., !e{ data["key"] }) are handled correctly.
    local result = string.gsub(content, "!e(%b{})", function(s)
        local inner = string.sub(s, 2, -2) -- Remove the outermost braces
        return "\\py{latex_escape(" .. inner .. ")}"
    end)
    
    -- Next, process !{...} : Raw output (for numbers, SymPy LaTeX output, etc.)
    result = string.gsub(result, "!(%b{})", function(s)
        local inner = string.sub(s, 2, -2)
        return "\\py{" .. inner .. "}"
    end)
    
    tex.print(result)
end
\end{luacode*}

% ============================================================
% 3. LaTeX Side: Environments and Commands
% ============================================================

% The pysub environment allows multi-line substitution
\NewEnviron{pysub}{%
    \directlua{process_pysub(\luastringO{\BODY})}%
}

% The \pys command allows inline substitution
\newcommand{\pys}[1]{%
    \directlua{process_pysub(\luastringO{#1})}%
}

[kzhk75]

I've spent a bit more time refining my initial implementation to make it more robust and feature-rich for practical use.

Here are the main updates in this revised version:

• Error Handling (safe_process): I wrapped the Lua string.gsub calls in a pcall. Now, if a substitution pattern fails, it gracefully throws a \PackageWarning instead of crashing the entire LaTeX build.
• New !t{...} tag: Added a dedicated tag and Python function (latex_escape_tt) for safely escaping and wrapping text in \texttt{}.
• Quality of Life: Added short Python aliases (esc, esctt) and a \pysubhelp command for quick reference.

I think this version is much safer and more convenient for general use. Let me know what you think!

Here is a Minimal Working Example (MWE) to demonstrate how these features work in practice:

\documentclass{article}
\usepackage{pyluatex-pysub} % See pyluatex-pysub below, not above

\begin{document}

\begin{python}
import math

# 1. Basic calculation
pi_val = math.pi

# 2. String containing LaTeX special characters (underscore, ampersand, percent, backslash)
danger_str = r"C:\path\to_file\name & 100% #magic"
\end{python}

\section*{Minimal Working Example}

% --- Block substitution using pysub environment ---
\begin{pysub}
\begin{itemize}
    \item \textbf{Raw output} (\texttt{!\{...\}}): \\
    The value of $\pi$ rounded to 4 decimal places is !{f"{pi_val:.4f}"}.
    
    \item \textbf{Escaped output} (\texttt{!e\{...\}}): \\
    Safe rendering of special characters: !e{danger_str}
    
    \item \textbf{Texttt output} (\texttt{!t\{...\}}): \\
    Formatted as code: !t{danger_str}
\end{itemize}
\end{pysub}

\vspace{1em}

% --- Inline substitution using \pys command ---
\noindent
We can also use the inline command \verb|\pys{...}|: \\
The square root of 2 is \pys{!{round(math.sqrt(2), 3)}}.

\end{document}

%% This package provides a PythonTeX-like variable substitution 
%% engine (!{}, !e{}, !t{}) based on PyLuaTeX.
\ProvidesPackage{pyluatex-pysub}[2026/02/13 v0.2.1 PyLuaTeX add-on for PythonTeX-like variable substitution]

\RequirePackage{pyluatex}
\RequirePackage{luacode}
\RequirePackage{environ}

% ============================================================
% 1. Python Side: Helper function for LaTeX escaping
% ============================================================
\begin{pythonq}
def latex_escape(s):
    """
    Escapes LaTeX special characters in a string to prevent compilation errors.
    Useful for safely printing arbitrary Python strings in LaTeX.
    """
    if s is None: 
        return ""
    s = str(s)
    # The order of replacement matters
    s = s.replace("\\", r"\textbackslash{}")
    s = s.replace("_", r"\_")
    s = s.replace("&", r"\&")
    s = s.replace("%", r"\%")
    s = s.replace("$", r"\$")
    s = s.replace("#", r"\#")
    s = s.replace("{", r"\{")
    s = s.replace("}", r"\}")
    s = s.replace("^", r"\textasciicircum{}")
    s = s.replace("~", r"\textasciitilde{}")
    return s

# Short alias
esc = latex_escape

def latex_escape_tt(s):
    """
    Escapes for use inside \texttt{} - minimal escaping.
    """
    if s is None:
        return ""
    s = str(s)
    s = s.replace("\\", r"\textbackslash{}")
    s = s.replace("_", r"\_")
    s = s.replace("&", r"\&")
    s = s.replace("%", r"\%")
    s = s.replace("$", r"\$")
    s = s.replace("#", r"\#")
    s = s.replace("{", r"\{")
    s = s.replace("}", r"\}")
    return s

# Short alias
esctt = latex_escape_tt
\end{pythonq}

% ============================================================
% 2. Lua Side: Substitution engine supporting nested braces
% ============================================================
\begin{luacode*}
-- Error handling wrapper
local function safe_process(content, pattern, replacement_func)
    local success, result = pcall(function()
        return string.gsub(content, pattern, replacement_func)
    end)
    if success then
        return result
    else
        -- Return original content with error marker
        tex.print("\\PackageWarning{pyluatex-pysub}{Substitution error: " .. tostring(result) .. "}")
        return content
    end
end

function process_pysub(content)
    if content == nil or content == "" then
        return
    end

    local result = content
    
    -- 1. Process !e{...} : Escaped output (safe for strings with special characters)
    result = safe_process(result, "!e(%b{})", function(s)
        local inner = string.sub(s, 2, -2) -- Remove the outermost braces
        return "\\py{latex_escape(" .. inner .. ")}"
    end)
    
    -- 2. Process !t{...} : Escaped for texttt
    result = safe_process(result, "!t(%b{})", function(s)
        local inner = string.sub(s, 2, -2)
        return "\\texttt{\\py{latex_escape_tt(" .. inner .. ")}}"
    end)
    
    -- 3. Process !{...} : Raw output (for numbers, SymPy LaTeX output, etc.)
    result = safe_process(result, "!(%b{})", function(s)
        local inner = string.sub(s, 2, -2)
        return "\\py{" .. inner .. "}"
    end)
    
    tex.print(result)
end
\end{luacode*}

% ============================================================
% 3. LaTeX Side: Environments and Commands
% ============================================================

% The pysub environment allows multi-line substitution
\NewEnviron{pysub}{%
    \directlua{process_pysub(\luastringO{\BODY})}%
}

% The \pys command allows inline substitution
\newcommand{\pys}[1]{%
    \directlua{process_pysub(\luastringO{#1})}%
}

% ============================================================
% 4. Documentation (accessible via \pysubhelp)
% ============================================================
\newcommand{\pysubhelp}{%
\begin{itemize}
\item \texttt{!\{expr\}} --- Raw output (numbers, LaTeX math)
\item \texttt{!e\{expr\}} --- Escaped output (strings with special chars)
\item \texttt{!t\{expr\}} --- Escaped + wrapped in \textbackslash texttt
\end{itemize}
}
\endinput

[tndrle]

Hi,

thank you for your suggestion. I appreciate your interest in PyLuaTeX and the work you put into this add-on. I agree that this is a usefull feature and it will help people migrate.

Some thoughts about your implementation:

• First off: I like how well-written and structured it is, also the documentation in this issue
• PyLuaTeX's API is different from that of PythonTeX. I would rename pysub to pythonsub, to be in line with the other environments: python, pythonq, pythonrepl. Of course, that makes migration harder again...
• I noticed that the \ in danger_str are typset as \{} in the PDF because s.replace("{", r"\{") follows s.replace("\\", r"\textbackslash{}") (?)
• Is pcall really necessary? What could go wrong in gsub?

I thought about how this could be integrated in the PyLuaTeX core and came up with this: 3a22793

Some comments:

• There is one new environment (pythonsub) and two new macros (pys and pyv)
• pythonsub and pys substitute !{} with \py{} and !v{} with \pyv{}
• The v means verbatim. It is similar to your e version. However, instead of actively escaping special characters, I use tex.print(-2, ...), which prints almost everything with catcode 12. So all the special characters are treated as "normal" characters (https://tex.stackexchange.com/a/302129).
• !t{} is not yet supported. It's easy to do (another content:gsub("!t(%b{})", "\\texttt{\\pyv%1}")), however, I wonder whether hard-coding \texttt is a good idea.
• Consider it a draft. It's not thoroughly tested, yet.

I could imagine extending PyLuaTeX that way. What do you think?
Of course, feel free to publish your package as an add-on, as well.

MWE for this commit:

\documentclass{article}
\usepackage{pyluatex}

\begin{document}
\begin{python}
import math

pi_val = math.pi

# String containing LaTeX special characters
danger_str = r"C:\path\to_file\name & 100% #magic"
\end{python}

\section*{Minimal Working Example}

\begin{pythonsub}
\begin{itemize}
    \item \textbf{Raw output} (\texttt{!\{...\}}): \\
    The value of $\pi$ rounded to 4 decimal places is !{f"{pi_val:.4f}"}.
    \item \textbf{Verbatim output} (\texttt{!v\{...\}}): \\
    Safe rendering of special characters: !v{
      danger_str
    }
\end{itemize}
\end{pythonsub}

We can also use the inline command \verb|\pys{...}|: \\
\pys{The square root of 2 is 
  !{round(math.sqrt(2), 3)}
}.

We can also use the new command \verb|\pyv{...}| directly for verbatim
output: \pyv{danger_str}
\end{document}

[kzhk75]

Hi, thank you so much for the detailed response and for taking the time to write up a draft commit!

I completely missed the possibility of using tex.print(-2, ...) with catcode 12. Your approach using !v{} and \pyv{} is much more elegant and robust than my Python-side string replacement. I totally agree that integrating this directly into the PyLuaTeX core is the best way forward!

Regarding your thoughts:

1. Renaming to pythonsub: Yes, keeping it consistent with the existing API is the right choice.
2. The pcall wrapper: You are right, it was unnecessary defensive programming. A direct gsub is perfectly fine.
3. About !t{}: I agree with your hesitation about hard-coding \texttt{}. Now that we have !v{...} which handles the escaping perfectly, users can simply write \texttt{!v{...}} themselves if they need monospace formatting. We probably don't need a dedicated !t{} tag, which keeps the core API simpler.

I haven't had a chance to test your sub branch locally yet, but I plan to check it out and run some tests on my end shortly.

If everything works well, I think this draft is fantastic. There is no need for me to publish a separate add-on package if this becomes a native feature. Let me know if there's anything else I can do to help with the testing or documentation for this new core feature!

[tndrle]

Hi, I thought about it a little more and realized that the purpose of pysub in PythonTeX is not only readble string interpolation but also (from the docs):

This command is useful for inserting Python-generated content in contexts
where the normal \py and \pyc would not function or would be inconvenient
due to the restrictions imposed by LaTeX.

PyLuaTeX's \py command can be used in places where PythonTeX's cannot. Still not in each and every place, though, I guess. So, it would definitely be helpful if PyLuaTeX would offer real substitution as well. The approach we discussed above is not real substitution but still requires \py. Therefore, I will think about how real substitution could be implemented and not add the sub branch to the core, right now. Sorry for the delay.

[kzhk75]

Thank you for pointing that out—I actually hadn't realised the full scope of pysub until your message prompted me to look deeper into it. Now I see exactly what you mean. The true power of PythonTeX's pysub isn't just syntactic sugar; it's the ability to inject evaluated content directly into contexts where a standard \py command might fail due to LaTeX's restrictions.

Just to visualise this architectural difference, here is a simplified look at how PythonTeX achieves this "real substitution" alongside what I understand to be your ideal target for PyLuaTeX. (I have forced a top-down layout so PythonTeX appears above PyLuaTeX).

graph LR
    subgraph PythonTeX [PythonTeX: 3-Pass Approach]
        direction LR
        Doc1[document.tex] -->|1. Bypass TeX| Py1[Python Engine]
        Py1 -->|2. Output Text| Txt1[Safe Text without \py]
        Txt1 -->|3. Compile| PDF1[Final PDF]
    end

    subgraph PyLuaTeX [PyLuaTeX Target: 1-Pass Approach]
        direction LR
        Doc2[document.tex] -->|1. Lua intercepts before TeX| Py2[Python Engine]
        Py2 -->|2. Injects evaluated value directly| PDF2[Final PDF]
    end

    %% Invisible link to force vertical layout
    Doc1 ~~~ Doc2

Loading

As you noted, if the !{} syntax in PyLuaTeX just expands to \py{}, it inherits all of \py's LaTeX-side parsing limitations, which defeats the purpose of having a dedicated substitution engine.

Since PyLuaTeX's greatest strength is its single-pass compilation, replicating this "real substitution" entirely within Lua—without falling back on \py—is a fascinating and highly complex architectural challenge.

I fully support your decision to hold off on merging the sub branch for now. Taking the time to figure out a robust substitution mechanism at the Lua level is definitely the right choice for the package's future.

Absolutely no problem about the delay! I'm really excited to see what you come up with. Let me know whenever you have a new concept or branch you'd like me to test!