LaTeXStrings repr with mixed text/math

[zmoon]

The current approach (7b5dd1d), stripping $ from beginning and end before passing to Markdown's LaTeX, does not work in the case of mixed text/math-mode, which LaTeXStrings does support.

Since it ends up getting rendered with MathJax, one solution would be to wrap the text (non-math) parts with \text{} before passing to LaTeX. If that seems like a good idea, I would be interested in trying to implement it.

[fonsp]

Aha - is that why the package exists? I'm always confused why we don't just create a macro L"..." that goes to Markdown.LaTeX. With your solution, you could have that feature and still just use Markdown.LaTeX.

Anyway, yes your solution sounds good - you should play around with it inside a Pluto notebook, instead of modifying the Pluto source and rebooting everything while you work on it. Let me know if you need help with that

[fonsp]

I guess the other reason why LaTeXStrings exists is that LaTeXString <: AbstractString? It's a shame that there need to be two different types for the same thing

[fonsp]

Also why can I not do this?

equation = L"\sqrt{100}"
md"I am $(equation) years old."

This would not be a problem if they used the same underlying type

[fonsp]

Sorry, all of that is not really relevant here

[zmoon]

Also why can I not do this?

equation = L"\sqrt{100}"
md"I am $(equation) years old."

I tried this in Pluto and seemed to work fine?

[zmoon]

Here is what I came up with so far...

begin

    # I'm sure there is a better way to do this...
    # e.g., `length(findall(...` ? with positive look-behind regex
    # or a comprehension
    function count_char_nonescaped(s::AbstractString, c)
        count = 0
        for i = 1:length(s)
            if i == 1
                if s[i] == c
                    count += 1
                end
            else
                if (s[i] == c) & (s[i-1] != '\\')
                    count += 1
                end		
            end
        end
        return count
    end
    
    function prep_for_md(s::LaTeXString)
        s = repr(mime_latex, s)
        
        # math-only case (allowing inner escaped $)
        if (s[1] == '$') & (s[end] == '$') & (count_char_nonescaped(s, '$') == 2)
            #strip(s, '$')  # < would remove leading/trailing escaped $ as well
            return s[begin+1:end-1]
        
        # mixed math/text
        else			
            # we need to find the indices of the pairs of $ that set off math text
            
            # this is an easy way to find $'s
            # not differentiating between latex-escaped or not tho
            # idollar = findall(isequal('$'), s)
            
            # we need to look left and make sure no `\` there
            imath = [i for i = 2:length(s) if (s[i] == '$') & (s[i-1] != '\\')]
            
            @assert length(imath) % 2 == 0
            
            # build new string
            # string substitution/replacement could be better?
            snew = ""
            for i in 1:2:length(imath)
                bt = i == 1 ? 1 : imath[i-1] + 1
                et = imath[i] - 1
                bm = imath[i] + 1  # don't include $
                em = imath[i+1] - 1
                snew *= "\\text{$(s[bt:et])}$(s[bm:em])"
            end
            if imath[end] != length(s)
                snew *= "\\text{$(s[imath[end]+1:end])}"
            end
                        
            return snew
        end
    end
    
end

I tested it just within Pluto environment like you suggested. Example:

[fonsp]

It's great that it works, but that is way too complex - can you do it in <10 lines?

[fonsp]

We can do it in 3 lines!

https://gist.github.com/fonsp/769c9d2c1dce6df0ea7db1ac173c39c8

We just wrap everything in a \text{} block, and then the dollar signs will be used by latex to create math inside the text block

[fonsp]

Still needs to be adapted to the show(io, MIME"text/latex"(), x) mechanism in https://github.com/fonsp/Pluto.jl/blob/master/src/runner/PlutoRunner.jl but I can do that if you get stuck - let me know!

[zmoon]

This is a neat idea, but it doesn't seem to work for my longer example

Maybe MathJax can't handle it?

[zmoon]

Here is a shorter regex version of mine.

function prep_for_md_2(s::LaTeXString)
    s = String(s)  # need String to use regex
    
    # determine indices of non-escaped $'s
    imath = findall(r"(?<!\\)\$", s)  # returns UnitRange of singleton Int64s
    
    # not sure the best way to collect but this works
    imath = [el[1] for el in imath]
    
    # math-only case (allowing inner escaped $)
    if (s[1] == '$') & (s[end] == '$') & (length(imath) == 2)
        return s[begin+1:end-1]  # `strip(s, '$')` would remove leading/trailing escaped $ as well
        
    # mixed math/text case
    else
        # just a check for now
        @assert iseven(length(imath))
        
        # build new string
        # string substitution/replacement could be better?
        snew = ""
        for i in 1:2:length(imath)
            bt = i == 1 ? 1 : imath[i-1] + 1
            et = imath[i] - 1
            bm = imath[i] + 1  # don't include $
            em = imath[i+1] - 1
            snew *= "\\text{$(s[bt:et])}$(s[bm:em])"
        end
        if imath[end][1] != length(s)
            snew *= "\\text{$(s[imath[end]+1:end])}"
        end
        
    end
end

Edit:

This could be used to do get the array of indices in one line:

imath = [m.offset for m in eachmatch(r"(?<!\\)\$", s)]

[fonsp]

I appreciate your careful attention, but let's go with the buggy one-liner.

[fonsp]

Does the bug also happen without the \$ at the end in your test case? Maybe we just need to add some escaping code to it?

[zmoon]

Maybe MathJax can't handle it?

Removing the \$'s did not fix it (still weirdness with things overlapping). But I discovered that after changing the MathJax renderer from SVG to CHTML it rendered correctly.

Additionally, in a clean HTML file,

<!DOCTYPE html>
<html>
<head>
<title>MathJax TeX Test Page</title>
<script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
<script type="text/javascript" id="MathJax-script" async
  src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-svg-full.js">
</script>
</head>
<body>

$$
\text{
I am $\sqrt{100}$ years old and have \$50 and $e^{i \pi} = -1$. These are more literal dollar signs:\$\$
}
$$

</body>
</html>

it happens with the tex-svg-full MathJax config that Pluto uses but not tex-svg (renders fine).

let's go with the buggy one-liner

definitely more elegant/convenient!

[fonsp]

Thanks for the experiment with HTML! What is the difference between tex-svg-full and tex-svg? Should we switch?

[zmoon]

The main difference seems to be that the -full one loads more packages/extensions by default. So in that sense not using -full could be better for Pluto, and extra extensions would still be loaded automatically upon need. But as for why switching between them would break the SVG rendering for my long test case idk. I will ask them.

[fonsp]

Ah I remember - we want to load all extensions, see the discussion here:
#101 (comment)

But I wouldn't worry too much about that test case - that particular use of LaTeXStrings might never happen again

[zmoon]

I did some more testing, even the simple example from the LaTeXStrings readme (an equation: $1 + \alpha^2$) doesn't work. Seems like anything more than one term in a math group messes it up. And I'm curious anyway, so I opened an issue: mathjax/MathJax#2524

Edit: seems that MathJax bug will be fixed in next release 👍

[fonsp]

Reopened, see #1164