BACKPORT: LaTeX and HTML show (#481)

* Show methods for HTML and LaTeX (#480)

* refactor markdown show methods

* show methods for latex and html

* formatting

* update mime docs

* escape sigma and chi square for latex

* document escape behavior

* xelatex

* more escaping

* news entry

* truncate to avoid exponent formatting in tests

* more exponent formatting

* tiny platform differences

* osx you are a pain

(cherry picked from commit 160783d)

* minor version bump

* news refresh

* merging is hard

* documenter os version

NEWS.md

@@ -1,3 +1,10 @@
+MixedModels v3.3.0 Release Notes
+========================
+* HTML and LaTeX `show` methods for `MixedModel`, `BlockDescription`,
+  `LikelihoodRatioTest`, `OptSummary` and `VarCorr`. Note that the interface for
+  these is not yet completely stable. In particular, rounding behavior may
+  change. [#480]
+
 MixedModels v3.2.0 Release Notes
 ========================
 * Markdown `show` methods for `MixedModel`, `BlockDescription`,
@@ -137,3 +144,4 @@ Package dependencies
 [#447]: https://github.com/JuliaStats/MixedModels.jl/issues/447
 [#449]: https://github.com/JuliaStats/MixedModels.jl/issues/449
 [#474]: https://github.com/JuliaStats/MixedModels.jl/issues/474
+[#480]: https://github.com/JuliaStats/MixedModels.jl/issues/480

Project.toml

@@ -1,7 +1,7 @@
 name = "MixedModels"
 uuid = "ff71e718-51f3-5ec2-a782-8ffcbfa3c316"
 author = ["Phillip Alday <me@phillipalday.com>", "Douglas Bates <dmbates@gmail.com>", "Jose Bayoan Santiago Calderon <jbs3hp@virginia.edu>"]
-version = "3.2.1"
+version = "3.3.0"
 
 [deps]
 Arrow = "69666777-d1a9-59fb-9406-91d4454c9d45"

docs/src/mime.md

@@ -6,7 +6,8 @@ For example, DataFrames are converted into nice HTML tables.
 In MixedModels, we recently (v3.2.0) introduced limited support for such pretty printing.
 (For more details on how the print and display system in Julia works, check out [this NextJournal post](https://nextjournal.com/sdanisch/julias-display-system).)
 
-In particular, we have defined Markdown output, i.e. `show` methods, for our types, which can be easily translated into HTML, LaTeX or even a MS Word Document using tools such as [pandoc](https://pandoc.org/).
+In particular, we have defined Markdown, HTML and LaTeX output, i.e. `show` methods, for our types.
+Note that the Markdown output can also be easily and more flexibly translated into HTML, LaTeX (e.g. with `booktabs`) or even a MS Word Document using tools such as [pandoc](https://pandoc.org/).
 Packages like `IJulia` and `Documenter` can often detect the presence of these display options and use them automatically.
 
 
@@ -15,7 +16,12 @@ using MixedModels
 form = @formula(rt_trunc ~ 1 + spkr * prec * load +
                           (1 + load | item) +
                           (1 + spkr + prec + load | subj))
-kbm = fit(MixedModel, form, MixedModels.dataset(:kb07))
+contr = Dict(:spkr => EffectsCoding(),
+             :prec => EffectsCoding(),
+             :load => EffectsCoding(),
+             :item => Grouping(),
+             :subj => Grouping())
+kbm = fit(MixedModel, form, MixedModels.dataset(:kb07); contrasts=contr)
 ```
 
 Note that the display here is more succinct than the standard REPL display:
@@ -25,8 +31,8 @@ using DisplayAs
 kbm |> DisplayAs.Text
 ```
 
-This brevity is intentional: we wanted these types work well with traditional academic publishing constraints on tables.
-The summary for a model fit presented in the REPL does not mesh well with being treated a single table (with columns shared between the random and fixed effects).
+This brevity is intentional: we wanted these types to work well with traditional academic publishing constraints on tables.
+The summary for a model fit presented in the REPL does not mesh well with being treated as a single table (with columns shared between the random and fixed effects).
 In our experience, this leads to difficulties in typesetting the resulting tables.
 We nonetheless encourage users to report fit statistics such as the log likelihood or AIC as part of the caption of their table.
 If the correlation parameters in the random effects are of interest, then [`VarCorr`](@ref) can also be pretty printed:
@@ -51,19 +57,42 @@ m1 = fit(MixedModel, @formula(reaction ~ 1 + days + (1+days|subj)), MixedModels.
 MixedModels.likelihoodratiotest(m0,m1)
 ```
 
-To explicitly invoke this behavior, we must specify the right `show` method:
+To explicitly invoke this behavior, we must specify the right `show` method.
+(The raw and not rendered output is intentionally shown here.)
 ```julia
 show(MIME("text/markdown"), m1)
 ```
 ```@example Main
 println(sprint(show, MIME("text/markdown"), kbm)) # hide
 ```
-(The raw and not rendered output is intentionally shown here.)
-
-In the future, we may directly support HTML and LaTeX as MIME types.
+```julia
+show(MIME("text/html"), m1)
+```
+```@example Main
+println(sprint(show, MIME("text/html"), kbm)) # hide
+```
+Note for that LaTeX, the column labels for the random effects are slightly changed: σ is placed into math mode and escaped and the grouping variable is turned into a subscript.
+Similarly for the likelihood ratio test, the χ² is escaped into math mode.
+This transformation improves pdfLaTeX and journal compatibility, but also means that XeLaTeX and LuaTeX may use a different font at this point.
+```julia
+show(MIME("text/latex"), m1)
+```
+```@example Main
+println(sprint(show, MIME("text/latex"), kbm)) # hide
+```
+This escaping behavior can be disabled by specifying `"text/xelatex"` as the MIME type.
+(Note that other symbols may still be escaped, as the internal conversion uses the `Markdown` module from the standard library, which performs some escaping on its own.)
+```julia
+show(MIME("text/xelatex"), m1)
+```
+```@example Main
+println(sprint(show, MIME("text/xelatex"), kbm)) # hide
+```
 
 This output can also be written directly to file:
 
 ```julia
-show(open("model.md", "w"), MIME("text/markdown"), kbm)
+open("model.md", "w") do io
+    show(io, MIME("text/markdown"), kbm)
+end
 ```

docs/src/rankdeficiency.md

@@ -30,8 +30,7 @@ As such, the handling of rank deficiency in `MixedModels.jl` should not be taken
 
 There is a widely accepted convention for how to make the coefficient estimates for these redundant columns well-defined: we set their value to zero and their standard errors to `NaN` (and thus also their $z$ and $p$-values).
 The values that have been defined to be zero, as opposed to evaluating to zero, are displayed as `-0.0` as an additional visual aid to distinguish them from the other coefficients.
-In practice the determination of rank and the redundant coefficients is done via a 'pivoting' scheme during a decomposition to
-move the surplus columns to the right side of the model matrix.
+In practice the determination of rank and the redundant coefficients is done via a 'pivoting' scheme during a decomposition to move the surplus columns to the right side of the model matrix.
 In subsequent calculations, these columns are effectively ignored (as their estimates are zero and thus won't contribute to any other computations).
 For display purposes, this pivoting is unwound when the `coef` values are displayed.
 

src/MixedModels.jl

@@ -174,6 +174,6 @@ include("simulate.jl")
 include("bootstrap.jl")
 include("blockdescription.jl")
 include("grouping.jl")
-include("mdshow.jl")
+include("mimeshow.jl")
 
 end # module

src/mdshow.jl → src/mimeshow.jl

@@ -1,28 +1,68 @@
 # for this type of union, the compiler will actually generate the necessary methods
 # but it's also type stable either way
 _MdTypes = Union{BlockDescription, LikelihoodRatioTest, OptSummary, VarCorr, MixedModel}
-Base.show(mime::MIME, x::_MdTypes) = Base.show(Base.stdout, mime, x)
+Base.show(mime::MIME, x::_MdTypes) = show(Base.stdout, mime, x)
+
+Base.show(io::IO, ::MIME"text/markdown", x::_MdTypes) = show(io, Markdown.MD(_markdown(x)))
+# let's not discuss why we need show above and println below,
+# nor what happens if we try display instead :)
+Base.show(io::IO, ::MIME"text/html", x::_MdTypes) = println(io, Markdown.html(_markdown(x)))
+# print and println because Julia already adds a newline line
+Base.show(io::IO, ::MIME"text/latex", x::_MdTypes) = print(io, Markdown.latex(_markdown(x)))
+Base.show(io::IO, ::MIME"text/xelatex", x::_MdTypes) = print(io, Markdown.latex(_markdown(x)))
+
+# not sure why this escaping doesn't work automatically
+# FIXME: find out a way to get the stdlib to do this
+function Base.show(io::IO, ::MIME"text/html", x::OptSummary)
+    out = Markdown.html(_markdown(x))
+    out = replace(out, r"&#96;([^[:space:]]*)&#96;" => s"<code>\1</code>")
+    out = replace(out, r"\*\*(.*?)\*\*" => s"<b>\1</b>")
+    println(io, out)
+end
+
+function Base.show(io::IO, ::MIME"text/latex", x::OptSummary)
+    out = Markdown.latex(_markdown(x))
+    out = replace(out, r"`([^[:space:]]*)`" => s"\\texttt{\1}")
+    out = replace(out, r"\*\*(.*?)\*\*" => s"\\textbf{\1}")
+    print(io, out)
+end
+
+function Base.show(io::IO, ::MIME"text/latex", x::MixedModel)
+    la = Markdown.latex(_markdown(x))
+    # take advantage of subscripting
+    # including preceding & prevents capturing coefficients
+    la = replace(la, r"& σ\\_([[:alnum:]]*) " => s"& $\\sigma_\\text{\1}$ ")
+    print(io, la)
+end
 
+function Base.show(io::IO, ::MIME"text/latex", x::LikelihoodRatioTest)
+    la = Markdown.latex(_markdown(x))
+    # take advantage of subscripting
+    # including preceding & prevents capturing coefficients
+    la = replace(la, r"χ²" => s"$\\chi^2$")
+    print(io, la)
+end
 
-function Base.show(io::IO, ::MIME"text/markdown", b::BlockDescription)
-    rowwidth = max(maximum(ndigits, b.blkrows) + 1, 5)
-    colwidth = max(maximum(textwidth, b.blknms) + 1, 14)
+function _markdown(b::BlockDescription)
     ncols = length(b.blknms)
-    print(io, "|", rpad("rows", rowwidth), "|")
-    println(io, ("$(cpad(bn, colwidth))|" for bn in b.blknms)...)
-    print(io, "|", rpad(":", rowwidth, "-"), "|")
-    println(io, (":$("-"^(colwidth-2)):|" for _ in b.blknms)...)
+    align = repeat([:l], ncols+1)
+    newrow = ["rows"; [bn for bn in b.blknms] ]
+    rows = [newrow]
+
     for (i, r) in enumerate(b.blkrows)
-        print(io, "|$(rpad(string(r), rowwidth))|")
+       newrow =  [string(r)]
         for j in 1:i
-            print(io, "$(rpad(b.ALtypes[i, j],colwidth))|")
+            push!(newrow, "$(b.ALtypes[i, j])")
         end
-        i < ncols && print(io, "$(" "^colwidth)|"^(ncols-i))
-        println(io)
+        i < ncols && append!(newrow, repeat([""], ncols-i))
+        push!(rows, newrow)
     end
+
+    tbl = Markdown.Table(rows, align)
+    return tbl
 end
 
-function Base.show(io::IO, ::MIME"text/markdown", lrt::LikelihoodRatioTest)
+function _markdown( lrt::LikelihoodRatioTest)
     Δdf = lrt.tests.dofdiff
     Δdev = lrt.tests.deviancediff
 
@@ -52,16 +92,13 @@ function Base.show(io::IO, ::MIME"text/markdown", lrt::LikelihoodRatioTest)
     end
 
     tbl = Markdown.Table(outrows, [:l, :r, :r, :r, :r, :l])
-
-    show(io, Markdown.MD(tbl))
+    return tbl
 end
 
-
-
 _dname(::GeneralizedLinearMixedModel) = "Dispersion"
 _dname(::LinearMixedModel) = "Residual"
 
-function Base.show(io::IO, ::MIME"text/markdown", m::MixedModel)
+function _markdown(m::MixedModel)
     if m.optsum.feval < 0
         @warn("Model has not been fit: results will be nonsense")
     end
@@ -80,7 +117,7 @@ function Base.show(io::IO, ::MIME"text/markdown", m::MixedModel)
     σwidth = _printdigits(σvec)
 
     newrow = ["", "Est.", "SE", "z", "p"]
-    align = [:l, :l, :r, :r, :r]
+    align = [:l, :r, :r, :r, :r]
 
     for rr in fnames(m)
         push!(newrow,"σ_$(rr)")
@@ -113,11 +150,10 @@ function Base.show(io::IO, ::MIME"text/markdown", m::MixedModel)
     end
 
     tbl = Markdown.Table(rows, align)
-    show(io, Markdown.MD(tbl))
+    return tbl
 end
 
-
-function Base.show(io::IO, ::MIME"text/markdown", s::OptSummary)
+function _markdown(s::OptSummary)
     rows = [["", ""],
 
             ["**Initialization**", ""],
@@ -126,7 +162,7 @@ function Base.show(io::IO, ::MIME"text/markdown", s::OptSummary)
 
             ["**Optimizer settings** ", ""],
             ["Optimizer (from NLopt)", "`$(s.optimizer)`"],
-            ["`Lower bounds`", string(s.lowerbd)],
+            ["Lower bounds", string(s.lowerbd)],
             ["`ftol_rel`", string(s.ftol_rel)],
             ["`ftol_abs`", string(s.ftol_abs)],
             ["`xtol_rel`", string(s.xtol_rel)],
@@ -141,10 +177,10 @@ function Base.show(io::IO, ::MIME"text/markdown", s::OptSummary)
             ["Return code", "`$(s.returnvalue)`"]]
 
     tbl = Markdown.Table(rows, [:l, :l])
-    show(io, Markdown.MD(tbl))
+    return tbl
 end
 
-function Base.show(io::IO, ::MIME"text/markdown", vc::VarCorr)
+function _markdown(vc::VarCorr)
     σρ = vc.σρ
     nmvec = string.([keys(σρ)...])
     cnmvec = string.(foldl(vcat, [keys(sig)...] for sig in getproperty.(values(σρ), :σ)))
@@ -202,6 +238,7 @@ function Base.show(io::IO, ::MIME"text/markdown", vc::VarCorr)
         append!(rr, repeat([" "], rowlen-length(rr)))
     end
     append!(align, repeat([:r], rowlen-length(align)))
+
     tbl = Markdown.Table(rows, align)
-    show(io, Markdown.MD(tbl))
+    return tbl
 end