"latexify does not support DataFrame" error when compiling my package

[nathanrboyer]

I wrote a package which defines some functions and constant tables. I am using Weave.jl on it from a file in the test folder to generate a PDF output. The Weave half is now working great (besides this Weave issue). However, I am getting a strange error now when I try to load my package with using KM620 on the line latexify(select(coefficients_table_forprinting, 1)).

julia> using KM620
[ Info: Precompiling KM620 [0ccbab1c-04f8-4773-914b-89ad8e9d3a96]
ERROR: LoadError: AssertionError: latexify does not support objects of type DataFrames.DataFrame.
Stacktrace:
  [1] _latexraw(args::DataFrames.DataFrame; kwargs::Base.Pairs{Symbol, Symbol, Tuple{Symbol}, NamedTuple{(:env,), Tuple{Symbol}}})
    @ Latexify C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexraw.jl:109
  [2] process_latexify(args::DataFrames.DataFrame; kwargs::Base.Pairs{Symbol, Symbol, Tuple{Symbol}, NamedTuple{(:env,), Tuple{Symbol}}})
    @ Latexify C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexify_function.jl:49
  [3] _latexinline(x::DataFrames.DataFrame; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Latexify C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexinline.jl:4
  [4] _latexinline(x::DataFrames.DataFrame)
    @ Latexify C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexinline.jl:3
  [5] process_latexify(args::DataFrames.DataFrame; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Latexify C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexify_function.jl:49
  [6] process_latexify
    @ C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexify_function.jl:40 [inlined]
  [7] latexify(args::DataFrames.DataFrame; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Latexify C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexify_function.jl:27
  [8] latexify(args::DataFrames.DataFrame)
    @ Latexify C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexify_function.jl:25
  [9] top-level scope
    @ s:\Julia\KM620\src\KM620_.jl:153
 [10] include(mod::Module, _path::String)
    @ Base .\Base.jl:419
 [11] include(x::String)
    @ KM620 s:\Julia\KM620\src\KM620.jl:1
 [12] top-level scope
    @ s:\Julia\KM620\src\KM620.jl:5
 [13] include
    @ .\Base.jl:419 [inlined]
 [14] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base .\loading.jl:1554
 [15] top-level scope
    @ stdin:1
in expression starting at s:\Julia\KM620\src\KM620_.jl:153
in expression starting at s:\Julia\KM620\src\KM620.jl:1
in expression starting at stdin:1
ERROR: Failed to precompile KM620 [0ccbab1c-04f8-4773-914b-89ad8e9d3a96] to C:\Users\nboyer.AIP\.julia\compiled\v1.8\KM620\jl_D268.tmp.
Stacktrace:
  [1] error(s::String)
    @ Base .\error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base .\loading.jl:1707
  [3] compilecache
    @ .\loading.jl:1651 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base .\loading.jl:1337
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base .\loading.jl:1200
  [6] macro expansion
    @ .\loading.jl:1180 [inlined]
  [7] macro expansion
    @ .\lock.jl:223 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base .\loading.jl:1144
  [9] eval
    @ .\boot.jl:368 [inlined]
 [10] eval
    @ .\Base.jl:65 [inlined]
 [11] repleval(m::Module, code::Expr, #unused#::String)
    @ VSCodeServer c:\Users\nboyer.AIP\.vscode\extensions\julialang.language-julia-1.38.2\scripts\packages\VSCodeServer\src\repl.jl:222
 [12] (::VSCodeServer.var"#107#109"{Module, Expr, REPL.LineEditREPL, REPL.LineEdit.Prompt})()
    @ VSCodeServer c:\Users\nboyer.AIP\.vscode\extensions\julialang.language-julia-1.38.2\scripts\packages\VSCodeServer\src\repl.jl:186
 [13] with_logstate(f::Function, logstate::Any)
    @ Base.CoreLogging .\logging.jl:511
 [14] with_logger
    @ .\logging.jl:623 [inlined]
 [15] (::VSCodeServer.var"#106#108"{Module, Expr, REPL.LineEditREPL, REPL.LineEdit.Prompt})()
    @ VSCodeServer c:\Users\nboyer.AIP\.vscode\extensions\julialang.language-julia-1.38.2\scripts\packages\VSCodeServer\src\repl.jl:187
 [16] #invokelatest#2
    @ .\essentials.jl:729 [inlined]
 [17] invokelatest(::Any)
    @ Base .\essentials.jl:726
 [18] macro expansion
    @ c:\Users\nboyer.AIP\.vscode\extensions\julialang.language-julia-1.38.2\scripts\packages\VSCodeServer\src\eval.jl:34 [inlined]
 [19] (::VSCodeServer.var"#61#62")()
    @ VSCodeServer .\task.jl:484

That line works fine when I run it in the REPL, so I don't understand why Latexify is telling me it does not support objects of type DataFrame.

julia> using DataFrames, Latexify

julia> latexify(DataFrame(rand(3,4),:auto))
                  x1                   x2                  x3                  x4
  –––––––––––––––––– –––––––––––––––––––– ––––––––––––––––––– –––––––––––––––––––
  0.8658906011977824   0.6204746003853868 0.39250644150930447 0.37950442125717754
  0.5553908465185704   0.8889941959270957  0.5758401699700543  0.8393666305977396
  0.4058410795595282 0.039306811749924675 0.39328115552984155  0.2908121260497748

julia> using DataFrames, Latexify, LaTeXStrings

julia> const coefficients_table_forprinting = DataFrame(
               "Material" => [L"Ferritic\ steel",
                               L"Austenitic\ stainless\ steel\ and\ nickel\-based\ alloys",
                               L"Duplex\ stainless\ steel",
                               L"Precipitation\ hardening,\ nickel\ based",
                               L"Aluminum",
                               L"Copper",
                               L"Titanium\ and\ zirconium"],
               "Max. Temp. (°F)" => [900,
                                       900,
                                       900,
                                       1000,
                                       250,
                                       150,
                                       500],
               L"m_2" => [:(0.60 * (1.00 - R)),
                       :(0.75 * (1.00 - R)),
                       :(0.70 * (0.95 - R)),
                       :(1.09 * (0.93 - R)),
                       :(0.52 * (0.98 - R)),
                       :(0.50 * (1.00 - R)),
                       :(0.50 * (0.98 - R))],
               L"m_3" => [:(2*log(1+(El/100))),
                       :(3*log(1+(El/100))),
                       :(2*log(1+(El/100))),
                       :(1*log(1+(El/100))),
                       :(1.3*log(1+(El/100))),
                       :(2*log(1+(El/100))),
                       :(1.3*log(1+(El/100)))],
               L"m_4" => [:(log(100 / (100 - RA))),
                       :(log(100 / (100 - RA))),
                       :(log(100 / (100 - RA))),
                       :(log(100 / (100 - RA))),
                       :(log(100 / (100 - RA))),
                       :(log(100 / (100 - RA))),
                       :(log(100 / (100 - RA)))],
               L"m_5" => [2.2,
                       0.6,
                       2.2,
                       2.2,
                       2.2,
                       2.2,
                       2.2],
               L"\epsilon_p" => [2.0E-5,
                       2.0E-5,
                       2.0E-5,
                       2.0E-5,
                       5.0E-6,
                       5.0E-6,
                       2.0E-5])
7×7 DataFrame
 Row │ Material                           Max. Temp. (°F)  $m_2$              $m_3$                    $m_4$                  $m_5$    $\\epsilon_p$ 
     │ LaTeXStr…                          Int64            Expr               Expr                     Expr                   Float64  Float64
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ $Ferritic\\ steel$                             900  0.6 * (1.0 - R)    2 * log(1 + El / 100)    log(100 / (100 - RA))      2.2         2.0e-5
   2 │ $Austenitic\\ stainless\\ steel\…              900  0.75 * (1.0 - R)   3 * log(1 + El / 100)    log(100 / (100 - RA))      0.6         2.0e-5
   3 │ $Duplex\\ stainless\\ steel$                   900  0.7 * (0.95 - R)   2 * log(1 + El / 100)    log(100 / (100 - RA))      2.2         2.0e-5
   4 │ $Precipitation\\ hardening,\\ ni…             1000  1.09 * (0.93 - R)  1 * log(1 + El / 100)    log(100 / (100 - RA))      2.2         2.0e-5
   5 │ $Aluminum$                                     250  0.52 * (0.98 - R)  1.3 * log(1 + El / 100)  log(100 / (100 - RA))      2.2         5.0e-6
   6 │ $Copper$                                       150  0.5 * (1.0 - R)    2 * log(1 + El / 100)    log(100 / (100 - RA))      2.2         5.0e-6
   7 │ $Titanium\\ and\\ zirconium$                   500  0.5 * (0.98 - R)   1.3 * log(1 + El / 100)  log(100 / (100 - RA))      2.2         2.0e-5

julia> latexify(select(coefficients_table_forprinting, 1))
                                                  Material
  ––––––––––––––––––––––––––––––––––––––––––––––––––––––––
                                           Ferritic\ steel
  Austenitic\ stainless\ steel\ and\ nickel\-based\ alloys
                                  Duplex\ stainless\ steel
                  Precipitation\ hardening,\ nickel\ based
                                                  Aluminum
                                                    Copper
                                  Titanium\ and\ zirconium

I initially tried to use @latexdefine in front of the DataFrame definition instead of calling latexify separately later, but that yielded a different error.

julia> using KM620
[ Info: Precompiling KM620 [0ccbab1c-04f8-4773-914b-89ad8e9d3a96]
ERROR: LoadError: Latexify.jl's latexoperation does not know what to do with one of the
          expressions provided (:("0.6 \\cdot \\left( 1.0 - R \\right)"))

Again, everything runs perfectly fine when I run weave on the file.

using DataFrames, Latexify, LaTeXStrings, Weave
set_chunk_defaults!(:echo => false)
weave("src/KM620_.jl"; doctype = "md2pdf", out_path = "KM620.pdf")

[nathanrboyer]

Possibly related to #131 ?

[gustaphe]

This one is tricky to debug, like you point out the function runs fine when called by itself.

The issue with @latexdefine is that it expects things to have a :raw representation (because it does something akin to "\$x = $(latexify(x; env=:raw))\$"). DataFrames don't have this, but only "latexify" as latexify(df; env=:mdtable). Of course you cannot put one of those inside an equation anyway so it's not much of a loss, but perhaps the error message is a bit unhelpful.

Do you need to print latex versions of all of these calls on package import? Perhaps try wrapping the printing in a function (__init__?), should make the whole thing easier to debug.

[nathanrboyer]

I don't need printing on package import. I only need printing when I Weave. I just can't figure out how to solve all these competing requirements.

1. Latexify requires that I print the function definitions at the same time I define them. I cannot print them later:

julia> KM620.H
H (generic function with 1 method)

julia> latexify(KM620.H)
ERROR: AssertionError: latexify does not support objects of type typeof(KM620.H).
Stacktrace:
 [1] _latexraw(args::Function; kwargs::Base.Pairs{Symbol, Symbol, Tuple{Symbol}, NamedTuple{(:env,), Tuple{Symbol}}})
   @ Latexify C:\Users\nboyer.AIP\.julia\packages\Latexify\2QVWl\src\latexraw.jl:109

julia> @latexrun H(σ_t, σ_ys, σ_uts, K) = @. 2 * (σ_t - (σ_ys + K * (σ_uts - σ_ys))) / (K * (σ_uts - σ_ys))
L"$H\left( \sigma_{t}, \sigma_{ys}, \sigma_{uts}, K \right) = \frac{2 \cdot \left( \sigma_{t} - \left( \sigma_{ys} + K \cdot \left( \sigma_{uts} - \sigma_{ys} \right) \right) \right)}{K \cdot \left( \sigma_{uts} - \sigma_{ys} \right)}$"

2. Latexify requires that I print the data frame definitions after I define them:

julia> table = DataFrame(x=1:3)
3×1 DataFrame
 Row │ x     
     │ Int64
─────┼───────
   1 │     1
   2 │     2
   3 │     3

julia> latexify(table)
  x
  –
  1
  2
  3

julia> @latexdefine table = DataFrame(x=1:3)
ERROR: AssertionError: latexify does not support objects of type DataFrame.

3. Weave requires that the weaved file be self-contained and be able to be broken into separate sections.
   This is why I had to move the module and end #module statements into a different file. Also, if I call weave on a file that just has latexify(x) calls, I will get undefined errors unless x is also defined in that file.

[gustaphe]

Okay I don't know what Weave does or how, so I'm not sure about any of this.

Perhaps what you need is someting like a macro that stores a function and its definition in the same object, which can then be latexified like

@latexrecipe f(x::FunctionAndExpr) = x.ex

But I don't know how sensible that would be. Does Weave really require you to latexify every line manually?

When it comes to the dataframes, it really doesn't make sense to put a table on the right hand side in a LaTeX equation. latexify(::DataFrame) doesn't produce valid LaTeX code but a markdown table, so if Weave expects LaTeX that's a dead end.

[nathanrboyer]

It is my first time using Weave.jl, so I am not totally clear on how it works. I think there is an intermediate markdown step: .jl -> .md -> .tex -> .pdf. It captures the output from running a .jl file and renders that to a PDF. In this case, I want the output to be latexified equations and tables, so I am manually latexifying everything. There might be a better way.

This discussion did help me to figure out a work-around for the error though. I have a working version of my code here! I moved the DataFrame definitions into their own file, and I moved the latexify(df) calls to their own file too. The downside is it now generates three separate PDF files, but I can combine those with a post-processing shell command. I will write more about it here.