Markdown Render failure

[gimmiereddy]

Bug description

When I try to preview or render a quarto document on my MacBook I get a failure to compile. I used VS Code and R studio, both give me the same failures.

Not sure where to go from here? Couldn't find a similar issue on github

Steps to reproduce

When compiling a simple hello world document I get an error saying dyld[XXXXX] missing symbol called.

This was the document:

---
title: "Quarto Basics"
format:
  html:
    code-fold: true
---

Hello world

compiling the document using export export QUARTO_PRINT_STACK=true I get some additional information using quarto render xxx.qmd:

pandoc 
  to: latex
  output-file: Introduction.tex
  standalone: true
  shift-heading-level-by: -1
  pdf-engine: xelatex
  variables:
    graphics: true
    tables: true
  default-image-extension: pdf
  toc: true
  number-sections: true
  
metadata
  documentclass: scrartcl
  classoption:
    - DIV=11
    - numbers=noendperiod
  papersize: letter
  header-includes:
    - '\KOMAoption{captions}{tableheading}'
  block-headings: true
  title: Quarto Basics
  colorlinks: true
  
dyld[10043]: missing symbol called
ERROR: Error
    at renderFiles (file:///Applications/quarto/bin/quarto.js:77056:29)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async render (file:///Applications/quarto/bin/quarto.js:81789:21)
    at async Command.fn (file:///Applications/quarto/bin/quarto.js:81936:32)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8102:13)
    at async quarto (file:///Applications/quarto/bin/quarto.js:114950:5)
    at async file:///Applications/quarto/bin/quarto.js:114968:9

Expected behavior

Rendering of Markdown

Actual behavior

Crash during compilation

Your environment

• IDE: VS code Version: 1.85.1
• OS: MacOS Sonoma 14.2.1 (23C71)

Quarto check output

Quarto 1.4.542
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.11: OK
      Dart Sass version 1.69.5: OK
      Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.542
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: v2024.01
      Chromium: 869685

[✓] Checking LaTeX....................OK
      Using: TinyTex
      Path: /Users/sidojensma/Library/TinyTeX/bin/universal-darwin
      Version: 2023

(\) Checking basic markdown render....dyld[10249]: missing symbol called
[✓] Checking basic markdown render....OK

ERROR: Error
    at renderFiles (file:///Applications/quarto/bin/quarto.js:77056:29)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async render (file:///Applications/quarto/bin/quarto.js:81789:21)
    at async file:///Applications/quarto/bin/quarto.js:98507:25
    at async withSpinner (file:///Applications/quarto/bin/quarto.js:75255:16)
    at async checkInstall (file:///Applications/quarto/bin/quarto.js:98493:5)
    at async check (file:///Applications/quarto/bin/quarto.js:98383:13)
    at async Command.fn (file:///Applications/quarto/bin/quarto.js:98666:5)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8102:13)
    at async quarto (file:///Applications/quarto/bin/quarto.js:114950:5)

[cscheid]

A few questions:

How did you install quarto?

Can you provide the output of quarto pandoc --version?

I suspect something is going bad when Pandoc is being executed.

[gimmiereddy]

I downloaded the .pkg from the Quarto website. And also updated to the new version 1.4.543, same issue.

Output from quarto pandoc --version is

pandoc 3.1.11
Features: +server +lua
Scripting engine: Lua 5.4
User data directory: /Users/sidojensma/.local/share/pandoc
Copyright (C) 2006-2023 John MacFarlane. Web: https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

[cscheid]

Is there anything unconventional about your environment? That error happens because some binary is attempting to call a dynamically-loaded library that's missing.

Unfortunately I can't repro the problem locally (I'm on macOS as well). Are you, by chance, on an Intel Mac laptop? We'll need more information about your specific setup to know more.

[gimmiereddy]

I am on an intel Mac laptop (2.2 GHz 6-Core Intel Core i7). I just downloaded the .pkg and installed it.

Also I just installed quarto 1.3.450 and this works with no problem on the same file.

The output from quarto check is (which includes the pandoc version)

[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.1: OK
      Dart Sass version 1.55.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.3.450
      Path: /Applications/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.1
      Path: /Users/sidojensma/Documents/gitea/Quarto/env/bin/python
      Jupyter: 5.7.1
      Kernels: python3

(|) Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.3.2
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library
      knitr: 1.45
      rmarkdown: 2.25

[✓] Checking Knitr engine render......OK

[cscheid]

I am on an intel Mac laptop (2.2 GHz 6-Core Intel Core i7). I just downloaded the .pkg and installed it.

Ok, this is great information, thanks! Let me investigate and I'll follow up.

1. We just tested on an Intel macOS running Ventura, and we still can't repro the issue. I'm updating a separate laptop to 14.2.1 now.

[cscheid]

@gimmiereddy Unfortunately, I just tested this on a 14.2.1 macOS laptop and I can't reproduce. It's also a 6-core i7.

Do you mind sharing the result of running env? Just please take a moment to investigate the output to ensure there's no secret material in there (if you feel more comfortable, please feel free to share it via email at carlos.scheidegger@posit.co)

[gimmiereddy]

I just email the output from env to the provided mail adres..

[cscheid]

After a long process (thank you so much @gimmiereddy for helping along) we've identified the bug.

• Pandoc's Lua interpreter has a default path for searching for .so files that are used in extensions. We confess to be somewhat confused by this, since we were operating under the assumption (as were some of the Pandoc developers) that Pandoc's Lua infrastructure would disallow loading shared objects. As a result, we were assuming that shared objects would never be loaded by Pandoc's Lua.
• In Quarto 1.4, we use the lpeg library, which has an .so implementation in some Lua installations
• In macOS, it seems that Pandoc's Lua will try to load .so files if found on LUA_CPATH.
• If there is a conflicting installation of Lua in the system that happens to have an incompatible version of the .so file, then we will trigger the problem.

The easy fix here here is for Quarto to hard-code LUA_CPATH to never use system paths.

[eo1989]

Apologies if this resurrects a dead thread, but reading the comments in pandoc.ts sent me here.
I'm trying to get quarto to run from cli in wsl2. Just git pulled the newest commit, and ran ./configure.sh. Before this (and it does run in native windows 11, installed from the msi package), quarto did also error out when trying to render from within nvim. Specifically regarding this error, when I run quarto check, I received this error:

❯ bat ./src/command/render/pandoc.ts --line-range=235:
❯ quarto check Quarto 99.9.9
[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.1.11: OK
Dart Sass version 1.69.5: OK
Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 99.9.9
Path: /home/rocket/repos/quarto-cli/package/dist/bin

[✓] Checking tools....................OK

  TinyTeX: v2023.01
  Chromium: (not installed)

[✓] Checking LaTeX....................OK

  Using: TinyTex
  Path: /home/rocket/.TinyTeX/bin/x86_64-linux
  Version: 2022

[✓] Checking basic markdown render....OK

ERROR: Missing environment variable: QUARTO_LUA_CPATH

Stack trace:

at getEnv (file:///home/rocket/repos/quarto-cli/package/src/util/utils.ts:9:13)
at setupPandocEnv (file:///home/rocket/repos/quarto-cli/src/command/render/pandoc.ts:244:32)
at runPandoc (file:///home/rocket/repos/quarto-cli/src/command/render/pandoc.ts:1161:3)
at eventLoopTick (ext:core/01_core.js:183:11)
at async renderPandoc (file:///home/rocket/repos/quarto-cli/src/command/render/render.ts:202:24)
at async Object.onRender (file:///home/rocket/repos/quarto-cli/src/command/render/render-files.ts:718:30)
at async renderFileInternal (file:///home/rocket/repos/quarto-cli/src/command/render/render-files.ts:680:9)
at async renderFiles (file:///home/rocket/repos/quarto-cli/src/command/render/render-files.ts:329:9)
at async render (file:///home/rocket/repos/quarto-cli/src/command/render/render-shared.ts:99:18)
at async file:///home/rocket/repos/quarto-cli/src/command/check/check.ts:209:20"

[cscheid]

@eo1989 I think this was a bad fix on my side. I think you have a LUA_CPATH variable set in your environment. Quarto can't use that, because it causes the issues described above. I believe that a workaround for you should be export QUARTO_LUA_CPATH="" while we fix the underlying problem.

[cscheid]

We now always override LUA_CPATH. If QUARTO_LUA_CPATH exists, we set LUA_CPATH to its value. Otherwise, we set it to "". This way, our default case remains that LUA_CPATH doesn't point to system libraries. But in the odd case that this is, in fact, what the user is looking to do, they can instead set QUARTO_LUA_CPATH.

[mroberts1]

Current output from quarto check. The Lua error is at the end.

Quarto 1.4.554
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.11: OK
      Dart Sass version 1.69.5: OK
      Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.554
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: v2024.05
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: TinyTex
      Path: /Users/dokoissho/Library/TinyTeX/bin/universal-darwin
      Version: 2024

(/) Checking basic markdown render....Error running Lua:
error loading module 'lpeg' from file '/usr/local/lib/lua/5.4/lpeg.so':
	dlopen(/usr/local/lib/lua/5.4/lpeg.so, 0x0006): tried: '/usr/local/lib/lua/5.4/lpeg.so' (code signature in <ADDBC93A-9F32-3E8C-A046-33C5CF6698F9> '/usr/local/Cellar/lpeg/1.1.0/lib/lua/5.4/lpeg.so' not valid for use in process: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.), '/usr/local/Cellar/lpeg/1.1.0/lib/lua/5.4/lpeg.so' (code signature in <ADDBC93A-9F32-3E8C-A046-33C5CF6698F9> '/usr/local/Cellar/lpeg/1.1.0/lib/lua/5.4/lpeg.so' not valid for use in process: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.)
stack traceback:
	[C]: in upvalue 'orig_require'
	[string "..."]:1442: in function 'require'
	/Applications/quarto/share/pandoc/datadir/lpegshortcode.lua:4: in main chunk
	[C]: in upvalue 'orig_require'
	[string "..."]:1442: in function 'require'
	/Applications/quarto/share/pandoc/datadir/readqmd.lua:6: in main chunk
	[C]: in upvalue 'orig_require'
	[string "..."]:1442: in function 'require'
	/Applications/quarto/share/filters/qmd-reader.lua:6: in main chunk
[✓] Checking basic markdown render....OK

[cscheid]

@mroberts1 what do you get from echo $LUA_PATH, echo $LUA_CPATH, or env more generally?

[cscheid]

@mroberts1 oh I'm so sorry, you're running 1.4. This fix is in 1.5!

[mroberts1]

Thank you, the upgrade to 1.5.37 finally fixed the Lua error.