render() (really pandoc) can hang under certain circumstances

[cswarth]

This is not a bug in rmarkdown. I report it here because it manifests itself by hanging rmarkdown::render() so there should be a record of what is really going on.

render() tries to establish the version of pandoc available. During this process 'pandoc --version' can hang forever (while using up all available memory) which causes render() to also hang forever.

I noticed this when trying to run render() in an ESS session running R on a remote server using tramp mode. The environment on the remote ubuntu system apparently sets LC_ALL=C as part of it's localization. As far as I can tell, on a ubuntu machine if the LC_ALL environment variable is set (to anything at all), the statically linked version of pandoc will hang.

This situation sounds exactly like ​(this bug)[https://ghc.haskell.org/trac/ghc/ticket/7344] reported 17 months ago.

Symptoms are 'pandoc --version' hangs and consumes memory until the process eventually dies with an OOM error. This seems to only happen when LC_ALL is set in the environment, and only when running the statically linked pandoc.

$ uname -a
Linux server 3.5.0-43-generic #66~precise1-Ubuntu SMP Thu Oct 24 14:52:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

$ wget https://s3.amazonaws.com/rstudio-buildtools/pandoc-1.12.3.zip

$ unzip pandoc-1.12.3.zip

$ file pandoc-1.12.3/linux/debian/x86_64/pandoc
pandoc-1.12.3/linux/debian/x86_64/pandoc: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.15, BuildID[sha1]=0x9bc69f51b7c213cf58e9f8692f4b46f9f05fb723, not stripped

# buggy behavior
$ LC_ALL=C strace -e trace=open,close  pandoc-1.12.3/linux/debian/x86_64/pandoc --version
open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib/gconv/gconv-modules", O_RDONLY) = -1 ENOENT (No such file or directory)
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
^C--- SIGINT (Interrupt) @ 0 (0) ---

# correct behavior
$ strace -e trace=open,close  pandoc-1.12.3/linux/debian/x86_64/pandoc --version
open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
close(3)                                = 0
pandoc 1.12.3
Compiled with texmath 0.6.6, highlighting-kate 0.5.6.
Syntax highlighting is supported for the following languages:
    actionscript, ada, apache, asn1, asp, awk, bash, bibtex, boo, c, changelog,
    clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css, curry, d,
    diff, djangotemplate, doxygen, doxygenlua, dtd, eiffel, email, erlang,
    fortran, fsharp, gnuassembler, go, haskell, haxe, html, ini, java, javadoc,

Note this problem does not occur if I run an older dynamically linked pandoc under the same circumstances.

$ LC_ALL=C /usr/bin/pandoc --version
pandoc 1.9.1.1
Compiled with citeproc-hs 0.3.4, texmath 0.6.0.3, highlighting-kate 0.5.0.5.
Syntax highlighting is supported for the following languages:
    Actionscript, Ada, Alert, Alert_indent, Apache, Asn1, Asp, Awk, Bash,
    Bibtex, Boo, C, Changelog, Clojure, Cmake, Coffeescript, Coldfusion,
    Commonlisp, Cpp, Cs, Css, D, Diff, Djangotemplate, Doxygen, Dtd, Eiffel, ...

[jjallaire]

Thanks for reporting this. What a bug! I've made this change to ensure that we never hit this when calling pandoc from rmarkdown:
3071eaa

[trestletech]

Turns out there are a couple of other environment variables worth checking:

HOME -- if not present, Pandoc will crash reasonably, but still before giving us the --version.
LANG -- amazingly, if not set, will do what was reported above, eating up all available RAM before dying a horrible death.

[jjallaire]

See Jeff's example here: https://gist.github.com/jjallaire/9390e6dfab153759423f

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.