New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rmarkdown::render freezes because pandoc freezes when LC_ALL and LANG are unset #17

Closed
MarcinKosinski opened this Issue Jul 20, 2016 · 14 comments

Comments

Projects
None yet
3 participants
@MarcinKosinski

MarcinKosinski commented Jul 20, 2016

Recently I have noticed thath rmarkdown::render fails in your container because pandoc freezes if one have LC_ALL and LAND unset. So if someone has a shiny app with a button that generates simple rmarkdown reports like this app (http://shiny.rstudio.com/articles/generating-reports.html) then he'll have serious diagnostics problems as report will not be rendered and pandoc will freeze for eternity.

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
95139ad68706        rocker/shiny        "/usr/bin/shiny-serve"   9 minutes ago       Up 9 minutes        0.0.0.0:3838->3838/tcp   small_leavitt
mkosinski@mkosinski-E540:~$ docker exec -it 95139ad68706 /bin/bash
root@95139ad68706:/# /opt/shiny-server/ext/pandoc/pandoc                                                                                                                                                                              
^C
root@95139ad68706:/# /opt/shiny-server/ext/pandoc/pandoc --version 
pandoc 1.12.3
Compiled with texmath 0.6.6, highlighting-kate 0.5.6.
Syntax highlighting is supported for the following languages:
    actionscript, ada, apache, asn1, asp, awk, bash, bibtex, boo, c, changelog,
    clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css, curry, d,
    diff, djangotemplate, doxygen, doxygenlua, dtd, eiffel, email, erlang,
    fortran, fsharp, gnuassembler, go, haskell, haxe, html, ini, java, javadoc,
    javascript, json, jsp, julia, latex, lex, literatecurry, literatehaskell,
    lua, makefile, mandoc, markdown, matlab, maxima, metafont, mips, modelines,
    modula2, modula3, monobasic, nasm, noweb, objectivec, objectivecpp, ocaml,
    octave, pascal, perl, php, pike, postscript, prolog, python, r,
    relaxngcompact, restructuredtext, rhtml, roff, ruby, rust, scala, scheme,
    sci, sed, sgml, sql, sqlmysql, sqlpostgresql, tcl, texinfo, verilog, vhdl,
    xml, xorg, xslt, xul, yacc, yaml
Default user data directory: /root/.pandoc
Copyright (C) 2006-2013 John MacFarlane
Web:  http://johnmacfarlane.net/pandoc
This is free software; see the source for copying conditions.  There is no
warranty, not even for merchantability or fitness for a particular purpose.
root@95139ad68706:/# unset LANG
root@95139ad68706:/# /opt/shiny-server/ext/pandoc/pandoc --version
pandoc 1.12.3
Compiled with texmath 0.6.6, highlighting-kate 0.5.6.
Syntax highlighting is supported for the following languages:
    actionscript, ada, apache, asn1, asp, awk, bash, bibtex, boo, c, changelog,
    clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css, curry, d,
    diff, djangotemplate, doxygen, doxygenlua, dtd, eiffel, email, erlang,
    fortran, fsharp, gnuassembler, go, haskell, haxe, html, ini, java, javadoc,
    javascript, json, jsp, julia, latex, lex, literatecurry, literatehaskell,
    lua, makefile, mandoc, markdown, matlab, maxima, metafont, mips, modelines,
    modula2, modula3, monobasic, nasm, noweb, objectivec, objectivecpp, ocaml,
    octave, pascal, perl, php, pike, postscript, prolog, python, r,
    relaxngcompact, restructuredtext, rhtml, roff, ruby, rust, scala, scheme,
    sci, sed, sgml, sql, sqlmysql, sqlpostgresql, tcl, texinfo, verilog, vhdl,
    xml, xorg, xslt, xul, yacc, yaml
Default user data directory: /root/.pandoc
Copyright (C) 2006-2013 John MacFarlane
Web:  http://johnmacfarlane.net/pandoc
This is free software; see the source for copying conditions.  There is no
warranty, not even for merchantability or fitness for a particular purpose.
root@95139ad68706:/# unset LC_ALL                                 
root@95139ad68706:/# /opt/shiny-server/ext/pandoc/pandoc --version

This command never stops. What is more, docker start using maximum available RAM for him.

LC_ALL is unset without user's knowledge in rmarkdown https://github.com/rstudio/rmarkdown/blob/master/R/pandoc.R#L601, so it is very easy to have problems with LANG and to freeze docker with no meaningful error message.

Do you think this is the case of your docker or I rather submit this discovery to rstudio/rmarkdown. In my opinion the solution is : pandoc should close after a while with message cant run when LC_ALL and LANG global env are unset` or something like this.

Start command for docker: docker run --rm -p 3838:3838 rocker/shiny

@wch

This comment has been minimized.

Show comment
Hide comment
@wch

wch Jul 20, 2016

Contributor

Hm, thanks for the detailed bug report. Are you using that basic Rmd template file from the example? I ask because we've encountered some cases where pandoc is very, very slow.

However, this bug was fixed in recent versions of pandoc (1.17 and up). It looks like you have pandoc 1.12.3. If you pull rocker/shiny, you should get an image that has pandoc 1.17.0.3.

Contributor

wch commented Jul 20, 2016

Hm, thanks for the detailed bug report. Are you using that basic Rmd template file from the example? I ask because we've encountered some cases where pandoc is very, very slow.

However, this bug was fixed in recent versions of pandoc (1.17 and up). It looks like you have pandoc 1.12.3. If you pull rocker/shiny, you should get an image that has pandoc 1.17.0.3.

@MarcinKosinski

This comment has been minimized.

Show comment
Hide comment
@MarcinKosinski

MarcinKosinski Jul 20, 2016

I used this .Rmd file rmarkdown::draft(file = "my_artilce.Rmd", template = "html_vignette", package = "rmarkdown", edit = FALSE)

However, this bug was fixed in recent versions of pandoc (1.17 and up). It looks like you have pandoc 1.12.3. If you pull rocker/shiny, you should get an image that has pandoc 1.17.0.3.

If I pull right now after some of your changes or if I pull anytime? I pulled yesterday and the version that was inside is printed in my previous post : )

MarcinKosinski commented Jul 20, 2016

I used this .Rmd file rmarkdown::draft(file = "my_artilce.Rmd", template = "html_vignette", package = "rmarkdown", edit = FALSE)

However, this bug was fixed in recent versions of pandoc (1.17 and up). It looks like you have pandoc 1.12.3. If you pull rocker/shiny, you should get an image that has pandoc 1.17.0.3.

If I pull right now after some of your changes or if I pull anytime? I pulled yesterday and the version that was inside is printed in my previous post : )

@wch

This comment has been minimized.

Show comment
Hide comment
@wch

wch Jul 20, 2016

Contributor

Hm, that's weird. According to the build log for the latest rocker/shiny, it has pandoc 1.17.0.3 installed: https://hub.docker.com/r/rocker/shiny/builds/b7stjamugk5en6nu9wex5ns/

Are you using that basic Rmd template from the example?

Contributor

wch commented Jul 20, 2016

Hm, that's weird. According to the build log for the latest rocker/shiny, it has pandoc 1.17.0.3 installed: https://hub.docker.com/r/rocker/shiny/builds/b7stjamugk5en6nu9wex5ns/

Are you using that basic Rmd template from the example?

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Jul 20, 2016

Member

@MarcinKosinski Then you are doing something wrong:

$ docker images rocker/shiny
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        2 weeks ago         1.525 GB
$ 
Member

eddelbuettel commented Jul 20, 2016

@MarcinKosinski Then you are doing something wrong:

$ docker images rocker/shiny
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        2 weeks ago         1.525 GB
$ 
@MarcinKosinski

This comment has been minimized.

Show comment
Hide comment
@MarcinKosinski

MarcinKosinski Jul 20, 2016

The same output

mkosinski@mkosinski-E540:~$ docker images rocker/shiny
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        2 weeks ago         1.525 GB
mkosinski@mkosinski-E540:~$ 

MarcinKosinski commented Jul 20, 2016

The same output

mkosinski@mkosinski-E540:~$ docker images rocker/shiny
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        2 weeks ago         1.525 GB
mkosinski@mkosinski-E540:~$ 
@MarcinKosinski

This comment has been minimized.

Show comment
Hide comment
@MarcinKosinski

MarcinKosinski Jul 20, 2016

@wch

I get freeze on this

rmarkdown::draft(file = "my_artilce.Rmd", template = "html_vignette", package = "rmarkdown", edit = FALSE)
render('my_article.Rmd')

and on the example from this site http://shiny.rstudio.com/articles/generating-reports.html

MarcinKosinski commented Jul 20, 2016

@wch

I get freeze on this

rmarkdown::draft(file = "my_artilce.Rmd", template = "html_vignette", package = "rmarkdown", edit = FALSE)
render('my_article.Rmd')

and on the example from this site http://shiny.rstudio.com/articles/generating-reports.html

@MarcinKosinski

This comment has been minimized.

Show comment
Hide comment
@MarcinKosinski

MarcinKosinski Jul 20, 2016

@wch talking about pandoc. Maybe you are talking about regular pandoc. I am checking version from pandoc that is inside /srv/shiny-server/ext/pandoc directory. I understand that there are 2 pandocs

mkosinski@mkosinski-E540:~$ docker exec -it 95139ad68706 /bin/bash
root@95139ad68706:/# pandoc --version                                                                                                                            
pandoc 1.17.0.3
Compiled with texmath 0.8.6.3, highlighting-kate 0.6.2.
Syntax highlighting is supported for the following languages:
    abc, actionscript, ada, agda, apache, asn1, asp, awk, bash, bibtex, boo, c,
    changelog, clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css,
    curry, d, diff, djangotemplate, dockerfile, dot, doxygen, doxygenlua, dtd,
    eiffel, elixir, email, erlang, fasm, fortran, fsharp, gcc, glsl,
    gnuassembler, go, hamlet, haskell, haxe, html, idris, ini, isocpp, java,
    javadoc, javascript, json, jsp, julia, kotlin, latex, lex, lilypond,
    literatecurry, literatehaskell, llvm, lua, m4, makefile, mandoc, markdown,
    mathematica, matlab, maxima, mediawiki, metafont, mips, modelines, modula2,
    modula3, monobasic, nasm, noweb, objectivec, objectivecpp, ocaml, octave,
    opencl, pascal, perl, php, pike, postscript, prolog, pure, python, r,
    relaxng, relaxngcompact, rest, rhtml, roff, ruby, rust, scala, scheme, sci,
    sed, sgml, sql, sqlmysql, sqlpostgresql, tcl, tcsh, texinfo, verilog, vhdl,
    xml, xorg, xslt, xul, yacc, yaml, zsh
Default user data directory: /root/.pandoc
Copyright (C) 2006-2016 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.
root@95139ad68706:/# /opt/shiny-server/ext/pandoc/pandoc --version                                                                                               
pandoc 1.12.3
Compiled with texmath 0.6.6, highlighting-kate 0.5.6.
Syntax highlighting is supported for the following languages:
    actionscript, ada, apache, asn1, asp, awk, bash, bibtex, boo, c, changelog,
    clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css, curry, d,
    diff, djangotemplate, doxygen, doxygenlua, dtd, eiffel, email, erlang,
    fortran, fsharp, gnuassembler, go, haskell, haxe, html, ini, java, javadoc,
    javascript, json, jsp, julia, latex, lex, literatecurry, literatehaskell,
    lua, makefile, mandoc, markdown, matlab, maxima, metafont, mips, modelines,
    modula2, modula3, monobasic, nasm, noweb, objectivec, objectivecpp, ocaml,
    octave, pascal, perl, php, pike, postscript, prolog, python, r,
    relaxngcompact, restructuredtext, rhtml, roff, ruby, rust, scala, scheme,
    sci, sed, sgml, sql, sqlmysql, sqlpostgresql, tcl, texinfo, verilog, vhdl,
    xml, xorg, xslt, xul, yacc, yaml
Default user data directory: /root/.pandoc
Copyright (C) 2006-2013 John MacFarlane
Web:  http://johnmacfarlane.net/pandoc
This is free software; see the source for copying conditions.  There is no
warranty, not even for merchantability or fitness for a particular purpose.

Before rendering I set RSTUDIO_PANDOC to be:

Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")

MarcinKosinski commented Jul 20, 2016

@wch talking about pandoc. Maybe you are talking about regular pandoc. I am checking version from pandoc that is inside /srv/shiny-server/ext/pandoc directory. I understand that there are 2 pandocs

mkosinski@mkosinski-E540:~$ docker exec -it 95139ad68706 /bin/bash
root@95139ad68706:/# pandoc --version                                                                                                                            
pandoc 1.17.0.3
Compiled with texmath 0.8.6.3, highlighting-kate 0.6.2.
Syntax highlighting is supported for the following languages:
    abc, actionscript, ada, agda, apache, asn1, asp, awk, bash, bibtex, boo, c,
    changelog, clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css,
    curry, d, diff, djangotemplate, dockerfile, dot, doxygen, doxygenlua, dtd,
    eiffel, elixir, email, erlang, fasm, fortran, fsharp, gcc, glsl,
    gnuassembler, go, hamlet, haskell, haxe, html, idris, ini, isocpp, java,
    javadoc, javascript, json, jsp, julia, kotlin, latex, lex, lilypond,
    literatecurry, literatehaskell, llvm, lua, m4, makefile, mandoc, markdown,
    mathematica, matlab, maxima, mediawiki, metafont, mips, modelines, modula2,
    modula3, monobasic, nasm, noweb, objectivec, objectivecpp, ocaml, octave,
    opencl, pascal, perl, php, pike, postscript, prolog, pure, python, r,
    relaxng, relaxngcompact, rest, rhtml, roff, ruby, rust, scala, scheme, sci,
    sed, sgml, sql, sqlmysql, sqlpostgresql, tcl, tcsh, texinfo, verilog, vhdl,
    xml, xorg, xslt, xul, yacc, yaml, zsh
Default user data directory: /root/.pandoc
Copyright (C) 2006-2016 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.
root@95139ad68706:/# /opt/shiny-server/ext/pandoc/pandoc --version                                                                                               
pandoc 1.12.3
Compiled with texmath 0.6.6, highlighting-kate 0.5.6.
Syntax highlighting is supported for the following languages:
    actionscript, ada, apache, asn1, asp, awk, bash, bibtex, boo, c, changelog,
    clojure, cmake, coffee, coldfusion, commonlisp, cpp, cs, css, curry, d,
    diff, djangotemplate, doxygen, doxygenlua, dtd, eiffel, email, erlang,
    fortran, fsharp, gnuassembler, go, haskell, haxe, html, ini, java, javadoc,
    javascript, json, jsp, julia, latex, lex, literatecurry, literatehaskell,
    lua, makefile, mandoc, markdown, matlab, maxima, metafont, mips, modelines,
    modula2, modula3, monobasic, nasm, noweb, objectivec, objectivecpp, ocaml,
    octave, pascal, perl, php, pike, postscript, prolog, python, r,
    relaxngcompact, restructuredtext, rhtml, roff, ruby, rust, scala, scheme,
    sci, sed, sgml, sql, sqlmysql, sqlpostgresql, tcl, texinfo, verilog, vhdl,
    xml, xorg, xslt, xul, yacc, yaml
Default user data directory: /root/.pandoc
Copyright (C) 2006-2013 John MacFarlane
Web:  http://johnmacfarlane.net/pandoc
This is free software; see the source for copying conditions.  There is no
warranty, not even for merchantability or fitness for a particular purpose.

Before rendering I set RSTUDIO_PANDOC to be:

Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")
@wch

This comment has been minimized.

Show comment
Hide comment
@wch

wch Jul 20, 2016

Contributor

I'm not able to reproduce your problem. I ran the following and it worked almost instantaneously (with the example code fixed). I also noticed that setting that environment variable didn't change the pandoc path that was actually being used by rmarkdown::render.

$ docker images rocker/shiny
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        2 weeks ago         1.525 GB
$ docker run --rm -ti rocker/shiny /bin/bash
root@6c3fdd2eca68:/# R

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette", package = "rmarkdown", edit = FALSE)
> rmarkdown::render('my_article.Rmd')


processing file: my_article.Rmd
  |.............                                                    |  20%
   inline R code fragments

  |..........................                                       |  40%
label: unnamed-chunk-1 (with options) 
List of 1
 $ fig.show: chr "hold"

  |.......................................                          |  60%
  ordinary text without R code

  |....................................................             |  80%
label: unnamed-chunk-2 (with options) 
List of 2
 $ echo   : logi FALSE
 $ results: chr "asis"

  |.................................................................| 100%
  ordinary text without R code


output file: my_article.knit.md

/usr/bin/pandoc +RTS -K512m -RTS my_article.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output my_article.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/lib/R/site-library/rmarkdown/rmd/h/default.html --css /usr/local/lib/R/site-library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --mathjax --variable 'mathjax-url:https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --highlight-style pygments 

Output created: my_article.html
> 
> Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")
> rmarkdown::render('my_article.Rmd')


processing file: my_article.Rmd
  |.............                                                    |  20%
   inline R code fragments

  |..........................                                       |  40%
label: unnamed-chunk-1 (with options) 
List of 1
 $ fig.show: chr "hold"

  |.......................................                          |  60%
  ordinary text without R code

  |....................................................             |  80%
label: unnamed-chunk-2 (with options) 
List of 2
 $ echo   : logi FALSE
 $ results: chr "asis"

  |.................................................................| 100%
  ordinary text without R code


output file: my_article.knit.md

/usr/bin/pandoc +RTS -K512m -RTS my_article.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output my_article.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/lib/R/site-library/rmarkdown/rmd/h/default.html --css /usr/local/lib/R/site-library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --mathjax --variable 'mathjax-url:https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --highlight-style pygments 

Output created: my_article.html
> 
Contributor

wch commented Jul 20, 2016

I'm not able to reproduce your problem. I ran the following and it worked almost instantaneously (with the example code fixed). I also noticed that setting that environment variable didn't change the pandoc path that was actually being used by rmarkdown::render.

$ docker images rocker/shiny
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        2 weeks ago         1.525 GB
$ docker run --rm -ti rocker/shiny /bin/bash
root@6c3fdd2eca68:/# R

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette", package = "rmarkdown", edit = FALSE)
> rmarkdown::render('my_article.Rmd')


processing file: my_article.Rmd
  |.............                                                    |  20%
   inline R code fragments

  |..........................                                       |  40%
label: unnamed-chunk-1 (with options) 
List of 1
 $ fig.show: chr "hold"

  |.......................................                          |  60%
  ordinary text without R code

  |....................................................             |  80%
label: unnamed-chunk-2 (with options) 
List of 2
 $ echo   : logi FALSE
 $ results: chr "asis"

  |.................................................................| 100%
  ordinary text without R code


output file: my_article.knit.md

/usr/bin/pandoc +RTS -K512m -RTS my_article.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output my_article.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/lib/R/site-library/rmarkdown/rmd/h/default.html --css /usr/local/lib/R/site-library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --mathjax --variable 'mathjax-url:https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --highlight-style pygments 

Output created: my_article.html
> 
> Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")
> rmarkdown::render('my_article.Rmd')


processing file: my_article.Rmd
  |.............                                                    |  20%
   inline R code fragments

  |..........................                                       |  40%
label: unnamed-chunk-1 (with options) 
List of 1
 $ fig.show: chr "hold"

  |.......................................                          |  60%
  ordinary text without R code

  |....................................................             |  80%
label: unnamed-chunk-2 (with options) 
List of 2
 $ echo   : logi FALSE
 $ results: chr "asis"

  |.................................................................| 100%
  ordinary text without R code


output file: my_article.knit.md

/usr/bin/pandoc +RTS -K512m -RTS my_article.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output my_article.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/lib/R/site-library/rmarkdown/rmd/h/default.html --css /usr/local/lib/R/site-library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --mathjax --variable 'mathjax-url:https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --highlight-style pygments 

Output created: my_article.html
> 
@MarcinKosinski

This comment has been minimized.

Show comment
Hide comment
@MarcinKosinski

MarcinKosinski Jul 20, 2016

Have you tried to unset LC_ALL and LANG, before you entered R, as I did
to freeze the docker?

2016-07-20 19:36 GMT+02:00 Winston Chang notifications@github.com:

I'm not able to reproduce your problem. I ran the following and it worked
almost instantaneously (with the example code fixed). I also noticed that
setting that environment variable didn't change the pandoc path that was
actually being used by rmarkdown::render.

$ docker images rocker/shiny
REPOSITORY TAG IMAGE ID CREATED SIZE
rocker/shiny latest fe1a132b5b98 2 weeks ago 1.525 GB
$ docker run --rm -ti rocker/shiny /bin/bash
root@6c3fdd2eca68:/# R

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette", package = "rmarkdown", edit = FALSE)
rmarkdown::render('my_article.Rmd')

processing file: my_article.Rmd
|............. | 20%
inline R code fragments

|.......................... | 40%
label: unnamed-chunk-1 (with options)
List of 1
$ fig.show: chr "hold"

|....................................... | 60%
ordinary text without R code

|.................................................... | 80%
label: unnamed-chunk-2 (with options)
List of 2
$ echo : logi FALSE
$ results: chr "asis"

|.................................................................| 100%
ordinary text without R code

output file: my_article.knit.md

/usr/bin/pandoc +RTS -K512m -RTS my_article.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output my_article.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/lib/R/site-library/rmarkdown/rmd/h/default.html --css /usr/local/lib/R/site-library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --mathjax --variable 'mathjax-url:https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --highlight-style pygments

Output created: my_article.html

Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")
rmarkdown::render('my_article.Rmd')

processing file: my_article.Rmd
|............. | 20%
inline R code fragments

|.......................... | 40%
label: unnamed-chunk-1 (with options)
List of 1
$ fig.show: chr "hold"

|....................................... | 60%
ordinary text without R code

|.................................................... | 80%
label: unnamed-chunk-2 (with options)
List of 2
$ echo : logi FALSE
$ results: chr "asis"

|.................................................................| 100%
ordinary text without R code

output file: my_article.knit.md

/usr/bin/pandoc +RTS -K512m -RTS my_article.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output my_article.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/lib/R/site-library/rmarkdown/rmd/h/default.html --css /usr/local/lib/R/site-library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --mathjax --variable 'mathjax-url:https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --highlight-style pygments

Output created: my_article.html


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#17 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGdazhOgP00GosqeN-D1yKT_t9RoxTwJks5qXly7gaJpZM4JQ18M
.

MarcinKosinski commented Jul 20, 2016

Have you tried to unset LC_ALL and LANG, before you entered R, as I did
to freeze the docker?

2016-07-20 19:36 GMT+02:00 Winston Chang notifications@github.com:

I'm not able to reproduce your problem. I ran the following and it worked
almost instantaneously (with the example code fixed). I also noticed that
setting that environment variable didn't change the pandoc path that was
actually being used by rmarkdown::render.

$ docker images rocker/shiny
REPOSITORY TAG IMAGE ID CREATED SIZE
rocker/shiny latest fe1a132b5b98 2 weeks ago 1.525 GB
$ docker run --rm -ti rocker/shiny /bin/bash
root@6c3fdd2eca68:/# R

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette", package = "rmarkdown", edit = FALSE)
rmarkdown::render('my_article.Rmd')

processing file: my_article.Rmd
|............. | 20%
inline R code fragments

|.......................... | 40%
label: unnamed-chunk-1 (with options)
List of 1
$ fig.show: chr "hold"

|....................................... | 60%
ordinary text without R code

|.................................................... | 80%
label: unnamed-chunk-2 (with options)
List of 2
$ echo : logi FALSE
$ results: chr "asis"

|.................................................................| 100%
ordinary text without R code

output file: my_article.knit.md

/usr/bin/pandoc +RTS -K512m -RTS my_article.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output my_article.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/lib/R/site-library/rmarkdown/rmd/h/default.html --css /usr/local/lib/R/site-library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --mathjax --variable 'mathjax-url:https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --highlight-style pygments

Output created: my_article.html

Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")
rmarkdown::render('my_article.Rmd')

processing file: my_article.Rmd
|............. | 20%
inline R code fragments

|.......................... | 40%
label: unnamed-chunk-1 (with options)
List of 1
$ fig.show: chr "hold"

|....................................... | 60%
ordinary text without R code

|.................................................... | 80%
label: unnamed-chunk-2 (with options)
List of 2
$ echo : logi FALSE
$ results: chr "asis"

|.................................................................| 100%
ordinary text without R code

output file: my_article.knit.md

/usr/bin/pandoc +RTS -K512m -RTS my_article.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output my_article.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/lib/R/site-library/rmarkdown/rmd/h/default.html --css /usr/local/lib/R/site-library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --mathjax --variable 'mathjax-url:https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --highlight-style pygments

Output created: my_article.html


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#17 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGdazhOgP00GosqeN-D1yKT_t9RoxTwJks5qXly7gaJpZM4JQ18M
.

@wch

This comment has been minimized.

Show comment
Hide comment
@wch

wch Jul 20, 2016

Contributor

It would be helpful if you would include a reproducible example to start with instead of having us figure out the pieces bit by bit.

Contributor

wch commented Jul 20, 2016

It would be helpful if you would include a reproducible example to start with instead of having us figure out the pieces bit by bit.

@wch

This comment has been minimized.

Show comment
Hide comment
@wch

wch Jul 20, 2016

Contributor

This set of commands works fine for me:

docker images rocker/shiny
docker run --rm -ti rocker/shiny /bin/bash
R

Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")

rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette",
    package = "rmarkdown", edit = FALSE)

rmarkdown::render('my_article.Rmd')

Sys.unsetenv("LC_ALL")
rmarkdown::render('my_article.Rmd')

Sys.unsetenv("LANG")
rmarkdown::render('my_article.Rmd')

Except that, again, it still uses /usr/bin/pandoc. If you want us to go any further, we'll need a reproducible example.

Contributor

wch commented Jul 20, 2016

This set of commands works fine for me:

docker images rocker/shiny
docker run --rm -ti rocker/shiny /bin/bash
R

Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")

rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette",
    package = "rmarkdown", edit = FALSE)

rmarkdown::render('my_article.Rmd')

Sys.unsetenv("LC_ALL")
rmarkdown::render('my_article.Rmd')

Sys.unsetenv("LANG")
rmarkdown::render('my_article.Rmd')

Except that, again, it still uses /usr/bin/pandoc. If you want us to go any further, we'll need a reproducible example.

@MarcinKosinski

This comment has been minimized.

Show comment
Hide comment
@MarcinKosinski

MarcinKosinski Jul 21, 2016

It would be helpful if you would include a reproducible example to start with instead of having us figure out the pieces bit by bit.

I think I have. In the first comment.

I really appreciate the work you put to provide such useful tools. I really enjoy them and use them. They are almost basic tools in my regular work, so opening an issue is a small attempt to payback.

Mainly I see 2 problems.

1) Issue with pandoc (THAT COMES WITH SHINY SERVER)

The below command never stops (unless I press Ctrl+C many times)

mkosinski@mkosinski-E540:~$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        3 weeks ago         1.525 GB
r-base              latest              d71dbb88b62d        3 weeks ago         960.3 MB
mkosinski@mkosinski-E540:~$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
95139ad68706        rocker/shiny        "/usr/bin/shiny-serve"   22 hours ago        Up 22 hours         0.0.0.0:3838->3838/tcp   small_leavitt
mkosinski@mkosinski-E540:~$ docker exec -it 95139ad68706 /bin/bash
root@95139ad68706:/# unset LC_ALL 
root@95139ad68706:/# unset LANG
root@95139ad68706:/# /opt/shiny-server/ext/pandoc/pandoc --version

2) problem with render caused by pandoc in specific case

Specific case: shiny user with specific LANG and LC_ALL

Code is below

docker images
docker ps
docker exec -it 95139ad68706 /bin/bash
sudo su shiny 
export LANG=pl_PL.UTF-8                                                        
export LC_ALL=en_US.UTF-8
R
Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")

rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette",
    package = "rmarkdown", edit = FALSE)

rmarkdown::render('my_article.Rmd')

effect

mkosinski@mkosinski-E540:~$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        3 weeks ago         1.525 GB
r-base              latest              d71dbb88b62d        3 weeks ago         960.3 MB
mkosinski@mkosinski-E540:~$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
95139ad68706        rocker/shiny        "/usr/bin/shiny-serve"   22 hours ago        Up 22 hours         0.0.0.0:3838->3838/tcp   small_leavitt
mkosinski@mkosinski-E540:~$ docker exec -it 95139ad68706 /bin/bash
root@95139ad68706:/# sudo su shiny                                                                                                                                                                                                    
$ export LANG=pl_PL.UTF-8                                                        
$ export LC_ALL=en_US.UTF-8
$ R

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")
> 
> rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette",
+     package = "rmarkdown", edit = FALSE)
Error in rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette",  : 
  The file 'my_article.Rmd' already exists.
> 
> rmarkdown::render('my_article.Rmd')

this freezes and when I type Ctrl+C many times i get such a warning

^C^CError in strsplit(version_info, "\n")[[1]] : subscript out of bounds

In my opinion in this case
LC_ALL is not unset in this chunk https://github.com/rstudio/rmarkdown/blob/master/R/pandoc.R#L599-L603
and LANG is not changed in this chunk https://github.com/rstudio/rmarkdown/blob/master/R/pandoc.R#L619-L623 because you only check identical(Sys.getenv("LANG"), "en_US")

I wish it may help you somehow. I got stack for a few days while running your docker with shiny user but with my default LC_ALL and LANG options before I have figured it out that my LC_ALL and LANG options my not satisfy checks in with_pandoc_safe_environment

MarcinKosinski commented Jul 21, 2016

It would be helpful if you would include a reproducible example to start with instead of having us figure out the pieces bit by bit.

I think I have. In the first comment.

I really appreciate the work you put to provide such useful tools. I really enjoy them and use them. They are almost basic tools in my regular work, so opening an issue is a small attempt to payback.

Mainly I see 2 problems.

1) Issue with pandoc (THAT COMES WITH SHINY SERVER)

The below command never stops (unless I press Ctrl+C many times)

mkosinski@mkosinski-E540:~$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        3 weeks ago         1.525 GB
r-base              latest              d71dbb88b62d        3 weeks ago         960.3 MB
mkosinski@mkosinski-E540:~$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
95139ad68706        rocker/shiny        "/usr/bin/shiny-serve"   22 hours ago        Up 22 hours         0.0.0.0:3838->3838/tcp   small_leavitt
mkosinski@mkosinski-E540:~$ docker exec -it 95139ad68706 /bin/bash
root@95139ad68706:/# unset LC_ALL 
root@95139ad68706:/# unset LANG
root@95139ad68706:/# /opt/shiny-server/ext/pandoc/pandoc --version

2) problem with render caused by pandoc in specific case

Specific case: shiny user with specific LANG and LC_ALL

Code is below

docker images
docker ps
docker exec -it 95139ad68706 /bin/bash
sudo su shiny 
export LANG=pl_PL.UTF-8                                                        
export LC_ALL=en_US.UTF-8
R
Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")

rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette",
    package = "rmarkdown", edit = FALSE)

rmarkdown::render('my_article.Rmd')

effect

mkosinski@mkosinski-E540:~$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
rocker/shiny        latest              fe1a132b5b98        3 weeks ago         1.525 GB
r-base              latest              d71dbb88b62d        3 weeks ago         960.3 MB
mkosinski@mkosinski-E540:~$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
95139ad68706        rocker/shiny        "/usr/bin/shiny-serve"   22 hours ago        Up 22 hours         0.0.0.0:3838->3838/tcp   small_leavitt
mkosinski@mkosinski-E540:~$ docker exec -it 95139ad68706 /bin/bash
root@95139ad68706:/# sudo su shiny                                                                                                                                                                                                    
$ export LANG=pl_PL.UTF-8                                                        
$ export LC_ALL=en_US.UTF-8
$ R

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc")
> 
> rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette",
+     package = "rmarkdown", edit = FALSE)
Error in rmarkdown::draft(file = "my_article.Rmd", template = "html_vignette",  : 
  The file 'my_article.Rmd' already exists.
> 
> rmarkdown::render('my_article.Rmd')

this freezes and when I type Ctrl+C many times i get such a warning

^C^CError in strsplit(version_info, "\n")[[1]] : subscript out of bounds

In my opinion in this case
LC_ALL is not unset in this chunk https://github.com/rstudio/rmarkdown/blob/master/R/pandoc.R#L599-L603
and LANG is not changed in this chunk https://github.com/rstudio/rmarkdown/blob/master/R/pandoc.R#L619-L623 because you only check identical(Sys.getenv("LANG"), "en_US")

I wish it may help you somehow. I got stack for a few days while running your docker with shiny user but with my default LC_ALL and LANG options before I have figured it out that my LC_ALL and LANG options my not satisfy checks in with_pandoc_safe_environment

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Jul 21, 2016

Member

so opening an issue is a small attempt to payback.

What makes you think it is anything but a net drain on our available time and concentration?

Re 1) That invokes a command not in the path. So what?

Re 2) Ditto. Why Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc") ?

If you have non-standard deployment patterns and it breaks, methinks you get to keep the pieces. I still see no issue here but I let @wch close it as he is closest to this project.

Member

eddelbuettel commented Jul 21, 2016

so opening an issue is a small attempt to payback.

What makes you think it is anything but a net drain on our available time and concentration?

Re 1) That invokes a command not in the path. So what?

Re 2) Ditto. Why Sys.setenv(RSTUDIO_PANDOC="/opt/shiny-server/ext/pandoc") ?

If you have non-standard deployment patterns and it breaks, methinks you get to keep the pieces. I still see no issue here but I let @wch close it as he is closest to this project.

@wch

This comment has been minimized.

Show comment
Hide comment
@wch

wch Jul 21, 2016

Contributor

I'd say this is an issue that should be filed with Shiny Server itself (instead of for this Docker image): https://github.com/rstudio/shiny-server/issues

And the solution is probably to update the version of pandoc that's packaged with Shiny Server. In the meantime, you can just use /usr/bin/pandoc instead of the one that comes with Shiny Server.

Contributor

wch commented Jul 21, 2016

I'd say this is an issue that should be filed with Shiny Server itself (instead of for this Docker image): https://github.com/rstudio/shiny-server/issues

And the solution is probably to update the version of pandoc that's packaged with Shiny Server. In the meantime, you can just use /usr/bin/pandoc instead of the one that comes with Shiny Server.

@wch wch closed this Jul 21, 2016

@rocker-org rocker-org locked and limited conversation to collaborators Jul 21, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.