Compatibility with shiny server

[RobinL]

I have been trying to develop a Shiny Server Docker image that is compatible with govdown.

The idea is that our analysts can produce an index.Rmd with the govdown yaml header, and deploy the Docker image to our analytical platform. On accessing the website running from the shiny server, their R Markdown document should render.

At the moment, in the logs you see something like:

processing file: index.Rmd
output file: /tmp/RtmpX5VcKZ/index.knit.md

Navigating to the output file, the file has rendered correctly and has all the assets (css etc). I think Shiny Server just somehow fails to then serve the html file to the browser.

which is the same as you see with the standard output: html_document. But then from the user's point of view, the shiny server just hangs on:

I suspect this may be something to do with passing through the arguments that shiny server is passing to rmarkdown:run.

I'd be happy to attempt at PR to fix this but might be worth a quick Slack convo first - would be good to get your input to see if I'd be starting off in the right direction

I'd be very happy to discuss and attempt a PR.

[RobinL]

Here's the current branch where I try to get it working:
https://github.com/moj-analytical-services/rmarkdown-vegawidget-template/tree/themed

[RobinL]

Shiny Server logs for the version using

output: html_document
runtime: shiny

look like this:

processing file: index.Rmd
output file: /tmp/Rtmp5e0zrZ/index.knit.md

Output created: /tmp/Rtmp5e0zrZ/file1310ca6bbe.html

Note we seem to need runtime: shiny to invalidate the cache on page refresh (we need this because the charts are being generated from csvs stored in s3 that update on a schedule.

Shiny server output for the version with the output set as the govdown yaml block, and runtime: shiny:

processing file: index.Rmd
output file: /tmp/RtmptFONVG/index.knit.md


Output created: /tmp/RtmptFONVG/file1446788fa4.html

Shiny server output for the version with the output set as the govdown yaml block, without runtime: shiny

processing file: index.Rmd
output file: /tmp/shiny/rmarkdown/8666bdd9ddb4ad0b0a6ed607242b788b/index.knit.md


Output created: /tmp/shiny/rmarkdown/8666bdd9ddb4ad0b0a6ed607242b788b/rmd_8666bdd9ddb4ad0b0a6ed607242b788b.html

Shiny server gets stuck on the loading screen either way

[RobinL]

Note on how Shiny server works, from here:

If a hosted directory does not include an app.R or a server.R file, Shiny Server will look to see if it contains any .Rmd files. If so, Shiny Server will host that directory in “R Markdown” mode using rmarkdown::run().

[RobinL]

Comparison of govdown and html_document
returned rmarkdown::output_formats

[RobinL]

This may be relevant: https://github.com/rstudio/shiny-server/blob/4b588a12f844713729991d1274764d854d90a18d/R/SockJSAdapter.R#L246

[RobinL]

from ?rmarkdown::run:
Note
Unlike render, run does not render the document to a file on disk. In most cases a Web browser will be started automatically to view the document; see launch.browser in the runApp documentation for details.

When using an external web browser with the server, specify the name of the R Markdown file to view in the URL (e.g. http://127.0.0.1:1234/foo.Rmd). A URL without a filename will show the default_file as described above.

[RobinL]

Possibly next step is to create a trivial govdown::govdown_document-esque function that contains no functionality (simply passes through its arguments to html_document ) and see if that works, then change the function until it breaks

[RobinL]

Perhaps not surprisingly, everything works fine with a custom package like:

Package called htmldocpass

my_fn <- function(...){
  rmarkdown::html_document(...)
}

Header:

title: "Template document"
author: "Robin Linacre"
date: "13/07/2019"
output:
  htmldocpass::my_fn
runtime: shiny

[RobinL]

This seems potentially relevant:
https://github.com/smckinney2718/RStudio/blob/b1622816da0200c4e4f8a38e681a7a72ae156853/src/cpp/session/modules/SessionRMarkdown.R#L53

[RobinL]

It's quite instructive It's quite instructive to add a debug i.e.:

my_fn <- function(...){
  debug(rmarkdown::html_document)
  rmarkdown::html_document(...)
}

This is what we see:

[RobinL]

Repeating the same exericise with govdown i.e. with

---
title: "Template document"
author: "Robin Linacre"
date: "13/07/2019"
output:
  htmldocpass::my_fn
runtime: shiny
---

and the package containing:

my_fn <- function(...){
  debug(govdown::govdown_document)
  govdown::govdown_document(...)
}

[RobinL]

HOLY MOLY I THINK I FIXED IT!!111 It only took 9 hours. I am a 0.01x programmer. 8 character PR incoming...

[nacnudus]

That was a thrilling read. I'm so sorry you had such a torrid time with it -- blame me for sweeping optional arguments under the carpet in the first place. I went down similar rabbit holes before deciding it would be simpler not to pass anything, but it looks like this works for you, so let's include it. Thanks for going to all this trouble!

[nacnudus]

Some notes from hours of debugging:

The fix is to add a ... argument to govdown_document(), and pass that onto rmarkdown::html_document_base(). Thanks @RobinL for discovering that.

But the reason why that works is complicated.

1. When an R Markdown document has runtime: shiny in the yaml, you should render it with rmarkdown::run() rather than rmarkdown::render(). RStudio uses rmarkdown::run() by default, so that's all good.
2. rmarkdown::run() creates a function called dependency_resolver(), and passes it into the output_options argument of rmarkdown::render(), which passes it into output_options of whatever format -- in this case govdown_format().
3. RMarkdown expects govdown_format() to pass the dependency_resolver() function into html_document_base(). If html_document_base() doesn't receive it, then it uses a default implementation called html_dependency_resolver().

Because govdown_document() didn' t have a dependency_resolver argument, and didn't have ... either, and even if it had, it didn't pass them into html_document_base(), the default implementation was used instead: html_dependency_resolver(). The outcome was that pandoc was told to insert a file into the header that contained links to several javascripts. These made the page appear blank.

<link href="/home/nacnudus/R/x86_64-pc-linux-gnu-library/3.6/rmarkdown/rmd/h/highlightjs/default.css" rel="stylesheet" />
<script src="/home/nacnudus/R/x86_64-pc-linux-gnu-library/3.6/rmarkdown/rmd/h/highlightjs/highlight.js"></script>
<script src="/home/nacnudus/R/x86_64-pc-linux-gnu-library/3.6/htmlwidgets/www/htmlwidgets.js"></script>
<script src="/home/nacnudus/R/x86_64-pc-linux-gnu-library/3.6/vegawidget/htmlwidgets/lib/vega/vega.min.js"></script>
<script src="/home/nacnudus/R/x86_64-pc-linux-gnu-library/3.6/vegawidget/htmlwidgets/lib/vega-lite/vega-lite.min.js"></script>
<link href="/home/nacnudus/R/x86_64-pc-linux-gnu-library/3.6/vegawidget/htmlwidgets/lib/vega-embed/vega-embed.css" rel="stylesheet" />
<script src="/home/nacnudus/R/x86_64-pc-linux-gnu-library/3.6/vegawidget/htmlwidgets/lib/vega-embed/vega-embed.js"></script>
<script src="/home/nacnudus/R/x86_64-pc-linux-gnu-library/3.6/vegawidget/htmlwidgets/vegawidget.js"></script>

Hence the solution is to accept ... into govdown_document() and pass them into html_document_base(). That way, anything shiny and rmarkdown choose to put in will reach html_document_base().

Why didn't we get the usual Error in (function () : unused argument (foo = whatever)? Don't know. Maybe some shiny environment reactive evaluation magic.

[nacnudus]

Hi @RobinL, if it isn't a lot of trouble, please could you test that this still works with the latest commit 04473a4?