Embed htmlwidgets in Examples #343

saurfang · 2017-05-18T21:51:53Z

I just want to thank you first for such an awesome package. It adds so much flesh to the documentation.

Currently, when HTML and htmlwidgets are called in RStudio, they automatically render in RStudio viewer and fails to be captured by evaluate.

This PR bypasses the print for these two classes and instead save them to a local HTML file. The HTML is then embedded as an iframe in the reference.

p.s. I want to gather your thoughts first while I work on the test cases.

hadley · 2017-05-18T22:05:20Z

Interesting idea! @jjallaire what do you think of this approach?

saurfang · 2017-05-18T22:27:12Z

R/replay-html.r

@@ -113,6 +113,29 @@ replay_html.recordedplot <- function(x, name_prefix, obj_id, ...) {
  paste0("<img src='", escape_html(path), "' alt='' width='540' height='400' />")


on an unrelated note, this fixed aspect ratio can make the plot look a bit funny, although I don't see it to be a problem on ggplot2 🤔

Why? We're setting the graphics device size above.

You are totally right. And I cannot reproduce what I saw anymore 😨 maybe I misremembered.

jjallaire · 2017-05-19T01:10:19Z

This shouldn't be necessary (it's possible to fully capture all of the dependencies and include them in the pkgdown page tag). The entire model of htmlwidgets presumes multiple rendering environments and can definitely be adapted to pkgdown with a bit of extra effort.

jjallaire · 2017-05-19T01:26:46Z

Note that HTML and htmlwidgets do in fact get capture by knitr/evaluate. They end up hitting the knit_print generic which ends up populating the html dependencies knit meta, which is then scooped up by the rendering environment, be it rmarkdown::render, bookdown, RStudio Viewer, or whatever. What's happening in pkgdown is that this info is just dropped on the floor. If we preserve it and inject it into the <head> everything will work as expected.

saurfang · 2017-05-19T01:54:07Z

Thanks for the pointers @jjallaire ! Before I try your approach, can you elaborate a little on the advantage? Is it just against using iframe or is there any other reasons?

It seems it will require propagating HTML dependency attributes all the way through reply_html/as_data/data_reference_topic functions and so wisker can fill a new placeholder in head.html. Are you proposing change evaluate::evaluate output handler to use knitr::knitr_print for all or still need the special check of the class that I am doing now and only do so for htmlwidgets/html for knitr_asis output type.

jjallaire · 2017-05-19T02:15:08Z

iframes are okay if a bit slower due to reloading an entire set of dependencies in each iframe) when htmlwidgets have a fixed size, but many widgets have varying heights (e.g. DT) and it's really, really hard to lay these out in iframes (there are libraries that allow two-way handshaking between iframe parents and iframe content to work out sizing but I don't know how well they work).

I'd argue that the easiest way to get html dependencies working is just to let rmarkdown::render handle the entire page. I realize though that pkgdown has a commitment to whisker and supporting other page composition environments like jekyll so that may not be practical.

Note that you can call rmarkdown::render with run_pandoc = FALSE (https://github.com/rstudio/rmarkdown/blob/master/R/render.R#L26) and you will get back the knit meta context that has all of the html dependencies (this is what both bookdown and the learnr package do). Then those dependencies can be fed though to whisker, etc.

hadley · 2017-05-19T09:46:38Z

@jjallaire so we'd generate an Rmd file including the examples, and then render that instead?

jjallaire · 2017-05-19T10:22:02Z

I see this is a more narrow case than I was assuming it was (I thought we were talking about the use of html/htmlwidgets within pkgdown articles, it looks like this is just related to rendering examples. For this case you wouldn't need to get render involved at all -- if you call knit_print and collect the knit_meta afterwards you'll get the html as well as the dependencies (in the knit_meta) which can then be fed through to whisker. Here are the docs on knit_meta: https://cran.r-project.org/web/packages/knitr/vignettes/knit_print.html Here's another blog post where someone has unrolled enough of this to figure out how to use htmlwidgets with jekyll: https://brendanrocks.com/htmlwidgets-knitr-jekyll/ Here's the technique I mentioned before about using render in two steps to capture the knit_meta then do something with it: x <- rmarkdown::render("file.RMD", run_pandoc = FALSE, clean = FALSE) knit_meta <- attr(x, "knit_meta") rmarkdown::render( input = 'file.knit.md' , knit_meta = knit_meta ) Once you have the dependencies you can do a combination of copying them to directories and building up the right stuff to put in the head tag, e.g. https://github.com/rstudio/htmltools/blob/master/R/html_print.R#L71 If you can possibly call render directly that will make everything much simpler to implement, but hopefully there is enough above to illustrate how to go fully manual.

…

On Fri, May 19, 2017 at 5:46 AM, Hadley Wickham ***@***.***> wrote: @jjallaire <https://github.com/jjallaire> so we'd generate an Rmd file including the examples, and then render that instead? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#343 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGXx7JHyjU8rO6IHEpBrnjS7-t2Bgd5ks5r7WT-gaJpZM4Nf1xV> .

hadley · 2017-05-19T16:08:31Z

@saurfang do you want to have a go at this? You'd need to replace print() with knit_print(), return the extra metadata as an attribute, and then in replay_html() collect all the metadata and combine together. If not, I'll tackle at some point in the future.

saurfang · 2017-05-19T17:16:24Z

Makes sense. Not right now. Maybe next week or so. Thanks for the help!

saurfang · 2017-05-25T06:36:40Z

This almost works. https://saurfang.github.io/pkgdown.example/reference/index.html

Except that DT doesn't render style correctly
https://saurfang.github.io/pkgdown.example/reference/datatable.html
and plotly jams in an unreproducible random id that makes pkgdown keeps generating different HTML (when Rd changes and even if the example output doesn't change).
https://saurfang.github.io/pkgdown.example/reference/ggplotly.html
https://github.com/ropensci/plotly/blob/2907bd40a715e203c3a4f09aa9c63be38cb28274/R/plotly.R#L364
https://github.com/ropensci/plotly/blob/15cd04927b8ee53b562601644e7ac66521c2b965/R/utils.R#L90

I'll take further look at the second issue and polish my test package a little. Any chance you guys have insights to the first one? I banged my head for a little while on this already.

jjallaire · 2017-05-26T13:56:39Z

What issue are you seeing with the DT style? This might simply be a matter of pkgdown adding a couple extra CSS rules for DT (I do this in flexdashboard already so it's not unheard of). If you are doing set.seed then plotly should have no reason to keep generating new IDs. If it is then I think you should take it up with them. J.J.

…

On Thu, May 25, 2017 at 2:36 AM, Forest Fang ***@***.***> wrote: This almost works. https://saurfang.github.io/pkgdown.example/reference/ index.html Except that DT doesn't render style correctly https://saurfang.github.io/pkgdown.example/reference/datatable.html and plotly jams in an unreproducible random id that makes pkgdown keeps generating different HTML (when Rd changes and even if the example output doesn't change). https://saurfang.github.io/pkgdown.example/reference/ggplotly.html https://github.com/ropensci/plotly/blob/2907bd40a715e203c3a4f09aa9c63b e38cb28274/R/plotly.R#L364 https://github.com/ropensci/plotly/blob/15cd04927b8ee53b562601644e7ac6 6521c2b965/R/utils.R#L90 I'll take further look at the second issue and polish my test package a little. Any chance you guys have insights to the first one? I banged my head for a little while on this already. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#343 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGXx_reyHpVwP9Aifx4az00TvWh-9Szks5r9SF4gaJpZM4Nf1xV> .

hadley · 2017-06-12T17:58:35Z

It looks like DT is having problems because it's embedded inside a pre.

Currently when HTML and htmlwidgets are called in RStudio, they automatically render in RStudio viewer and fail to be captured by `evaluate`. This bypasses the `print` for these two classes and instead save them to a local html file. The html is then embedded as an iframe in the reference.

Freeze the outer id to obj_id to avoid crazy diffs.

This might be benefitial in case there are nested widgets in the future.

Use `knit_print` to print examples so that HTML output can be correctly captured. The dependencies are pryed off from `knit_meta`, collected and injected in `head.html` template.

This prevents `pre` tampering style for `knit_asis` output, which is often htlmwidgets and can have strict styling requirments.

saurfang · 2017-06-13T05:08:10Z

You are right. I can see if I close the pre tag, it would render fine. However, I couldn't figure out which style rule is causing it. Would you consider this acceptable though (by closing and open pre just for knit_asis output)?
https://saurfang.github.io/pkgdown.example/reference/datatable.html

The naive implementation leaves an empty pre at the end and I can work to remove. Just want to get some feedback on this approach first.

hadley · 2017-06-13T14:27:14Z

R/rd-data.R

-    expr <- evaluate::evaluate(text, code_env, new_device = TRUE)
+    output_handler <- evaluate::new_output_handler(
+      value = function(x) {
+        if (isS4(x)) {


Is this branch really necessary?

Good point. It looks like the knit_print.default calls knitr:::normal_print() which already took care of this.

hadley · 2017-06-13T14:28:31Z

R/replay-html.r

-    pieces[i] <- replay_html(parts[[i]], obj_id = i, ...)
+    output <- replay_html(parts[[i]], obj_id = i, ...)
+    dependencies[[i]] <- attr(output, "knit_meta")
+    pieces[i] <- output


Should also use [[

I thought we generally use [ for primitive vectors?

hadley · 2017-06-13T14:29:50Z

R/replay-html.r

@@ -113,6 +113,29 @@ replay_html.recordedplot <- function(x, name_prefix, obj_id, ...) {
  paste0("<img src='", escape_html(path), "' alt='' width='540' height='400' />")


Why? We're setting the graphics device size above.

hadley · 2017-06-13T14:31:22Z

R/replay-html.r

@@ -113,6 +137,14 @@ replay_html.recordedplot <- function(x, name_prefix, obj_id, ...) {
  paste0("<img src='", escape_html(path), "' alt='' width='540' height='400' />")
 }

+#' @export
+replay_html.knit_asis <- function(x, name_prefix, obj_id, ...) {
+  output <- paste0("</pre><div class='well knit_asis'>", x, "</div><pre class=\"example\">")


We need to cross-reference this with the example starting and ending tags to make sure it stays consistent.

By "cross-referece", do you mean applying examples css class?

hadley · 2017-06-13T14:32:19Z

inst/templates/head.html

@@ -6,6 +6,9 @@

 <!-- jquery -->
 <script src="https://code.jquery.com/jquery-3.1.0.min.js" integrity="sha384-nrOSfDHtoPMzJHjVTdCopGqIqeYETSXhZDFyniQ8ZHcVy08QesyHcnOUpMpqnmWq" crossorigin="anonymous"></script>
+<script type="text/javascript">
+var jQuery_3_1_0 = $.noConflict(false);


Can we just make this jquery?

I think so because pkgdown.js comes before HTML dependencies so it wouldn't have been overridden yet. I also put it inside an anonymous function to scope it better.

1. Remove import from methods 2. Correctly apply css class and styles 3. Hide empty dangling pre tag 4. Relocate jQuery noConflict call

saurfang

All comments addressed. Everything looks good from my end: https://saurfang.github.io/pkgdown.example/reference/datatable.html

Let me know if you have any other thoughts or concerns. Otherwise, I can squash before merging.

I also filed an issue for plotly's unreproducible object: plotly/plotly.R#1048

saurfang · 2017-06-16T05:00:33Z

R/replay-html.r

@@ -113,6 +113,29 @@ replay_html.recordedplot <- function(x, name_prefix, obj_id, ...) {
  paste0("<img src='", escape_html(path), "' alt='' width='540' height='400' />")


You are totally right. And I cannot reproduce what I saw anymore 😨 maybe I misremembered.

saurfang · 2017-06-16T05:01:52Z

R/rd-data.R

-    expr <- evaluate::evaluate(text, code_env, new_device = TRUE)
+    output_handler <- evaluate::new_output_handler(
+      value = function(x) {
+        if (isS4(x)) {


Good point. It looks like the knit_print.default calls knitr:::normal_print() which already took care of this.

saurfang · 2017-06-16T05:04:44Z

R/replay-html.r

-    pieces[i] <- replay_html(parts[[i]], obj_id = i, ...)
+    output <- replay_html(parts[[i]], obj_id = i, ...)
+    dependencies[[i]] <- attr(output, "knit_meta")
+    pieces[i] <- output


I thought we generally use [ for primitive vectors?

saurfang · 2017-06-16T05:27:33Z

R/replay-html.r

@@ -113,6 +137,14 @@ replay_html.recordedplot <- function(x, name_prefix, obj_id, ...) {
  paste0("<img src='", escape_html(path), "' alt='' width='540' height='400' />")
 }

+#' @export
+replay_html.knit_asis <- function(x, name_prefix, obj_id, ...) {
+  output <- paste0("</pre><div class='well knit_asis'>", x, "</div><pre class=\"example\">")


By "cross-referece", do you mean applying examples css class?

saurfang · 2017-06-16T05:39:54Z

inst/templates/head.html

@@ -6,6 +6,9 @@

 <!-- jquery -->
 <script src="https://code.jquery.com/jquery-3.1.0.min.js" integrity="sha384-nrOSfDHtoPMzJHjVTdCopGqIqeYETSXhZDFyniQ8ZHcVy08QesyHcnOUpMpqnmWq" crossorigin="anonymous"></script>
+<script type="text/javascript">
+var jQuery_3_1_0 = $.noConflict(false);


I think so because pkgdown.js comes before HTML dependencies so it wouldn't have been overridden yet. I also put it inside an anonymous function to scope it better.

hadley · 2017-06-16T12:25:21Z

R/replay-html.r

+#' @export
+replay_html.knit_asis <- function(x, name_prefix, obj_id, ...) {
+  # close and open pre tags because it sometimes breaks stylesheet of htmlwidgets
+  output <- paste0("</pre><div class='examples'><div class='knit_asis'>", x, "</div></div><pre class='examples'>")


By cross-reference, I mean that we've now defined how to open and close and example block in two places. If we forget, and only update one, we'll get a weird display problem that probably won't get noticed for a while.

hadley · 2017-06-16T12:25:39Z

R/replay-html.r

-    pieces[i] <- replay_html(parts[[i]], obj_id = i, ...)
+    output <- replay_html(parts[[i]], obj_id = i, ...)
+    dependencies[[i]] <- attr(output, "knit_meta")
+    pieces[[i]] <- output


The rule is to use [[ whenever you're extracting a single value.

saurfang

Understood. Instead of pasting everything together and wrapping pre in whisker, I pulled the pre wrapping inside so we avoid the open/close hack for HTML pieces.

saurfang · 2017-06-16T21:07:58Z

R/replay-html.r

+    ) %>%
+    purrr::flatten() %>%
+    purrr::map_chr(as.character) %>%
+    unname()


I tried the functional approach, which doesn't look great IMO. If I implement it imperatively by looping through elements and merging, the logic is probably even less readable. I can also implement this as reduce_right by constantly checking the class of the first element in the accumulator. Let me know if you have any suggestions.

saurfang · 2017-06-16T21:18:34Z

inst/templates/content-reference-topic.html

-    <pre class="examples">{{{ . }}}</pre>
+    {{#bundles}}
+    <div class="examples">{{{ . }}}</div>
+    {{/bundles}}


Is there a way in mustache to avoid this nesting?

Right now I had to make it like so so that the anchor for Examples only shows if the examples are present:

{ "examples": { "bundles": [ "...bundle 1...", "...bundle 2...", "..." ] } }

KasperSkytte · 2017-10-23T14:32:56Z

Hi!

First, thanks for an awesome package! It takes documentation to the next level!
I'm not really into the details you discuss here, but I seem to have a similar issue with ggplotly objects in the examples section of a function not showing up, it gets rendered in the RStudio viewer instead. It works fine if embedded in a markdown page in Articles, however, but then the current section indicated at the sidebar menu doesn't highlight when scrolling down the page. I can provide more information about what I've done if needed, but I believe it should work as-is when simply running ggplotly(any-ggplot2-object) within the examples section of a function?

I'm running Ubuntu 16.04 LTS with everything updated to the newest version as of today.

saurfang · 2017-10-24T05:44:44Z

This was in a working state minus the parts I mentioned before that were beyond pkgdown's control. I can rebase this if Hadley has the bandwidth to review and merge this.

In your case, with this PR plotly widgets will render correctly but the source will change during every render due to unreproducible random seed used to generate widget id.

hadley · 2018-04-24T00:27:04Z

I'm going to close this since pkgdown implementation has now changed radically since this was proposed. Sorry for not finding the time to merge it earlier!

@saurfang

Based on previous work in #343 by @saurfang Closes #617

saurfang commented May 18, 2017

View reviewed changes

Forest Fang added 6 commits June 12, 2017 22:04

Fix htmlwidgets id

b933144

Freeze the outer id to obj_id to avoid crazy diffs.

Import methods

7242672

Use exported API to freeze widget id

616fb59

This might be benefitial in case there are nested widgets in the future.

Use knit_print and Inject Deps to Header

8e792f1

Use `knit_print` to print examples so that HTML output can be correctly captured. The dependencies are pryed off from `knit_meta`, collected and injected in `head.html` template.

Close and open pre for knit_asis

aa1598b

This prevents `pre` tampering style for `knit_asis` output, which is often htlmwidgets and can have strict styling requirments.

saurfang force-pushed the replay_html branch from c9ac9ae to aa1598b Compare June 13, 2017 05:05

hadley reviewed Jun 13, 2017

View reviewed changes

Address Review Comments

3e5b7b4

1. Remove import from methods 2. Correctly apply css class and styles 3. Hide empty dangling pre tag 4. Relocate jQuery noConflict call

saurfang commented Jun 16, 2017

View reviewed changes

saurfang changed the title ~~[WIP] Replay HTML and htmlwidgets as iframe~~ Embed htmlwidgets in Examples Jun 16, 2017

hadley reviewed Jun 16, 2017

View reviewed changes

Bundle Pieces into pre Instead

831f91e

saurfang commented Jun 16, 2017

View reviewed changes

hadley mentioned this pull request Apr 2, 2018

htmlwidgets - Examples #617

Closed

hadley closed this Apr 24, 2018

jayhesselberth added a commit that referenced this pull request Jun 1, 2018

Use htmlwidgets in examples

461b796

Based on previous work in #343 by @saurfang Closes #617

jayhesselberth mentioned this pull request Jun 1, 2018

Use htmlwidgets in examples #708

Closed

		@@ -113,6 +113,29 @@ replay_html.recordedplot <- function(x, name_prefix, obj_id, ...) {
		paste0("<img src='", escape_html(path), "' alt='' width='540' height='400' />")

Embed htmlwidgets in Examples #343

Embed htmlwidgets in Examples #343

Conversation

saurfang commented May 18, 2017

hadley commented May 18, 2017

saurfang May 18, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jjallaire commented May 19, 2017

jjallaire commented May 19, 2017

saurfang commented May 19, 2017

jjallaire commented May 19, 2017

hadley commented May 19, 2017

jjallaire commented May 19, 2017 via email

hadley commented May 19, 2017

saurfang commented May 19, 2017

saurfang commented May 25, 2017

jjallaire commented May 26, 2017 via email

hadley commented Jun 12, 2017

saurfang commented Jun 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurfang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurfang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saurfang Jun 16, 2017 • edited

Choose a reason for hiding this comment

KasperSkytte commented Oct 23, 2017

saurfang commented Oct 24, 2017

hadley commented Apr 24, 2018

saurfang May 18, 2017 •

edited

saurfang Jun 16, 2017 •

edited