Cooperation with Beaker Browser

[johnsonlab]

In testing out Beaker Browser I thought I'd copy over the "docs" output of one of my workflowr-generated sites. It worked flawlessly. This may be of real utility for sharing wflowr generated sites. Would love to hear strategies to keep the Beaker Browser folder updated...could you create and populate a BB site inside the workflowr "docs" file?

[jdblischak]

@johnsonlab Thanks for using workflowr and sharing this interesting idea!

You are correct that the directory docs/ contains a standalone static website. The default setup is to have this website hosted for free with GitHub Pages since this service is available to anyone with a GitHub account. However, you can of course copy this directory to another server if you'd prefer. For example, let's say you had a shared server that hosted lab resources, you could clone your workflowr site there (and potentially automate updating it with a CRON job), something like:

git pull origin master
cp -R docs/ /var/www/mysite

For the Beaker Browser, the files appear to be hosted from your local machine. Thus maybe you could create a symlink from your workflowr docs/ directory to the directory where Beaker stores its sites. The line below links the docs/ directory of a hypothetical "workflowr-project" saved in ~/github/ to the hypothetical beaker subdirectory:

ln -s ~/github/workflowr-project/docs ~/beaker/workflowr-project

Hopefully that was somewhat informative. Would you be able to share with me more details about how you were thinking of combining workflowr and the Beaker Browser?

[pfrazee]

@jdblischak I agree with all your saying. We're changing Beaker's behaviors a bit, so I can't say for sure what the final flow would be, but it'll be something like:

• Use the dat CLI to publish the output, or
• Create a site in Beaker and then import the output

[johnsonlab]

Thanks to both of you above. @jdblischak- that is a super informative answer, thanks particularly for the symlink line.

My interest in combining the two is based on the relative apparent security of a Beaker Browser "dat://key" site. We're a wet biology lab with a number of collaborations going at any time. We use R/Rstudio to analyze our data and produce reports for sharing data with our collaborating labs. Storing some of these data on a third-party site like GitHub, however secure, is not optimal for some projects. I think it would be ideal to share these reports directly, BB style. In summary our needs are:

Wet lab data -> Live, up to the minute R/Rstudio/knitr "reproducible research caliber" report -> share with collaborator in a secure fashion.

It's hard enough to convince the unfamiliar to sign up for GitHub, and I anticipate meeting resistance when I begin to ask collaborators to install Beaker Browser also. To get around this, I attempted to open a test shared Beaker Browser site by using https://datproject.org/dat://site_key, but no luck.

Question:

@pfrazee, is it possible to share a link to a self-hosted dat:// site with someone that's using a non-Beaker Browser, like Chrome? Just for viewing the data report? Thanks very much!

[pcarbo]

Just wanted to say I'm following this conversation and I'm finding this direction very intriguing (though I agree the technical aspects are a bit of a barrier).

[pfrazee]

@pfrazee, is it possible to share a link to a self-hosted dat:// site with someone that's using a non-Beaker Browser, like Chrome? Just for viewing the data report? Thanks very much!

@johnsonlab You can do that using an HTTPS mirror. We have two of them:

• https://github.com/beakerbrowser/dathttpd for self deployment
• https://hashbase.io for a hosted version

The Beaker & Dat ecosystem is moving fast but it's still young. The goal is to be super easy for anybody to understand; realistically we're still a few steps away from that.

Just to double check: your team isn't comfortable with the command line, are they?

[johnsonlab]

Wonderful, thanks again @pfrazee. I will share progress at my end right here and will summarize things in a blog post at https://johnsonlab.github.io/year-archive/. I will share a dat:// link to an example "report site", generated in Rstudio and kept up to date using the @jdblischak symlink suggestion.

Team members are definitely not command-line-comfortable, and would be absolutely thrilled with the ability to click a link to see a secure report that's self hosted by me (apparently, a la dathttpd). This will meet my needs for complete control of where the data are hosted but allow the team to keep updated. This may be niche workflow for scientists, but seeing @pcarbo chime in makes me think it could be useful to more than just us two!

[pfrazee]

Awesome, feel free to ping me for help in the future. I'm sure there are ways we can improve the flow over time. To that end -- The Dat team would love to know about your usecase; they made Dat with scientists and academia in mind. If you use IRC, feel free to stop by #dat in freenode sometime.

…

On Tue, Aug 22, 2017 at 7:32 PM, Josh Johnson ***@***.***> wrote: Wonderful, thanks again @pfrazee <https://github.com/pfrazee>. I will share progress at my end right here and will summarize things in a blog post at https://johnsonlab.github.io/year-archive/. I will share a dat:// link to an example "report site", generated in Rstudio and kept up to date using the @jdblischak <https://github.com/jdblischak> symlink suggestion. Team members are definitely not command-line-comfortable, and would be absolutely thrilled with the ability to click a link to see a secure report that's self hosted by me (apparently, a la dathttpd). This will meet my needs for complete control of where the data are hosted but allow the team to keep updated. This may be niche workflow for scientists, but seeing @pcarbo <https://github.com/pcarbo> chime in makes me think it could be useful to more than just us two! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABNhU9H5PzFgqwvLCBTPCMOL-LZbiMTAks5sa3MUgaJpZM4O-B_D> .

[jdblischak]

I am happy to help facilitate this use case. The workflowr functions are mainly focused on combining version control (via libgit2/git2r) and literate programming (via knitr/rmarkdown). Thus I'd imagine I would contribute to this effort by adding instructions to document the process of hosting via Beaker Browser instead of GitHub Pages. I know there are many researchers that are uneasy with sharing everything, especially at the beginning of a project, so I'd welcome a solution for easily hosting/sharing private websites.

@johnsonlab and @pfrazee Please keep me updated with any progress and let me know if you'd like me to test something out.

[pfrazee]

Sounds good, John.

…

On Wed, Aug 23, 2017 at 9:16 AM, John Blischak ***@***.***> wrote: I am happy to help facilitate this use case. The workflowr functions are mainly focused on combining version control (via libgit2/git2r) and literate programming (via knitr/rmarkdown). Thus I'd imagine I would contribute to this effort by adding instructions to document the process of hosting via Beaker Browser instead of GitHub Pages. I know there are many researchers that are uneasy with sharing everything, especially at the beginning of a project, so I'd welcome a solution for easily hosting/sharing private websites. @johnsonlab <https://github.com/johnsonlab> and @pfrazee <https://github.com/pfrazee> Please keep me updated with any progress and let me know if you'd like me to test something out. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABNhUx-23wAkfBu_Xuc9rbr4QGnYAnzPks5sbDRbgaJpZM4O-B_D> .

[johnsonlab]

OK, I've produced an example workflowr site by importing workflowr "docs" to Beaker Browser (including the symlink between the folder of origin and the BB Site folder) and published to:

dat://adef21aa8bbac5e93b0c20a97c6f57f93150cf4e7f5eb1eb522eb88e682309bc

The experience is written up in the blog post https://johnsonlab.github.io/blog-post-22/. Maybe I'm closer to being able to share data in this way than I think I am. Thanks everyone!! JJ

[jdblischak]

OK, I've produced an example workflowr site by importing workflowr "docs" to Beaker Browser (including the symlink between the folder of origin and the BB Site folder) and published to:

dat://adef21aa8bbac5e93b0c20a97c6f57f93150cf4e7f5eb1eb522eb88e682309bc

@johnsonlab This is great! I installed Beaker Browser and was able to access your workflowr site.

The experience is written up in the blog post https://johnsonlab.github.io/blog-post-22/. Maybe I'm closer to being able to share data in this way than I think I am. Thanks everyone!! JJ

This is also great! A few comments on your blog post:

• You are not the first person to be interested in PDF output, so I'll open an issue to create a function to do this. The formatting will be cruder than the nicely formatted website, but it'll work well enough when you really need a PDF. For the time being you can always create a PDF version of any of your workflowr pages by directly running the rmarkdown function render("analysis/file.Rmd", pdf_document()).

• Once you have a useable worflow combining workflowr and Beaker Browser, would you be interested in writing an answer for a workflowr FAQ? I was thinking the question would be something like: "Is it possible to create a private website that only my collaborators can access?"

• In your blog post, you wrote that I had mentioned hashbase.io. Since it was @pfrazee that mentioned it (and it is his project), could you please correct this?

[johnsonlab]

Hey! I have updated the blog post in line with your above comments and corrections:

1. added mention of the render(...) command.
2. ABSOLUTELY I would be very happy to provide this answer for the FAQ.
3. corrected to @pfrazee
   Really pleased with the workflow as is, and more widespread adoption of Beaker Browser and dat:// will make it easy to share data securely and directly!

[jdblischak]

@johnsonlab Great. I'm going to close this issue since the central issue has been resolved, namely that combining workflowr output with Beaker Browser distribution seems promising for those researchers that can't make their data public from the start of the project.

For your future PR, here are some guidelines:

• Please cross-reference this Issue in the PR description
• If possible, create the change on the "dev" branch (if you aren't comfortable with branching, then don't worry about this. I don't want a technical barrier to prevent you from submitting content, and I can take care of the technical part on my end)
• You don't have to re-build the documentation for the PR. You can just submit the changes to vignettes/wflow-05-faq.Rmd

Please feel free to continue to ask questions on this Issue even though it is closed. I'm closing it to reduce the amount of Issues on my "to-do" list, not to shut down the conversation. Thanks!