diff --git a/blogdown/md5sum.txt b/blogdown/md5sum.txt
index 6e35fdfe6..4c1fee296 100644
--- a/blogdown/md5sum.txt
+++ b/blogdown/md5sum.txt
@@ -60,7 +60,7 @@
 "content/en/sps/dev/spscomps/server.Rmarkdown" "bbac38b3912e031093f84803cc96a3c3"
 "content/en/sps/dev/spscomps/server.Rmd" "bbac38b3912e031093f84803cc96a3c3"
 "content/en/sps/dev/spscomps/ui.Rmarkdown" "b94a428e18b0864e8617baf7ccbc30bc"
-"content/en/sps/dev/spscomps/ui.Rmd" "37f4c5a7f46de3dc0d2b343959ff87a2"
+"content/en/sps/dev/spscomps/ui.Rmd" "13db680921b7d61c46c37d25480c1127"
 "content/en/sps/dev/spsutil.Rmarkdown" "68d2201326131622572726d5fd794fe5"
 "content/en/sps/dev/spsutil.Rmd" "68d2201326131622572726d5fd794fe5"
 "content/en/sps/newtab.Rmarkdown" "4eda65b3a7c124dfb45e21e221958ea7"
diff --git a/content/en/sps/dev/spscomps/_index.md b/content/en/sps/dev/spscomps/_index.md
index 232400a2e..a72bd3012 100644
--- a/content/en/sps/dev/spscomps/_index.md
+++ b/content/en/sps/dev/spscomps/_index.md
@@ -5,14 +5,23 @@ type: docs
 weight: 1
 ---
 
-
 ## SPS Components package
 
 systemPipeShiny Components (spsComps) package is a collection of custom UI  and 
 server components that are used in SPS main framework. If you see a component 
 in SPS but want to outside the SPS framework, like in your own Shiny apps, 
-take a look at these components. Here is a **[online demo](https://lezhang.shinyapps.io/spsComps)**
-of these components. 
+take a look at these components. 
+
+## Demos
+
+| Demo | type | source code |
+|---|---|---|
+|[shiny{blk}](https://lezhang.shinyapps.io/spsComps)|shinyapps.io|[Github{blk}](https://github.com/lz100/spsComps/tree/master/examples/demo)|
+|[Rmd](https://systempipe.org/sps/dev/spscomps/ui/)|Rmarkdown rendered|[Raw{blk}](https://raw.githubusercontent.com/systemPipeR/systemPipeR.github.io/main/content/en/sps/dev/spscomps/ui.Rmd)|
+
+<style>
+table {font-size: 1.5rem}
+</style>
 
 ## Installation 
 
diff --git a/content/en/sps/dev/spscomps/ui.Rmd b/content/en/sps/dev/spscomps/ui.Rmd
index 723fffc6b..58d80f574 100644
--- a/content/en/sps/dev/spscomps/ui.Rmd
+++ b/content/en/sps/dev/spscomps/ui.Rmd
@@ -4,7 +4,7 @@ linkTitle: "UI components"
 type: docs
 weight: 1
 ---
-## UI Components 
+
 For most of the UI components, you can view them in the [online Shiny demo{blk}](https://lezhang.shinyapps.io/spsComps).
 Most but **not all** UI components work in a Rmarkdown document. Here we demostrate 
 how you could use some of them in a Rmarkdown doc. The source code of this 
diff --git a/content/en/sps/dev/spscomps/ui.md b/content/en/sps/dev/spscomps/ui.md
index 29bd61157..ad504edb1 100644
--- a/content/en/sps/dev/spscomps/ui.md
+++ b/content/en/sps/dev/spscomps/ui.md
@@ -85,7 +85,7 @@ weight: 1
 <script src="/rmarkdown-libs/spsComps-css-loader-js/js/sps_cssloader.js"></script>
 
 <link href="/rmarkdown-libs/spsComps-css-loader-css/css/css_loader.css" rel="stylesheet" />
-\#\# UI Components
+
 For most of the UI components, you can view them in the [online Shiny demo{blk}](https://lezhang.shinyapps.io/spsComps).
 Most but **not all** UI components work in a Rmarkdown document. Here we demostrate
 how you could use some of them in a Rmarkdown doc. The source code of this
@@ -169,46 +169,46 @@ gallery(
 )
 ```
 
-<div id="gallery1872830" class="col sps-gallery" style="">
+<div id="gallery2917503" class="col sps-gallery" style="">
 <p class="text-center h2" style="color: #0275d8;">Gallery</p>
-<div class="row" style="  margin: 10px;"><div  id=gallery1872830-1 class="col-sm-4 sps-tab-link" style="right: 1px;">
+<div class="row" style="  margin: 10px;"><div  id=gallery2917503-1 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/1.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-1")
+    onclick=galEnlarge("#gallery2917503-1")
   >
   <a href="https://github.com/lz100/spsComps/blob/master/img/1.jpg?raw=true" ><p class="text-center h4 ">p1</p></a>
-</div> <div  id=gallery1872830-2 class="col-sm-4 sps-tab-link" style="right: 1px;">
+</div> <div  id=gallery2917503-2 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/2.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-2")
+    onclick=galEnlarge("#gallery2917503-2")
   >
   <a href="https://github.com/lz100/spsComps/blob/master/img/2.jpg?raw=true" ><p class="text-center h4 ">p2</p></a>
-</div> <div  id=gallery1872830-3 class="col-sm-4 sps-tab-link" style="right: 1px;">
+</div> <div  id=gallery2917503-3 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/3.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-3")
+    onclick=galEnlarge("#gallery2917503-3")
   >
   <a  ><p class="text-center h4 gallery-nohover">&nbsp;</p></a>
-</div> <div  id=gallery1872830-4 class="col-sm-4 sps-tab-link" style="right: 1px;">
+</div> <div  id=gallery2917503-4 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/4.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-4")
+    onclick=galEnlarge("#gallery2917503-4")
   >
   <a href="https://github.com/lz100/spsComps/blob/master/img/4.jpg?raw=true" ><p class="text-center h4 ">p4</p></a>
-</div> <div  id=gallery1872830-5 class="col-sm-4 sps-tab-link" style="right: 1px;">
+</div> <div  id=gallery2917503-5 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/5.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-5")
+    onclick=galEnlarge("#gallery2917503-5")
   >
   <a href="https://github.com/lz100/spsComps/blob/master/img/5.jpg?raw=true" ><p class="text-center h4 ">p5</p></a>
 </div></div>
@@ -219,7 +219,7 @@ gallery(
 <div class="gallery-caption"></div>
 </div>
 <!--/SHINY.SINGLETON[7fdec3ac87205a269983f4a367a45b9599c89d0b]-->
-<script>fixGalHeight("gallery1872830")</script>
+<script>fixGalHeight("gallery2917503")</script>
 </div>
 
 You can show a gallery of plots you make in the Rmd and when people click it,
@@ -303,7 +303,7 @@ hrefTab(
 )
 ```
 
-<div id="list-tab6143613" class="col">
+<div id="list-tab9709371" class="col">
 <p class="h4" style="color: #0275d8; text-align: left;">Different background and text colors</p>
 <div><a
   href="#" 
@@ -343,7 +343,7 @@ hrefTable(
 )
 ```
 
-<table id="list-table2346973" class="table table-hover table-href table-striped" style="display: table;">
+<table id="list-table9981148" class="table table-hover table-href table-striped" style="display: table;">
 <caption class="text-center h2" style="color: #0275d8;">Change button color and text color</caption>
 <thead>
   <tr class="info">
@@ -395,7 +395,7 @@ hrefTable(
 )
 ```
 
-<table id="list-table7826614" class="table table-hover table-href table-striped" style="display: table;">
+<table id="list-table7341719" class="table table-hover table-href table-striped" style="display: table;">
 <caption class="text-center h2" style="color: #0275d8;">Change row name colors and width</caption>
 <thead>
   <tr class="info">
@@ -754,10 +754,10 @@ There are 12 different default loaders: “circle,” “dual-ring,” “facebo
 cssLoader(height = "100px")
 ```
 
-<div id="spsloader-594362718" class="sps-cssloader" style="height: 100px; width: 100px; display: block; opacity: 1;">
+<div id="spsloader-857314629" class="sps-cssloader" style="height: 100px; width: 100px; display: block; opacity: 1;">
 <script>
   $(function(){
-    $("#spsloader-594362718").prepend(chooseLoader("spsloader-594362718", "default", "", "#337ab7", "100px", "100px"));
+    $("#spsloader-857314629").prepend(chooseLoader("spsloader-857314629", "default", "", "#337ab7", "100px", "100px"));
   });
 </script>
 </div>
@@ -770,10 +770,10 @@ customize it:
 cssLoader(type = "grid", height = "150px", color = "orange")
 ```
 
-<div id="spsloader-917283456" class="sps-cssloader" style="height: 150px; width: 150px; display: block; opacity: 1;">
+<div id="spsloader-481753296" class="sps-cssloader" style="height: 150px; width: 150px; display: block; opacity: 1;">
 <script>
   $(function(){
-    $("#spsloader-917283456").prepend(chooseLoader("spsloader-917283456", "grid", "", "orange", "150px", "150px"));
+    $("#spsloader-481753296").prepend(chooseLoader("spsloader-481753296", "grid", "", "orange", "150px", "150px"));
   });
 </script>
 </div>
@@ -792,10 +792,10 @@ tags$button(
 ```
 
 <button>
-<i id="spsloader-561273984" class="sps-cssloader" style="height: 1.5rem; width: 1.5rem; display: inline-block; opacity: 1;">
+<i id="spsloader-369175824" class="sps-cssloader" style="height: 1.5rem; width: 1.5rem; display: inline-block; opacity: 1;">
 <script>
   $(function(){
-    $("#spsloader-561273984").prepend(chooseLoader("spsloader-561273984", "default", "", "#3a7bd5", "1.5rem", "1.5rem"));
+    $("#spsloader-369175824").prepend(chooseLoader("spsloader-369175824", "default", "", "#3a7bd5", "1.5rem", "1.5rem"));
   });
 </script>
 </i>
diff --git a/content/en/sps/sps_funcs.md b/content/en/sps/sps_funcs.md
index 7843d453c..4b512615b 100644
--- a/content/en/sps/sps_funcs.md
+++ b/content/en/sps/sps_funcs.md
@@ -20,13 +20,16 @@ click links below to go to the function reference manual of each package. Detail
 function usage, example code, and example running results are shown inside. 
 
 
-- [systemPipeShiny -- main package](/sps/funcs/sps/reference)
+|Package|Description|
+|---|---|
+|[systemPipeShiny](/sps/funcs/sps/reference)|main package|
+|[spsComps](/sps/funcs/spscomps/reference)|SPS components|
+|[drawer](/sps/funcs/drawer/reference)|image editing tool|
+|[spsUtil](/sps/funcs/spsutil/reference)|utility functions|
 
-- [spsComps -- SPS components](/sps/funcs/spscomps/reference)
-
-- [drawer -- image editing tool](/sps/funcs/drawer/reference)
-
-- [spsUtil -- utility functions ](/sps/funcs/spsutil/reference)
+<style>
+table {font-size: 1.5rem}
+</style>
 
 *****
 
diff --git a/public/404.html b/public/404.html
index 58c24bf32..db33e94e8 100644
--- a/public/404.html
+++ b/public/404.html
@@ -492,7 +492,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="./offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="./offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="./"
   data-offline-search-max-results="10"
 >
diff --git a/public/about/acknowledgement/index.html b/public/about/acknowledgement/index.html
index c2f0407e7..a5e9a538f 100644
--- a/public/about/acknowledgement/index.html
+++ b/public/about/acknowledgement/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/about/contact/index.html b/public/about/contact/index.html
index 401affd22..675fd6666 100644
--- a/public/about/contact/index.html
+++ b/public/about/contact/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/about/index.html b/public/about/index.html
index 615575a4b..5e222559f 100644
--- a/public/about/index.html
+++ b/public/about/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/about/overview/index.html b/public/about/overview/index.html
index 2ac91d8ee..ee4cae13b 100644
--- a/public/about/overview/index.html
+++ b/public/about/overview/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/about/publications/index.html b/public/about/publications/index.html
index dc8051ed3..955a72a53 100644
--- a/public/about/publications/index.html
+++ b/public/about/publications/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/about/team/index.html b/public/about/team/index.html
index c17b32c96..661292889 100644
--- a/public/about/team/index.html
+++ b/public/about/team/index.html
@@ -526,7 +526,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -558,7 +558,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/community/index.html b/public/community/index.html
index 2362fa7df..4fa678bc3 100644
--- a/public/community/index.html
+++ b/public/community/index.html
@@ -493,7 +493,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/index.html b/public/index.html
index 5588826ef..fd104442f 100644
--- a/public/index.html
+++ b/public/index.html
@@ -493,7 +493,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="./offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="./offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="./"
   data-offline-search-max-results="10"
 >
diff --git a/public/news/index.html b/public/news/index.html
index 948a22c8e..a84c7e855 100644
--- a/public/news/index.html
+++ b/public/news/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/news/release_3_12/index.html b/public/news/release_3_12/index.html
index c1afbf2e6..dced5db8c 100644
--- a/public/news/release_3_12/index.html
+++ b/public/news/release_3_12/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/offline-search-index.8b789866927612b8a1270b2bdea1abcb.json b/public/offline-search-index.8b789866927612b8a1270b2bdea1abcb.json
new file mode 100644
index 000000000..981433b7c
--- /dev/null
+++ b/public/offline-search-index.8b789866927612b8a1270b2bdea1abcb.json
@@ -0,0 +1,59 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+[{"body":"SPS has many options you can change. These options will determine how the app behaves.\n Config SPS Let us start by creating an example SPS project. For demo purpose, we are using the /tmp folder but one should use a regular location instead of the temp in a real case.\nsuppressPackageStartupMessages(library(systemPipeShiny)) spsInit(app_path = tempdir(), project_name = \"config_demo\", overwrite = TRUE, change_wd = FALSE)  ## [SPS-DANGER] 2021-04-25 19:43:24 Done, Db created at '/tmp/RtmphrNOOX/config_demo/config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-25 19:43:24 Key md5 ee81377a81fdb757fbbbb6af38644756 ## [SPS-INFO] 2021-04-25 19:43:24 SPS project setup done!  ## save project path (sps_dir \u003c- file.path(tempdir(), \"config_demo\"))  ## [1] \"/tmp/RtmphrNOOX/config_demo\"  To reproduce code locally, run the following chunk instead.\nlibrary(systemPipeShiny) spsInit() sps_dir \u003c- normalizePath(\".\")  PRE.fansi SPAN {padding-top: .25em; padding-bottom: .25em};  SPS structure  SPS_xx/ ├── server.R | ├── global.R | Most important server, UI and global files, unless special needs, `global.R` is the only file you need to edit ├── ui.R | ├── deploy.R | Deploy helper file ├── config | Important app config files. Do not edit them if you don't know │ ├── sps.db | SPS database │ ├── sps_options.yaml | SPS default option list │ └── tabs.csv | SPS tab registration information ├── data | App example data files │ ├── xx.csv ├── R | All SPS custom tab files and helper R function files │ ├── tab_xx.R ├── README.md ├── results | To store data generated from the app, like the workflow module │ └── README.md └── www | Internet resources ├── css | CSS files │ └── sps.css ├── img | App image resources │ └── xx.png ├── js | Javascripts │ └── xx.js ├── loading_themes | Loading screen files │ └── xx.html └── plot_list | Image files for plot gallery └── plot_xx.jpg  This is a reminder of what you will get when a SPS project is initiated with spsInit().\n For most users, the global.R file is the only file that one needs to make change. The second important files are the files inside config folder. For normal users, these files are controlled by SPS functions. No need to make any modification. For advanced users, deep customization is possible.  sps_options.yaml stores all default and valid values for SPS, details are listed below tabs.csv all SPS tab registration information. Read Manage tabs sps.db A SQLite database to store data generated in SPS. Read SPS database for more information.   R folder stores all custom tab files, your helper functions. This .R or .r files under this folder will be automatically sourced when SPS starts. This is discussed in Manage tabs. www folder is where you add the internet resources, like images you want to show in the app, css style sheets to attach. Read more here.  App options View/Set all options App options in SPS are controlled by “SPS options”. These options can change app appearance, debugging level, server behaviors, etc. The valid options can be found and change on the global.R file. They are similar to Shiny options, but unlike shiny options that are single values, SPS options are passed using the Option(sps = list(...)) function in global.R as a group. To view all options and their default, valid values('*' means any value is okay) see global.R from the line starting with ## SPS options. We provided some comments below that line to generally describe what each option is and valid values for options. Use function spsOptDefaults to see the default and other valid options.\nspsOptDefaults(app_path = sps_dir)  ## title: ## Default: systemPipeShiny ## Other: * ## title_logo: ## Default: img/sps_small.png ## Other: * ## mode: ## Default: local ## Other: server ## login_screen: ## Default: FALSE ## Other: TRUE ## login_theme: ## Default: random ## Other: * ## use_crayon: ## Default: TRUE ## Other: FALSE ## verbose: ## Default: FALSE ## Other: TRUE ## admin_page: ## Default: TRUE ## Other: FALSE ## admin_url: ## Default: admin ## Other: * ## warning_toast: ## Default: FALSE ## Other: TRUE ## module_wf: ## Default: TRUE ## Other: FALSE ## module_rnaseq: ## Default: TRUE ## Other: FALSE ## module_ggplot: ## Default: TRUE ## Other: FALSE ## tab_welcome: ## Default: TRUE ## Other: FALSE ## tab_vs_main: ## Default: TRUE ## Other: FALSE ## tab_canvas: ## Default: TRUE ## Other: FALSE ## tab_about: ## Default: TRUE ## Other: FALSE ## note_url: ## Default: https://raw.githubusercontent.com/systemPipeR/systemPipeShiny/master/inst/remote_resource/notifications.yaml ## Other: * ## traceback: ## Default: FALSE ## Other: TRUE ## is_demo: ## Default: FALSE ## Other: TRUE ## welcome_guide: ## Default: TRUE ## Other: FALSE ## * means any value will be accepted  After the app has started once, you can use spsOptions() to see all current settings.\nspsOptions(app_path = sps_dir)  ## Current project option settings: ## title: ## systemPipeShiny  ## title_logo: ## img/sps_small.png  ## mode: ## local  ## login_screen: ## FALSE  ## login_theme: ## random  ## use_crayon: ## TRUE  ## verbose: ## FALSE  ## admin_page: ## TRUE  ## admin_url: ## admin  ## warning_toast: ## FALSE  ## module_wf: ## TRUE  ## module_rnaseq: ## TRUE  ## module_ggplot: ## TRUE  ## tab_welcome: ## TRUE  ## tab_vs_main: ## TRUE  ## tab_canvas: ## TRUE  ## tab_about: ## TRUE  ## note_url: ## https://raw.githubusercontent.com/systemPipeR/systemPipeShiny/master/inst/remote_resource/notifications.yaml  ## traceback: ## FALSE  ## is_demo: ## FALSE  ## welcome_guide: ## TRUE  ## ******** ## Option legend: ## known options Hidden/custom options* and values+ ## Value legend: ## same as default values  different from defaults+  A copy of options in global.R:\noptions(sps = list( title = \"systemPipeShiny\", title_logo = \"img/sps_small.png\", mode = \"local\", warning_toast = FALSE, login_screen = FALSE, login_theme = \"random\", use_crayon = TRUE, verbose = FALSE, admin_page = TRUE, admin_url = \"admin\", note_url = 'https://raw.githubusercontent.com/systemPipeR/systemPipeShiny/master/inst/remote_resource/notifications.yaml', tab_welcome = TRUE, tab_vs_main = TRUE, tab_canvas = TRUE, tab_about = TRUE, module_wf = TRUE, module_rnaseq = TRUE, module_ggplot = TRUE, traceback = FALSE, is_demo = FALSE, welcome_guide = TRUE ))  Note: Do not worry if you set some invalid values, on app start, `sps()` will check all SPS options, ignore unknown values and set invalid values back to default. You will see warning messages on console to tell you specifically what is wrong with your options.\n   Option Description Default Other     mode running mode “local” “server”   title App title “systemPipeShiny” any string   title_logo App logo to display on browser tab “img/sps_small.png” any path   warning_toast show security warnings? TRUE FALSE   login_screen add login screen? TRUE FALSE   login_theme login screen theme “random” see details   use_crayon colorful console message? TRUE FALSE   verbose more details for SPS functions? FALSE TRUE   admin_page enable admin page? FALSE TRUE   admin_url admin_page query url “admin” any string   warning_toast for internal test only TRUE FALSE   module_wf load workflow module? TRUE FALSE   module_rnaseq load RNAseq module? TRUE FALSE   module_ggplot load quick ggplot module? TRUE FALSE   tab_welcome load welcome tab? TRUE FALSE   tab_vs_main load custom visualization main tab? TRUE FALSE   tab_canvas load Canvas tab? TRUE FALSE   tab_about load about tab? TRUE FALSE   note_url SPS notification remote URL see code above any URL   is_demo useful if deploy the app as a demo FALSE TRUE   welcome_guide enable the welcome guide TRUE FALSE   app_path hidden, automatically added N.A. N.A.    Details  mode: see [App security] this option will change how the upload files are selected. title \u0026 title_logo: see [Other customizations] warning_toast: see [App security], A toast pop-up message to help you check pre-deploy for security problems. login_screen \u0026 login_theme \u0026 admin_page \u0026 admin_url: see [Accounts, Login and Admin]. verbose: Give you more information on debugging. Most SPS core functions has this option. If it is on, more debugging information will be printed on console. See [Debugging] module_xx – tab_xx: see [Toggle tabs] for loading and unloading tabs. tab_xx: see [Overwrite tabs] for customizing core SPS default tabs. note_url: see [Notification system] for customizing SPS notifications. is_demo: see [Workflow module]. welcome_guide: whether to enable the welcome guide on app start, see first image on SPS guide. app_path: a hidden option. This will be added after the app starts. If not specified in sps(), use current working directory.  View/Set a single option SPS values are globally set, which means you can get/change the these options at inside any R code, R functions and while the app is running (change options after app started is not recommended).\nTo view a single option value, use spsOption(opt = \"OPTION_NAME\"); to overwrite a single option, use spsOption(opt = \"OPTION_NAME\", value = \"NEW_VALUE\").\nspsOption(opt = \"mode\")  ## [1] \"local\"  To overwrite the “mode” option:\nspsOption(opt = \"mode\", \"local\")  Check again, the value has changed to “local”:\nspsOption(opt = \"mode\")  ## [1] \"local\"  If any option does not exist, or the value is “empty” or 0, when getting the value spsOption will return FALSE. Common “empty” values:\n NA NULL length(value) == 0 \"\" (empty string)  Read the help file of ?emptyIsFalse for more information.\nspsOption(opt = \"random_opt\")  ## [1] FALSE  However, these “empty” values can be meaningful in some cases, so use empty_is_false = FALSE to return the original value instead of FALSE\nspsOption(opt = \"random_opt\", empty_is_false = FALSE)  ## NULL  Add your own options SPS is very flexible which allows you to add your own options. To do so, you need to edit the “config/sps_options.yaml” file under your project root.\nYou can use other options as templates to add more. There are two required entries:\n default: will be used as default when you load SPS package. other: Other valid options. If your user provided a value that is other than the default or the other, SPS will show warnings and use default instead. You can write \"*\" as the other value. It means any value will be accepted. SPS will skip to check other valid values for this option.  Currently, the default value can only be length of 1 but other value can be a yaml array, which use [] to define: [value1, value2, ...].\nFor example, we can add some but opening the file with a text editor, here we do it programmatically:\nnew_options \u003c- ' my_opt1: default: true other: [false] my_opt2: default: \"a\" other: [\"*\"] ' write(x = new_options, file = file.path(sps_dir, \"config\", \"sps_options.yaml\"), append = TRUE)  Then we can use spsOptDefaults to check\nspsOptDefaults(app_path = sps_dir)  ## title: ## Default: systemPipeShiny ## Other: * ## title_logo: ## Default: img/sps_small.png ## Other: * ## mode: ## Default: local ## Other: server ## login_screen: ## Default: FALSE ## Other: TRUE ## login_theme: ## Default: random ## Other: * ## use_crayon: ## Default: TRUE ## Other: FALSE ## verbose: ## Default: FALSE ## Other: TRUE ## admin_page: ## Default: TRUE ## Other: FALSE ## admin_url: ## Default: admin ## Other: * ## warning_toast: ## Default: FALSE ## Other: TRUE ## module_wf: ## Default: TRUE ## Other: FALSE ## module_rnaseq: ## Default: TRUE ## Other: FALSE ## module_ggplot: ## Default: TRUE ## Other: FALSE ## tab_welcome: ## Default: TRUE ## Other: FALSE ## tab_vs_main: ## Default: TRUE ## Other: FALSE ## tab_canvas: ## Default: TRUE ## Other: FALSE ## tab_about: ## Default: TRUE ## Other: FALSE ## note_url: ## Default: https://raw.githubusercontent.com/systemPipeR/systemPipeShiny/master/inst/remote_resource/notifications.yaml ## Other: * ## traceback: ## Default: FALSE ## Other: TRUE ## is_demo: ## Default: FALSE ## Other: TRUE ## welcome_guide: ## Default: TRUE ## Other: FALSE ## my_opt1: ## Default: TRUE ## Other: FALSE ## my_opt2: ## Default: a ## Other: * ## * means any value will be accepted  You can see the my_opt1 and my_opt2 have been added to SPS options.\n","categories":"","description":"","excerpt":"SPS has many options you can change. These options will determine how …","ref":"/sps/adv_features/config/","tags":"","title":"Configuration"},{"body":"             pre code { white-space: pre !important; overflow-x: scroll !important; word-break: keep-all !important; word-wrap: initial !important; }  document.addEventListener(\"DOMContentLoaded\", function() { document.querySelector(\"h1\").className = \"title\"; });  document.addEventListener(\"DOMContentLoaded\", function() { var links = document.links; for (var i = 0, linksLength = links.length; i Data Visualization with systemPipeR systemPipeTools package extends the widely used systemPipeR (SPR) (H Backman and Girke 2016) workflow environment with enhanced toolkit for data visualization, including utilities to automate the analysis of differentially expressed genes (DEGs). systemPipeTools provides functions for data transformation and data exploration via scatterplots, hierarchical clustering heatMaps, principal component analysis, multidimensional scaling, generalized principal components, t-Distributed Stochastic Neighbor embedding (t-SNE), and MA and volcano plots. All these utilities can be integrated with the modular design of the systemPipeR environment that allows users to easily substitute any of these features and/or custom with alternatives.\nMetadata and Reads Counting Information The first step is importing the targets file and raw reads counting table.\n The targets file defines all FASTQ files and sample comparisons of the analysis workflow. The raw reads counting table represents all the reads that map to gene (row) for each sample (columns).  ## Targets file targetspath \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") targets \u003c- read.delim(targetspath, comment = \"#\") cmp \u003c- systemPipeR::readComp(file = targetspath, format = \"matrix\", delim = \"-\") ## Count table file countMatrixPath \u003c- system.file(\"extdata\", \"countDFeByg.xls\", package = \"systemPipeR\") countMatrix \u003c- read.delim(countMatrixPath, row.names = 1) showDT(countMatrix)   {\"x\":{\"filter\":\"none\",\"extensions\":[\"FixedColumns\",\"Scroller\"],\"data\":[[\"AT1G01010\",\"AT1G01020\",\"AT1G01030\",\"AT1G01040\",\"AT1G01050\",\"AT1G01060\",\"AT1G01070\",\"AT1G01073\",\"AT1G01080\",\"AT1G01090\",\"AT1G01100\",\"AT1G01110\",\"AT1G01115\",\"AT2G01008\",\"AT2G01021\",\"AT2G01023\",\"AT3G01010\",\"AT3G01015\",\"AT3G01020\",\"AT3G01030\",\"AT3G01040\",\"AT3G01050\",\"AT3G01060\",\"AT3G01070\",\"AT3G01080\",\"AT3G01085\",\"AT3G01090\",\"AT3G01100\",\"AT3G01120\",\"AT3G01130\",\"AT3G01140\",\"AT3G01150\",\"AT3G01160\",\"AT3G01170\",\"AT4G00005\",\"AT4G00020\",\"AT4G00026\",\"AT4G00030\",\"AT4G00040\",\"AT4G00050\",\"AT4G00060\",\"AT4G00070\",\"AT4G00080\",\"AT4G00090\",\"AT4G00100\",\"AT4G00110\",\"AT4G00120\",\"AT4G00124\",\"AT4G00130\",\"AT4G00140\",\"AT5G01010\",\"AT5G01015\",\"AT5G01020\",\"AT5G01030\",\"AT5G01040\",\"AT5G01050\",\"AT5G01060\",\"AT5G01070\",\"AT5G01075\",\"AT5G01080\",\"AT5G01090\",\"AT5G01100\",\"AT5G01110\",\"AT5G01120\",\"AT5G01130\",\"AT5G01140\",\"AT5G01150\",\"AT5G01160\",\"ATCG00020\",\"ATCG00040\",\"ATCG00050\",\"ATCG00070\",\"ATCG00080\",\"ATCG00120\",\"ATCG00130\",\"ATCG00140\",\"ATCG00150\",\"ATCG00160\",\"ATCG00170\",\"ATCG00180\",\"ATCG00190\",\"ATCG00210\",\"ATCG00220\",\"ATCG00270\",\"ATCG00280\",\"ATCG00300\",\"ATCG00330\",\"ATCG00340\",\"ATCG00350\",\"ATCG00360\",\"ATCG00380\",\"ATCG00420\",\"ATCG00430\",\"ATCG00440\",\"ATCG00470\",\"ATCG00480\",\"ATCG00490\",\"ATCG00500\",\"ATCG00510\",\"ATMG00010\",\"ATMG00030\",\"ATMG00040\",\"ATMG00050\",\"ATMG00060\",\"ATMG00070\",\"ATMG00080\",\"ATMG00090\",\"ATMG00110\",\"ATMG00120\",\"ATMG00130\",\"ATMG00140\",\"ATMG00150\",\"ATMG00160\",\"ATMG00170\",\"ATMG00180\",\"ATMG00200\"],[57,23,41,180,60,26,0,0,98,331,230,6,0,28,6267,0,0,0,0,0,93,31,235,0,30,0,70,38,709,54,0,37,33,33,0,4,19,69,61,12,48,0,0,21,186,22,0,0,6,8,143,16,87,31,49,4,0,0,1,0,17,162,2,0,0,0,0,28,1917,9,0,5,15,52,34,36,17,7,7,5,37,6,3,105,283,4,52,147,192,1,2,7,21,12,54,180,856,8,10,2,60,11,0,0,8,7,5,6,3,1,0,0,5,3,18,0],[244,93,98,684,127,264,0,0,379,1027,432,10,0,68,3061,14,0,0,0,0,454,121,801,8,29,0,274,132,2358,140,14,126,238,131,0,25,89,106,176,138,359,0,0,59,384,78,0,0,9,6,442,26,227,170,125,7,0,4,28,0,103,1002,26,2,0,0,0,124,1696,28,3,23,50,210,127,46,53,8,39,49,145,5,83,300,723,34,103,412,480,6,21,97,312,44,339,1191,1029,13,1,0,40,56,0,3,18,18,72,27,0,4,7,1,31,15,59,3],[201,69,73,522,102,59,2,0,179,779,516,11,0,61,7559,6,0,0,0,0,364,108,553,6,151,0,315,87,2031,149,4,186,123,161,0,6,60,255,189,20,162,0,0,89,402,75,0,0,32,30,542,11,184,91,179,5,0,0,9,0,41,754,6,0,0,0,0,119,970,10,1,14,13,113,37,60,22,3,14,22,49,1,5,165,293,5,56,222,191,6,8,18,45,12,92,369,793,4,5,0,79,38,0,4,5,20,30,7,0,0,0,0,25,5,44,1],[169,126,58,664,166,56,1,0,456,1343,458,5,0,33,2203,10,0,1,0,0,578,167,674,2,14,0,373,126,2222,131,39,170,329,212,0,66,75,121,147,117,381,0,0,88,432,147,0,0,4,11,646,17,207,219,103,3,0,0,23,0,121,1185,27,2,0,0,0,127,661,19,1,59,53,190,167,42,88,14,94,68,208,16,75,212,641,45,135,530,520,10,24,147,347,34,423,1501,871,10,1,7,35,50,0,3,47,31,120,34,14,1,5,0,72,17,62,4],[365,107,94,585,125,8,6,0,287,647,468,4,0,48,3125,0,0,0,0,0,375,104,226,5,56,0,334,100,1674,242,2,138,133,110,0,16,41,127,158,23,275,0,0,56,403,81,0,0,28,30,606,4,165,97,150,7,0,0,2,0,62,819,11,2,0,0,0,103,2258,12,0,23,34,160,74,44,26,12,22,32,95,10,25,126,403,10,50,313,366,4,10,26,36,24,102,403,977,2,15,2,43,45,0,2,19,16,25,13,0,1,2,3,26,14,24,1],[229,88,156,680,303,26,6,0,538,1831,849,17,0,53,1168,3,0,1,0,0,735,146,366,6,97,0,349,203,2310,254,0,380,126,296,0,49,43,193,282,19,406,0,0,138,966,131,0,0,16,21,1031,8,463,106,427,10,2,0,5,1,89,2267,41,4,0,0,0,238,1232,6,0,24,56,277,66,33,54,11,55,61,186,3,29,250,375,23,58,509,455,1,10,63,112,15,347,984,1161,10,2,3,35,39,0,7,21,19,41,15,1,3,2,1,46,13,58,3],[41,18,9,162,116,5,2,0,102,350,522,3,0,39,12653,0,0,0,0,0,46,43,8,0,15,0,92,28,780,84,14,45,51,51,0,5,56,73,102,63,70,0,0,42,360,42,0,0,3,7,230,15,153,22,2,0,0,0,11,0,23,24,6,0,0,0,0,60,1677,4,0,5,4,37,8,41,2,0,1,11,18,2,4,63,186,2,19,121,116,0,0,7,8,2,19,73,592,0,2,1,89,19,0,2,4,8,11,5,0,1,0,0,8,6,27,2],[38,25,13,163,180,2,4,0,211,524,634,3,0,38,12386,1,0,0,0,0,76,55,59,0,25,0,139,50,1217,116,14,95,80,54,0,11,66,124,114,45,119,3,0,66,440,40,0,0,2,13,334,58,221,27,2,0,0,0,11,0,36,99,8,0,0,0,0,76,1184,6,0,1,7,42,20,18,16,1,6,15,22,2,2,59,190,4,13,152,168,0,3,11,9,1,30,163,681,2,1,0,35,37,0,4,9,4,9,3,0,1,2,0,16,8,26,2],[152,20,8,249,139,0,5,0,79,323,926,5,0,97,9799,0,0,0,0,0,80,52,13,0,72,0,148,74,1176,160,21,67,92,73,0,11,93,157,185,151,122,0,0,78,656,48,0,0,3,20,461,2,260,57,11,0,0,0,8,0,39,188,4,0,0,0,0,83,1201,0,0,1,11,41,7,23,5,0,6,2,15,2,0,56,132,2,23,75,122,0,2,4,10,4,22,93,475,4,0,0,117,31,0,5,1,5,12,7,2,1,3,0,19,9,33,0],[46,5,6,66,37,0,2,0,42,140,183,2,0,23,10386,1,0,0,0,0,23,21,1,0,7,0,48,19,270,27,7,14,20,22,0,1,12,21,48,8,43,0,0,22,113,12,0,0,1,3,109,10,50,29,2,3,0,0,1,0,13,42,0,0,0,0,0,33,1261,4,0,2,3,42,15,26,4,0,4,5,4,6,1,65,138,9,17,106,126,2,4,6,4,0,12,54,749,6,1,0,30,4,0,0,0,2,3,1,0,0,0,0,2,0,7,0],[294,88,36,697,236,0,5,0,150,600,1333,1,0,82,9414,3,0,2,0,0,162,112,14,0,57,0,236,126,1737,341,15,126,178,132,0,23,146,209,317,158,210,0,0,191,1005,46,0,0,12,22,731,0,423,100,7,0,0,0,12,0,68,277,5,0,0,0,0,128,1192,1,0,2,9,52,9,28,8,3,1,7,22,2,3,83,172,4,25,112,112,1,0,11,14,6,29,110,446,2,4,3,101,56,0,5,10,9,20,14,0,1,1,0,36,9,44,2],[405,151,147,1060,679,4,13,0,551,1693,2401,16,0,112,3265,4,0,0,0,0,260,292,41,2,33,0,468,280,2768,376,50,299,278,112,0,36,175,337,471,169,463,2,0,334,1831,107,0,0,12,13,1229,53,903,127,12,3,0,0,32,0,168,146,33,0,0,0,0,266,398,6,0,4,11,75,26,18,18,3,17,11,67,2,3,71,176,7,28,163,186,2,7,18,23,4,52,345,493,0,4,0,43,88,0,8,25,18,39,14,4,11,4,0,54,20,53,2],[117,43,32,338,362,89,9,0,250,800,762,7,0,142,6908,6,0,0,0,0,143,53,103,0,27,0,257,128,1998,110,52,130,65,95,0,19,38,245,362,472,298,0,0,74,703,51,0,0,5,23,449,6,732,62,2,0,0,0,12,0,46,42,2,2,0,0,0,157,1406,8,0,11,6,51,13,9,5,0,5,13,28,2,2,100,126,2,2,128,194,1,3,12,30,2,63,270,445,2,0,0,64,6,0,1,9,18,6,7,0,0,5,1,21,7,25,1],[139,74,29,604,746,203,43,0,612,1662,1805,38,0,230,3676,2,0,0,0,0,131,211,178,4,34,0,390,194,1861,323,81,191,125,93,0,28,132,397,598,499,414,1,0,166,1319,117,0,0,11,29,861,65,1008,100,2,0,0,2,29,0,128,80,35,0,0,0,0,274,473,0,0,3,16,91,32,23,13,0,5,19,51,2,7,65,167,1,26,121,132,1,0,8,23,1,65,346,377,0,0,0,75,49,0,2,20,16,23,25,1,5,7,0,29,10,26,0],[132,33,12,360,194,118,8,0,78,281,761,4,0,85,11154,2,0,0,0,0,79,87,39,2,88,0,176,121,781,184,19,74,59,80,0,11,52,164,203,239,176,0,0,91,600,37,0,0,6,13,599,2,265,70,0,0,0,0,5,0,41,85,0,0,0,0,0,91,1136,0,0,6,11,34,12,38,10,0,2,9,9,2,2,48,135,3,15,88,108,1,0,2,6,3,25,82,429,2,0,0,114,23,0,5,6,4,10,2,0,4,0,0,16,6,17,0],[64,18,9,80,50,24,0,0,26,122,102,0,0,18,3301,0,0,0,0,0,34,25,6,0,13,0,32,24,219,51,13,20,26,19,0,1,13,22,37,12,48,0,0,20,106,10,0,0,1,2,121,1,95,21,2,0,0,0,3,0,14,33,0,0,0,0,0,38,1860,8,0,3,4,40,26,13,0,0,1,4,1,5,0,61,259,1,28,147,176,2,3,21,11,0,17,84,662,2,2,1,23,15,0,2,9,3,1,3,1,1,0,0,5,0,8,0],[230,43,70,203,214,62,9,0,78,391,627,0,0,100,9421,5,0,0,0,0,105,43,52,0,56,0,127,79,734,97,18,70,28,61,0,4,33,92,112,178,140,0,0,93,565,17,0,0,1,5,331,4,283,29,0,0,0,0,9,0,33,155,4,0,0,0,0,118,1336,0,0,8,4,27,5,8,8,6,2,10,7,1,4,52,112,1,10,81,106,1,0,8,19,3,37,140,440,2,2,0,66,10,0,7,7,5,13,7,0,3,6,1,16,0,28,2],[120,31,85,171,181,32,1,0,55,273,476,4,0,46,8486,2,0,0,0,0,55,24,24,0,33,0,76,45,291,98,12,42,39,29,0,4,26,57,69,59,62,0,0,51,362,17,0,0,1,5,207,3,195,28,1,0,0,0,4,0,17,67,3,0,0,0,0,86,1236,3,0,1,5,39,14,31,12,0,2,2,18,5,1,48,172,0,14,118,118,0,7,0,8,1,19,116,529,4,0,0,46,18,0,3,3,4,5,6,0,0,2,2,12,2,10,0]],\"container\":\"\\n \\n \\n  \\n M1A\\n M1B\\n A1A\\n A1B\\n V1A\\n V1B\\n M6A\\n M6B\\n A6A\\n A6B\\n V6A\\n V6B\\n M12A\\n M12B\\n A12A\\n A12B\\n V12A\\n V12B\\n \\n \\n\",\"options\":{\"scrollX\":true,\"fixedColumns\":true,\"deferRender\":true,\"scrollY\":200,\"scroller\":true,\"columnDefs\":[{\"className\":\"dt-right\",\"targets\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]},{\"orderable\":false,\"targets\":0}],\"order\":[],\"autoWidth\":false,\"orderClasses\":false}},\"evals\":[],\"jsHooks\":[]} Data Transformation For gene differential expression, raw counts are required, however for data visualization or clustering, it can be useful to work with transformed count data. exploreDDS function is convenience wrapper to transform raw read counts using the DESeq2 package transformations methods. The input file has to contain all the genes, not just differentially expressed ones. Supported methods include variance stabilizing transformation (vst) (Anders and Huber (2010)), and regularized-logarithm transformation or rlog (Love, Huber, and Anders (2014)).\nexploredds \u003c- exploreDDS(countMatrix, targets, cmp = cmp[[1]], preFilter = NULL, transformationMethod = \"rlog\") exploredds  ## class: DESeqTransform ## dim: 116 18 ## metadata(1): version ## assays(1): '' ## rownames(116): AT1G01010 AT1G01020 ... ATMG00180 ATMG00200 ## rowData names(51): baseMean baseVar ... dispFit rlogIntercept ## colnames(18): M1A M1B ... V12A V12B ## colData names(2): condition sizeFactor  Users are strongly encouraged to consult the DESeq2 vignette for more detailed information on this topic and how to properly run DESeq2 on data sets with more complex experimental designs.\nScatterplot To decide which transformation to choose, we can visualize the transformation effect comparing two samples or a grid of all samples, as follows:\nexploreDDSplot(countMatrix, targets, cmp = cmp[[1]], preFilter = NULL, samples = c(\"M12A\", \"M12A\", \"A12A\", \"A12A\"), scattermatrix = TRUE)  ## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`. ## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`. ## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`. ## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.  The scatterplots are created using the log2 transform normalized reads count, variance stabilizing transformation (VST) (Anders and Huber (2010)), and regularized-logarithm transformation or rlog (Love, Huber, and Anders (2014)).\nHierarchical Clustering Dendrogram The following computes the sample-wise correlation coefficients using the stats::cor() function from the transformed expression values. After transformation to a distance matrix, hierarchical clustering is performed with the stats::hclust function and the result is plotted as a dendrogram, as follows:\nhclustplot(exploredds, method = \"spearman\")  The function provides the utility to save the plot automatically.\nHierarchical Clustering HeatMap This function performs hierarchical clustering on the transformed expression matrix generated within the DESeq2 package. It uses, by default, a Pearson correlation-based distance measure and complete linkage for cluster join. If samples selected in the clust argument, it will be applied the stats::dist() function to the transformed count matrix to get sample-to-sample distances. Also, it is possible to generate the pheatmap or plotly plot format.\n## Samples plot heatMaplot(exploredds, clust = \"samples\", plotly = TRUE)   {\"x\":{\"visdat\":{\"34341250a552\":[\"function () \",\"plotlyVisDat\"]},\"cur_data\":\"34341250a552\",\"attrs\":{\"34341250a552\":{\"x\":[\"M1A\",\"M1B\",\"A1A\",\"A1B\",\"V1A\",\"V1B\",\"M6A\",\"M6B\",\"A6A\",\"A6B\",\"V6A\",\"V6B\",\"M12A\",\"M12B\",\"A12A\",\"A12B\",\"V12A\",\"V12B\"],\"y\":[\"M1A\",\"M1B\",\"A1A\",\"A1B\",\"V1A\",\"V1B\",\"M6A\",\"M6B\",\"A6A\",\"A6B\",\"V6A\",\"V6B\",\"M12A\",\"M12B\",\"A12A\",\"A12B\",\"V12A\",\"V12B\"],\"z\":[[0,7.04650716301563,5.70678310405774,8.20976445672658,5.82667776225818,7.3279781790244,8.39986657356742,8.16733063204217,9.58637200173571,7.68841587215867,10.1640441508758,10.7676019921496,10.0039632569807,11.32498784981,9.8146750121175,7.74473230078491,9.45716343166999,7.95378561107353],[7.04650716301563,0,6.66154862259465,4.25865887783483,6.67960023884926,6.26668680998727,9.91630309634838,9.25451827523275,10.9564120825428,9.93575571094716,11.099388064366,10.6754670535994,9.90683653800534,10.9561298226477,10.7399675586854,9.18116169782986,9.77109863659917,9.40323019102986],[5.70678310405774,6.66154862259465,0,7.98994675562316,4.94593038167375,5.72836463615867,8.11235158724083,7.65256178153187,8.15065142907597,8.76007256265634,8.30988507282082,8.91309701678265,8.66629724080143,9.82426010379448,8.30870129249907,8.54268071001664,8.35323474316893,7.70897398227057],[8.20976445672658,4.25865887783483,7.98994675562316,0,6.99532785790822,6.43469620310431,10.4096764476556,9.51652950962973,11.1939050957909,10.4627147426448,11.1689702900908,10.546647320753,10.5708484745604,11.5547562063225,11.6160197603858,9.88406363111819,10.7020155017471,10.4675703563444],[5.82667776225818,6.67960023884926,4.94593038167375,6.99532785790822,0,4.8575131388009,8.46288752842871,8.0319315058583,8.726172089783,8.73161790330149,8.27286514155669,8.88152783442813,9.62675093908279,10.8109068917939,9.50867890759956,8.56129430593498,9.21166728284744,8.48571139105286],[7.3279781790244,6.26668680998727,5.72836463615867,6.43469620310431,4.8575131388009,0,10.051631844146,8.92833694901717,9.88801350128227,10.1968579735558,9.80606329541741,9.47694677716336,10.2032985748696,10.9502439173503,10.7118343889731,10.2485736407499,9.95124988126944,9.6455696901528],[8.39986657356742,9.91630309634838,8.11235158724083,10.4096764476556,8.46288752842871,10.051631844146,0,4.32289188181976,5.45401419571659,6.66929785847857,5.82333805363524,6.89960979808065,6.60846441830892,7.49378480640748,6.09490379797155,6.92856692552639,6.73658240504653,6.04084631483171],[8.16733063204217,9.25451827523275,7.65256178153187,9.51652950962973,8.0319315058583,8.92833694901717,4.32289188181976,0,5.86548263173596,6.92170483627566,6.43771467140554,5.936553276735,6.66918890392232,7.10201521498403,6.87338305585792,7.70858313971616,6.93977726065822,6.12205683109543],[9.58637200173571,10.9564120825428,8.15065142907597,11.1939050957909,8.726172089783,9.88801350128227,5.45401419571659,5.86548263173596,0,8.10337075683393,4.02970654063498,6.37598537189006,6.96355226928132,7.79890861008178,5.50539366529938,8.64913887544163,6.82883422186719,6.22801929982399],[7.68841587215867,9.93575571094716,8.76007256265634,10.4627147426448,8.73161790330149,10.1968579735558,6.66929785847857,6.92170483627566,8.10337075683393,0,9.22853479532283,10.1030591897721,9.35527845594351,11.0912154866989,8.92785474831708,6.32121021712703,9.14034499886722,7.20502415178336],[10.1640441508758,11.099388064366,8.30988507282082,11.1689702900908,8.27286514155669,9.80606329541741,5.82333805363524,6.43771467140554,4.02970654063498,9.22853479532283,0,5.40796455622463,7.23562402229216,8.05893271349359,5.83000947413891,8.94623155465764,6.52802367477244,6.94139936721835],[10.7676019921496,10.6754670535994,8.91309701678265,10.546647320753,8.88152783442813,9.47694677716336,6.89960979808065,5.936553276735,6.37598537189006,10.1030591897721,5.40796455622463,0,7.14309602773391,6.1610771391521,7.4911181113735,10.0381028337193,7.56298264238769,7.68916986877059],[10.0039632569807,9.90683653800534,8.66629724080143,10.5708484745604,9.62675093908279,10.2032985748696,6.60846441830892,6.66918890392232,6.96355226928132,9.35527845594351,7.23562402229216,7.14309602773391,0,5.69259437645614,5.58694500597765,8.50447659713418,5.53119139458227,6.33933832045536],[11.32498784981,10.9561298226477,9.82426010379448,11.5547562063225,10.8109068917939,10.9502439173503,7.49378480640748,7.10201521498403,7.79890861008178,11.0912154866989,8.05893271349359,6.1610771391521,5.69259437645614,0,6.42494730487994,10.5078546330517,7.19994521380774,7.82561653509729],[9.8146750121175,10.7399675586854,8.30870129249907,11.6160197603858,9.50867890759956,10.7118343889731,6.09490379797155,6.87338305585792,5.50539366529938,8.92785474831708,5.83000947413891,7.4911181113735,5.58694500597765,6.42494730487994,0,8.18110233128943,5.25899641633563,5.72577800197228],[7.74473230078491,9.18116169782986,8.54268071001664,9.88406363111819,8.56129430593498,10.2485736407499,6.92856692552639,7.70858313971616,8.64913887544163,6.32121021712703,8.94623155465764,10.0381028337193,8.50447659713418,10.5078546330517,8.18110233128943,0,7.99272460895717,6.58008064549415],[9.45716343166999,9.77109863659917,8.35323474316893,10.7020155017471,9.21166728284744,9.95124988126944,6.73658240504653,6.93977726065822,6.82883422186719,9.14034499886722,6.52802367477244,7.56298264238769,5.53119139458227,7.19994521380774,5.25899641633563,7.99272460895717,0,5.56397338345294],[7.95378561107353,9.40323019102986,7.70897398227057,10.4675703563444,8.48571139105286,9.6455696901528,6.04084631483171,6.12205683109543,6.22801929982399,7.20502415178336,6.94139936721835,7.68916986877059,6.33933832045536,7.82561653509729,5.72577800197228,6.58008064549415,5.56397338345294,0]],\"alpha_stroke\":1,\"sizes\":[10,100],\"spans\":[1,20],\"type\":\"heatmap\"}},\"layout\":{\"margin\":{\"b\":40,\"l\":60,\"t\":25,\"r\":10},\"xaxis\":{\"domain\":[0,1],\"automargin\":true,\"title\":[]},\"yaxis\":{\"domain\":[0,1],\"automargin\":true,\"title\":[]},\"scene\":{\"zaxis\":{\"title\":[]}},\"hovermode\":\"closest\",\"showlegend\":false,\"legend\":{\"yanchor\":\"top\",\"y\":0.5}},\"source\":\"A\",\"config\":{\"showSendToCloud\":false},\"data\":[{\"colorbar\":{\"title\":\"\",\"ticklen\":2,\"len\":0.5,\"lenmode\":\"fraction\",\"y\":1,\"yanchor\":\"top\"},\"colorscale\":[[\"0\",\"rgba(68,1,84,1)\"],[\"0.0416666666666667\",\"rgba(70,19,97,1)\"],[\"0.0833333333333333\",\"rgba(72,32,111,1)\"],[\"0.125\",\"rgba(71,45,122,1)\"],[\"0.166666666666667\",\"rgba(68,58,128,1)\"],[\"0.208333333333333\",\"rgba(64,70,135,1)\"],[\"0.25\",\"rgba(60,82,138,1)\"],[\"0.291666666666667\",\"rgba(56,93,140,1)\"],[\"0.333333333333333\",\"rgba(49,104,142,1)\"],[\"0.375\",\"rgba(46,114,142,1)\"],[\"0.416666666666667\",\"rgba(42,123,142,1)\"],[\"0.458333333333333\",\"rgba(38,133,141,1)\"],[\"0.5\",\"rgba(37,144,140,1)\"],[\"0.541666666666667\",\"rgba(33,154,138,1)\"],[\"0.583333333333333\",\"rgba(39,164,133,1)\"],[\"0.625\",\"rgba(47,174,127,1)\"],[\"0.666666666666667\",\"rgba(53,183,121,1)\"],[\"0.708333333333333\",\"rgba(79,191,110,1)\"],[\"0.75\",\"rgba(98,199,98,1)\"],[\"0.791666666666667\",\"rgba(119,207,85,1)\"],[\"0.833333333333333\",\"rgba(147,214,70,1)\"],[\"0.875\",\"rgba(172,220,52,1)\"],[\"0.916666666666667\",\"rgba(199,225,42,1)\"],[\"0.958333333333333\",\"rgba(226,228,40,1)\"],[\"1\",\"rgba(253,231,37,1)\"]],\"showscale\":true,\"x\":[\"M1A\",\"M1B\",\"A1A\",\"A1B\",\"V1A\",\"V1B\",\"M6A\",\"M6B\",\"A6A\",\"A6B\",\"V6A\",\"V6B\",\"M12A\",\"M12B\",\"A12A\",\"A12B\",\"V12A\",\"V12B\"],\"y\":[\"M1A\",\"M1B\",\"A1A\",\"A1B\",\"V1A\",\"V1B\",\"M6A\",\"M6B\",\"A6A\",\"A6B\",\"V6A\",\"V6B\",\"M12A\",\"M12B\",\"A12A\",\"A12B\",\"V12A\",\"V12B\"],\"z\":[[0,7.04650716301563,5.70678310405774,8.20976445672658,5.82667776225818,7.3279781790244,8.39986657356742,8.16733063204217,9.58637200173571,7.68841587215867,10.1640441508758,10.7676019921496,10.0039632569807,11.32498784981,9.8146750121175,7.74473230078491,9.45716343166999,7.95378561107353],[7.04650716301563,0,6.66154862259465,4.25865887783483,6.67960023884926,6.26668680998727,9.91630309634838,9.25451827523275,10.9564120825428,9.93575571094716,11.099388064366,10.6754670535994,9.90683653800534,10.9561298226477,10.7399675586854,9.18116169782986,9.77109863659917,9.40323019102986],[5.70678310405774,6.66154862259465,0,7.98994675562316,4.94593038167375,5.72836463615867,8.11235158724083,7.65256178153187,8.15065142907597,8.76007256265634,8.30988507282082,8.91309701678265,8.66629724080143,9.82426010379448,8.30870129249907,8.54268071001664,8.35323474316893,7.70897398227057],[8.20976445672658,4.25865887783483,7.98994675562316,0,6.99532785790822,6.43469620310431,10.4096764476556,9.51652950962973,11.1939050957909,10.4627147426448,11.1689702900908,10.546647320753,10.5708484745604,11.5547562063225,11.6160197603858,9.88406363111819,10.7020155017471,10.4675703563444],[5.82667776225818,6.67960023884926,4.94593038167375,6.99532785790822,0,4.8575131388009,8.46288752842871,8.0319315058583,8.726172089783,8.73161790330149,8.27286514155669,8.88152783442813,9.62675093908279,10.8109068917939,9.50867890759956,8.56129430593498,9.21166728284744,8.48571139105286],[7.3279781790244,6.26668680998727,5.72836463615867,6.43469620310431,4.8575131388009,0,10.051631844146,8.92833694901717,9.88801350128227,10.1968579735558,9.80606329541741,9.47694677716336,10.2032985748696,10.9502439173503,10.7118343889731,10.2485736407499,9.95124988126944,9.6455696901528],[8.39986657356742,9.91630309634838,8.11235158724083,10.4096764476556,8.46288752842871,10.051631844146,0,4.32289188181976,5.45401419571659,6.66929785847857,5.82333805363524,6.89960979808065,6.60846441830892,7.49378480640748,6.09490379797155,6.92856692552639,6.73658240504653,6.04084631483171],[8.16733063204217,9.25451827523275,7.65256178153187,9.51652950962973,8.0319315058583,8.92833694901717,4.32289188181976,0,5.86548263173596,6.92170483627566,6.43771467140554,5.936553276735,6.66918890392232,7.10201521498403,6.87338305585792,7.70858313971616,6.93977726065822,6.12205683109543],[9.58637200173571,10.9564120825428,8.15065142907597,11.1939050957909,8.726172089783,9.88801350128227,5.45401419571659,5.86548263173596,0,8.10337075683393,4.02970654063498,6.37598537189006,6.96355226928132,7.79890861008178,5.50539366529938,8.64913887544163,6.82883422186719,6.22801929982399],[7.68841587215867,9.93575571094716,8.76007256265634,10.4627147426448,8.73161790330149,10.1968579735558,6.66929785847857,6.92170483627566,8.10337075683393,0,9.22853479532283,10.1030591897721,9.35527845594351,11.0912154866989,8.92785474831708,6.32121021712703,9.14034499886722,7.20502415178336],[10.1640441508758,11.099388064366,8.30988507282082,11.1689702900908,8.27286514155669,9.80606329541741,5.82333805363524,6.43771467140554,4.02970654063498,9.22853479532283,0,5.40796455622463,7.23562402229216,8.05893271349359,5.83000947413891,8.94623155465764,6.52802367477244,6.94139936721835],[10.7676019921496,10.6754670535994,8.91309701678265,10.546647320753,8.88152783442813,9.47694677716336,6.89960979808065,5.936553276735,6.37598537189006,10.1030591897721,5.40796455622463,0,7.14309602773391,6.1610771391521,7.4911181113735,10.0381028337193,7.56298264238769,7.68916986877059],[10.0039632569807,9.90683653800534,8.66629724080143,10.5708484745604,9.62675093908279,10.2032985748696,6.60846441830892,6.66918890392232,6.96355226928132,9.35527845594351,7.23562402229216,7.14309602773391,0,5.69259437645614,5.58694500597765,8.50447659713418,5.53119139458227,6.33933832045536],[11.32498784981,10.9561298226477,9.82426010379448,11.5547562063225,10.8109068917939,10.9502439173503,7.49378480640748,7.10201521498403,7.79890861008178,11.0912154866989,8.05893271349359,6.1610771391521,5.69259437645614,0,6.42494730487994,10.5078546330517,7.19994521380774,7.82561653509729],[9.8146750121175,10.7399675586854,8.30870129249907,11.6160197603858,9.50867890759956,10.7118343889731,6.09490379797155,6.87338305585792,5.50539366529938,8.92785474831708,5.83000947413891,7.4911181113735,5.58694500597765,6.42494730487994,0,8.18110233128943,5.25899641633563,5.72577800197228],[7.74473230078491,9.18116169782986,8.54268071001664,9.88406363111819,8.56129430593498,10.2485736407499,6.92856692552639,7.70858313971616,8.64913887544163,6.32121021712703,8.94623155465764,10.0381028337193,8.50447659713418,10.5078546330517,8.18110233128943,0,7.99272460895717,6.58008064549415],[9.45716343166999,9.77109863659917,8.35323474316893,10.7020155017471,9.21166728284744,9.95124988126944,6.73658240504653,6.93977726065822,6.82883422186719,9.14034499886722,6.52802367477244,7.56298264238769,5.53119139458227,7.19994521380774,5.25899641633563,7.99272460895717,0,5.56397338345294],[7.95378561107353,9.40323019102986,7.70897398227057,10.4675703563444,8.48571139105286,9.6455696901528,6.04084631483171,6.12205683109543,6.22801929982399,7.20502415178336,6.94139936721835,7.68916986877059,6.33933832045536,7.82561653509729,5.72577800197228,6.58008064549415,5.56397338345294,0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"frame\":null}],\"highlight\":{\"on\":\"plotly_click\",\"persistent\":false,\"dynamic\":false,\"selectize\":false,\"opacityDim\":0.2,\"selected\":{\"opacity\":1},\"debounce\":0},\"shinyEvents\":[\"plotly_hover\",\"plotly_click\",\"plotly_selected\",\"plotly_relayout\",\"plotly_brushed\",\"plotly_brushing\",\"plotly_clickannotation\",\"plotly_doubleclick\",\"plotly_deselect\",\"plotly_afterplot\",\"plotly_sunburstclick\"],\"base_url\":\"https://plot.ly\"},\"evals\":[],\"jsHooks\":[]} If ind selected in the clust argument, it is necessary to provide the list of differentially expressed genes for the exploredds subset.\n## Individuals genes identified in DEG analysis DEG analysis with `systemPipeR` degseqDF \u003c- systemPipeR::run_DESeq2(countDF = countMatrix, targets = targets, cmp = cmp[[1]], independent = FALSE) DEG_list \u003c- systemPipeR::filterDEGs(degDF = degseqDF, filter = c(Fold = 2, FDR = 10))  ### Plot heatMaplot(exploredds, clust = \"ind\", DEGlist = unique(as.character(unlist(DEG_list[[1]]))))  The function provides the utility to save the plot automatically.\nPrincipal Component Analysis This function plots a Principal Component Analysis (PCA) from transformed expression matrix. This plot shows samples variation based on the expression values and identifies batch effects.\nPCAplot(exploredds, plotly = FALSE)  The function provides the utility to save the plot automatically.\nMultidimensional scaling with MDSplot This function computes and plots multidimensional scaling analysis for dimension reduction of count expression matrix. Internally, it is applied the stats::dist() function to the transformed count matrix to get sample-to-sample distances.\nMDSplot(exploredds, plotly = FALSE)  The function provides the utility to save the plot automatically.\nDimension Reduction with GLMplot This function computes and plots generalized principal components analysis for dimension reduction of count expression matrix.\nexploredds_r \u003c- exploreDDS(countMatrix, targets, cmp = cmp[[1]], preFilter = NULL, transformationMethod = \"raw\") GLMplot(exploredds_r, plotly = FALSE)  The function provides the utility to save the plot automatically.\nMA plot This function plots log2 fold changes (y-axis) versus the mean of normalized counts (on the x-axis). Statistically significant features are colored.\nMAplot(degseqDF, comparison = \"M12-A12\", filter = c(Fold = 1, FDR = 20), genes = \"ATCG00280\")  The function provides the utility to save the plot automatically.\nt-Distributed Stochastic Neighbor embedding with tSNEplot This function computes and plots t-Distributed Stochastic Neighbor embedding (t-SNE) analysis for unsupervised nonlinear dimensionality reduction of count expression matrix. Internally, it is applied the Rtsne::Rtsne() (Krijthe 2015) function, using the exact t-SNE computing with theta=0.0.\ntSNEplot(countMatrix, targets, perplexity = 5)  Volcano plot A simple function that shows statistical significance (p-value) versus magnitude of change (log2 fold change).\nvolcanoplot(degseqDF, comparison = \"M12-A12\", filter = c(Fold = 1, FDR = 20), genes = \"ATCG00280\")  Version information sessionInfo()  ## R Under development (unstable) (2021-02-04 r79940) ## Platform: x86_64-pc-linux-gnu (64-bit) ## Running under: Ubuntu 20.04.2 LTS ## ## Matrix products: default ## BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 ## LAPACK: /home/dcassol/src/R-devel/lib/libRlapack.so ## ## locale: ## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C ## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 ## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 ## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C ## [9] LC_ADDRESS=C LC_TELEPHONE=C ## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C ## ## attached base packages: ## [1] stats4 parallel stats graphics grDevices utils datasets ## [8] methods base ## ## other attached packages: ## [1] systemPipeR_1.25.11 ShortRead_1.49.2 ## [3] GenomicAlignments_1.27.2 SummarizedExperiment_1.21.3 ## [5] Biobase_2.51.0 MatrixGenerics_1.3.1 ## [7] matrixStats_0.58.0 BiocParallel_1.25.5 ## [9] Rsamtools_2.7.2 Biostrings_2.59.2 ## [11] XVector_0.31.1 GenomicRanges_1.43.4 ## [13] GenomeInfoDb_1.27.11 IRanges_2.25.9 ## [15] S4Vectors_0.29.15 BiocGenerics_0.37.2 ## [17] systemPipeTools_0.9.1 BiocStyle_2.19.2 ## ## loaded via a namespace (and not attached): ## [1] backports_1.2.1 BiocFileCache_1.99.6 plyr_1.8.6 ## [4] lazyeval_0.2.2 splines_4.1.0 crosstalk_1.1.1 ## [7] ggplot2_3.3.3 digest_0.6.27 htmltools_0.5.1.1 ## [10] fansi_0.4.2 magrittr_2.0.1 checkmate_2.0.0 ## [13] memoise_2.0.0 BSgenome_1.59.2 base64url_1.4 ## [16] limma_3.47.12 annotate_1.69.2 prettyunits_1.1.1 ## [19] jpeg_0.1-8.1 colorspace_2.0-0 blob_1.2.1 ## [22] rappdirs_0.3.3 ggrepel_0.9.1 xfun_0.22 ## [25] dplyr_1.0.5 crayon_1.4.1 RCurl_1.98-1.3 ## [28] jsonlite_1.7.2 genefilter_1.73.1 VariantAnnotation_1.37.1 ## [31] brew_1.0-6 survival_3.2-10 ape_5.4-1 ## [34] glue_1.4.2 gtable_0.3.0 zlibbioc_1.37.0 ## [37] DelayedArray_0.17.10 V8_3.4.0 scales_1.1.1 ## [40] pheatmap_1.0.12 DBI_1.1.1 GGally_2.1.1 ## [43] edgeR_3.33.3 Rcpp_1.0.6 viridisLite_0.4.0 ## [46] xtable_1.8-4 progress_1.2.2 tidytree_0.3.3 ## [49] bit_4.0.4 rsvg_2.1.1 DT_0.18 ## [52] htmlwidgets_1.5.3 httr_1.4.2 RColorBrewer_1.1-2 ## [55] ellipsis_0.3.1 farver_2.1.0 pkgconfig_2.0.3 ## [58] reshape_0.8.8 XML_3.99-0.6 dbplyr_2.1.1 ## [61] sass_0.3.1 locfit_1.5-9.4 utf8_1.2.1 ## [64] labeling_0.4.2 later_1.1.0.1 tidyselect_1.1.0 ## [67] rlang_0.4.10 AnnotationDbi_1.53.1 munsell_0.5.0 ## [70] tools_4.1.0 cachem_1.0.4 generics_0.1.0 ## [73] RSQLite_2.2.7 evaluate_0.14 stringr_1.4.0 ## [76] fastmap_1.1.0 yaml_2.2.1 ggtree_2.5.2 ## [79] knitr_1.32 bit64_4.0.5 purrr_0.3.4 ## [82] KEGGREST_1.31.1 nlme_3.1-152 mime_0.10 ## [85] formatR_1.9 aplot_0.0.6 biomaRt_2.47.7 ## [88] compiler_4.1.0 filelock_1.0.2 plotly_4.9.3 ## [91] curl_4.3 png_0.1-7 treeio_1.15.7 ## [94] tibble_3.1.1 geneplotter_1.69.0 bslib_0.2.4 ## [97] stringi_1.5.3 highr_0.9 blogdown_1.3 ## [100] GenomicFeatures_1.43.8 lattice_0.20-41 Matrix_1.3-2 ## [103] glmpca_0.2.0 vctrs_0.3.7 pillar_1.6.0 ## [106] lifecycle_1.0.0 BiocManager_1.30.12 jquerylib_0.1.3 ## [109] data.table_1.14.0 bitops_1.0-6 httpuv_1.5.5 ## [112] rtracklayer_1.51.5 patchwork_1.1.1 BiocIO_1.1.2 ## [115] R6_2.5.0 latticeExtra_0.6-29 hwriter_1.3.2 ## [118] promises_1.2.0.1 bookdown_0.22 codetools_0.2-18 ## [121] MASS_7.3-53.1 assertthat_0.2.1 rjson_0.2.20 ## [124] DESeq2_1.31.18 withr_2.4.2 batchtools_0.9.15 ## [127] GenomeInfoDbData_1.2.4 hms_1.0.0 grid_4.1.0 ## [130] tidyr_1.1.3 DOT_0.1 rmarkdown_2.7.12 ## [133] rvcheck_0.1.8 Rtsne_0.15 shiny_1.6.0 ## [136] restfulr_0.0.13  Funding This project is funded by NSF award ABI-1661152.\nReferences Anders, Simon, and Wolfgang Huber. 2010. “Differential Expression Analysis for Sequence Count Data.” Genome Biol. 11 (10): R106.\n H Backman, Tyler W, and Thomas Girke. 2016. “systemPipeR: NGS workflow and report generation environment.” BMC Bioinformatics 17 (1): 388. https://doi.org/10.1186/s12859-016-1241-0.\n Krijthe, Jesse H. 2015. Rtsne: T-Distributed Stochastic Neighbor Embedding Using Barnes-Hut Implementation. https://github.com/jkrijthe/Rtsne.\n Love, Michael, Wolfgang Huber, and Simon Anders. 2014. “Moderated Estimation of Fold Change and Dispersion for RNA-seq Data with DESeq2.” Genome Biol. 15 (12): 550. https://doi.org/10.1186/s13059-014-0550-8.\n  ","categories":"","description":"","excerpt":"             pre code { white-space: pre !important; overflow-x: …","ref":"/sp/spt/systempipetools/","tags":"","title":"Data Visualizations"},{"body":"document.addEventListener(\"DOMContentLoaded\", function() { document.querySelector(\"h1\").className = \"title\"; });  document.addEventListener(\"DOMContentLoaded\", function() { var links = document.links; for (var i = 0, linksLength = links.length; i Note: if you use systemPipeR in published research, please cite: Backman, T.W.H and Girke, T. (2016). systemPipeR: NGS Workflow and Report Generation Environment. BMC Bioinformatics, 17: 388. 10.1186/s12859-016-1241-0.\nIntroduction systemPipeR provides flexible utilities for building and running automated end-to-end analysis workflows for a wide range of research applications, including next-generation sequencing (NGS) experiments, such as RNA-Seq, ChIP-Seq, VAR-Seq and Ribo-Seq (H Backman and Girke 2016). Important features include a uniform workflow interface across different data analysis applications, automated report generation, and support for running both R and command-line software, such as NGS aligners or peak/variant callers, on local computers or compute clusters (Figure 1). The latter supports interactive job submissions and batch submissions to queuing systems of clusters. For instance, systemPipeR* can be used with most command-line aligners such as BWA (Heng Li 2013; H. Li and Durbin 2009), HISAT2 (Kim, Langmead, and Salzberg 2015), TopHat2 (Kim et al. 2013) and Bowtie2 (Langmead and Salzberg 2012), as well as the R-based NGS aligners Rsubread* (Liao, Smyth, and Shi 2013) and gsnap (gmapR) (Wu and Nacu 2010). Efficient handling of complex sample sets (e.g. FASTQ/BAM files) and experimental designs are facilitated by a well-defined sample annotation infrastructure which improves reproducibility and user-friendliness of many typical analysis workflows in the NGS area (Lawrence et al. 2013).\nThe main motivation and advantages of using systemPipeR for complex data analysis tasks are:\n Facilitates the design of complex NGS workflows involving multiple R/Bioconductor packages Common workflow interface for different NGS applications Makes NGS analysis with Bioconductor utilities more accessible to new users Simplifies usage of command-line software from within R Reduces the complexity of using compute clusters for R and command-line software Accelerates runtime of workflows via parallelization on computer systems with multiple CPU cores and/or multiple compute nodes Improves reproducibility by automating analyses and generation of analysis reports    Figure 1: Relevant features in systemPipeR. Workflow design concepts are illustrated under (A \u0026 B). Examples of systemPipeR’s visualization functionalities are given under (C). \nA central concept for designing workflows within the systemPipeR environment is the use of workflow management containers. In previous versions, systemPipeR used a custom command-line interface called SYSargs (see Figure 3) and for this purpose will continue to be supported for some time. With the latest Bioconductor Release 3.9, we are adopting for this functionality the widely used community standard Common Workflow Language (CWL) for describing analysis workflows in a generic and reproducible manner, introducing SYSargs2 workflow control class (see Figure 2). Using this community standard in systemPipeR has many advantages. For instance, the integration of CWL allows running systemPipeR workflows from a single specification instance either entirely from within R, from various command-line wrappers (e.g., cwl-runner) or from other languages (, e.g., Bash or Python). systemPipeR includes support for both command-line and R/Bioconductor software as well as resources for containerization, parallel evaluations on computer clusters along with the automated generation of interactive analysis reports.\nAn important feature of systemPipeR's CWL interface is that it provides two options to run command-line tools and workflows based on CWL. First, one can run CWL in its native way via an R-based wrapper utility for cwl-runner or cwl-tools (CWL-based approach). Second, one can run workflows using CWL’s command-line and workflow instructions from within R (R-based approach). In the latter case the same CWL workflow definition files (e.g. *.cwl and *.yml) are used but rendered and executed entirely with R functions defined by systemPipeR, and thus use CWL mainly as a command-line and workflow definition format rather than software to run workflows. In this regard systemPipeR also provides several convenience functions that are useful for designing and debugging workflows, such as a command-line rendering function to retrieve the exact command-line strings for each data set and processing step prior to running a command-line.\nThis overview introduces the design of a new CWL S4 class in systemPipeR, as well as the custom command-line interface, combined with the overview of all the common analysis steps of NGS experiments.\nWorkflow design structure using SYSargs2 The flexibility of systemPipeR's new interface workflow control class is the driving factor behind the use of as many steps necessary for the analysis, as well as the connection between command-line- or R-based software. The connectivity among all workflow steps is achieved by the SYSargs2 workflow control class (see Figure 3). This S4 class is a list-like container where each instance stores all the input/output paths and parameter components required for a particular data analysis step. SYSargs2 instances are generated by two constructor functions, loadWorkflow and renderWF, using as data input targets or yaml files as well as two cwl parameter files (for details see below). When running preconfigured workflows, the only input the user needs to provide is the initial targets file containing the paths to the input files (e.g. FASTQ) along with unique sample labels. Subsequent targets instances are created automatically. The parameters required for running command-line software is provided by the parameter (.cwl) files described below.\nWe also introduce the SYSargs2Pipe class that organizes one or many SYSargs2 containers in a single compound object capturing all information required to run, control and monitor complex workflows from start to finish. This design enhances the systemPipeR workflow framework with a generalized, flexible, and robust design.\n  Figure 2: Workflow steps with input/output file operations are controlled by SYSargs2 objects. Each SYSargs2 instance is constructed from one targets and two param files. The only input provided by the user is the initial targets file. Subsequent targets instances are created automatically, from the previous output files. Any number of predefined or custom workflow steps are supported. One or many SYSargs2 objects are organized in an SYSargs2Pipe container.\nWorkflow Management using SYSargsList systemPipeR allows creation (multi-step analyses) and execution of workflow entirely for R, with control, flexibility, and scalability of the all process. The execution of the workflow can be sent to a HPC, can be parallelizes, accelerating results acquisition. A workflow management system provides an infrastructure for the set-up, performance and monitoring of a defined sequence of tasks, arranged as a workflow application.\n  Figure 3: Workflow Management using SYSargsList.\nWorkflow design structure using SYSargs: Previous version Instances of this S4 object class are constructed by the systemArgs function from two simple tabular files: a targets file and a param file. The latter is optional for workflow steps lacking command-line software. Typically, a SYSargs instance stores all sample-level inputs as well as the paths to the corresponding outputs generated by command-line- or R-based software generating sample-level output files, such as read preprocessors (trimmed/filtered FASTQ files), aligners (SAM/BAM files), variant callers (VCF/BCF files) or peak callers (BED/WIG files). Each sample level input/output operation uses its own SYSargs instance. The outpaths of SYSargs usually define the sample inputs for the next SYSargs instance. This connectivity is established by writing the outpaths with the writeTargetsout function to a new targets file that serves as input to the next systemArgs call. Typically, the user has to provide only the initial targets file. All downstream targets files are generated automatically. By chaining several SYSargs steps together one can construct complex workflows involving many sample-level input/output file operations with any combination of command-line or R-based software.\n  Figure 4: Workflow design structure of systemPipeR using SYSargs.\nReference H Backman, Tyler W, and Thomas Girke. 2016. “systemPipeR: NGS workflow and report generation environment.” BMC Bioinformatics 17 (1): 388. https://doi.org/10.1186/s12859-016-1241-0.\n Kim, Daehwan, Ben Langmead, and Steven L Salzberg. 2015. “HISAT: A Fast Spliced Aligner with Low Memory Requirements.” Nat. Methods 12 (4): 357–60.\n Kim, Daehwan, Geo Pertea, Cole Trapnell, Harold Pimentel, Ryan Kelley, and Steven L Salzberg. 2013. “TopHat2: Accurate Alignment of Transcriptomes in the Presence of Insertions, Deletions and Gene Fusions.” Genome Biol. 14 (4): R36. https://doi.org/10.1186/gb-2013-14-4-r36.\n Langmead, Ben, and Steven L Salzberg. 2012. “Fast Gapped-Read Alignment with Bowtie 2.” Nat. Methods 9 (4): 357–59. https://doi.org/10.1038/nmeth.1923.\n Lawrence, Michael, Wolfgang Huber, Hervé Pagès, Patrick Aboyoun, Marc Carlson, Robert Gentleman, Martin T Morgan, and Vincent J Carey. 2013. “Software for Computing and Annotating Genomic Ranges.” PLoS Comput. Biol. 9 (8): e1003118. https://doi.org/10.1371/journal.pcbi.1003118.\n Li, H, and R Durbin. 2009. “Fast and Accurate Short Read Alignment with Burrows-Wheeler Transform.” Bioinformatics 25 (14): 1754–60. https://doi.org/10.1093/bioinformatics/btp324.\n Li, Heng. 2013. “Aligning Sequence Reads, Clone Sequences and Assembly Contigs with BWA-MEM.” arXiv [q-Bio.GN], March. http://arxiv.org/abs/1303.3997.\n Liao, Yang, Gordon K Smyth, and Wei Shi. 2013. “The Subread Aligner: Fast, Accurate and Scalable Read Mapping by Seed-and-Vote.” Nucleic Acids Res. 41 (10): e108. https://doi.org/10.1093/nar/gkt214.\n Wu, T D, and S Nacu. 2010. “Fast and SNP-tolerant Detection of Complex Variants and Splicing in Short Reads.” Bioinformatics 26 (7): 873–81. https://doi.org/10.1093/bioinformatics/btq057.\n  ","categories":"","description":"","excerpt":"document.addEventListener(\"DOMContentLoaded\", function() { …","ref":"/sp/spr/introduction/","tags":"","title":"Introduction"},{"body":" Main functionalities Currently, SPS includes 3 main functional categories (Fig 1):\n Some pre-defined modules (tabs) include: a. A workbench for designing and configuring data analysis workflows, b. Downstream analysis and visualization tools for RNA-Seq, and c. A space to make quick ggplots. A section with user custom tabs: users define their own shiny tabs. An image editing tab “Canvas” which allows users to edit plots made from the previous two categories.  Besides, SPS provides many functions to extend the default Shiny development, like more UI components, server functions. Also, SPS has some useful general R ulitlies like error catching, logging, and more.\n Figure 1. Design of SPS\nThe framework provides an interactive web interface for workflow management and data visualization.\nSPS tabs Within the functional categories, SPS functions are modularized into sub-components, here referred to as SPS tabs that are similar to menu tabs in other GUI applications that organize related and inter-connected functionalies into groups. On the backend, SPS tabs are based on Shiny modules, that are stored in separate files. This modular structure is highly extensible and greatly simplifies the design of new SPS tabs by both users and/or developers. Details about extending existing tabs and designing new ones are provided in Manage tabs section on our website.\n","categories":"","description":"","excerpt":" Main functionalities Currently, SPS includes 3 main functional …","ref":"/sps/intro/","tags":"","title":"Introduction"},{"body":"SPS Components package systemPipeShiny Components (spsComps) package is a collection of custom UI and server components that are used in SPS main framework. If you see a component in SPS but want to outside the SPS framework, like in your own Shiny apps, take a look at these components.\nDemos    Demo type source code     shiny{blk} shinyapps.io Github{blk}   Rmd Rmarkdown rendered Raw{blk}     table {font-size: 1.5rem}  Installation Read the developer tools main page, not repeating here.\nCategory {spsComps} can be divided into two major categories: UI and server.\n UI: Shiny or HTML UI components, for example, a box, a gallery, a button, etc. With these most of components, you do NOT need a server, so they are compatible with R markdown documents. See the UI page and its source code how we use the components in a Rmd doucment. However, some UI components has server side functions, mostly like updateXXX functions. Mainly these functions are used to update/change the UI based on user behaviors. It is totally okay to use the UI functions without the server functions (you will get static UI). server: can only be run in the Shiny server. These functions are designed to make back-end progress easier, extensions of original shiny Server functions.  Functions reference manual In documents, we only highlight some important functions. Please read the reference manuals for details of every function.\nsome screenshots of spsComps Animations Loaders Buttons Code display button Go top button Input buttons Button groups Table of buttons Gallery Logos Progress tracking Porgress panel Timeline ","categories":"","description":"","excerpt":"SPS Components package systemPipeShiny Components (spsComps) package …","ref":"/sps/dev/spscomps/","tags":"","title":"spsComps"},{"body":"","categories":"","description":"","excerpt":"","ref":"/sp/spr/","tags":"","title":"systemPipeR"},{"body":"                     For most of the UI components, you can view them in the online Shiny demo{blk}. Most but not all UI components work in a Rmarkdown document. Here we demostrate how you could use some of them in a Rmarkdown doc. The source code of this document is on Github{blk}.\nload package To start to use spsComps, load it in your Shiny app file or Rmarkdown file\nlibrary(spsComps)  ## Loading required package: shiny ## Loading required package: spsUtil  library(magrittr)  So you can see it depends on shiny and spsUtil. When you load it, there is no need to additionally load shiny or spsUtil.\nspsGoTop A go top button.\nspsGoTop()      It will not be display inline of the Rmd, just simply call it and maybe change the style as you want. By default, a “go to top” button will be created on the bottom-right corner. Now scroll this page, and you should see it (the rocket button).\ngallery texts \u003c- c(\"p1\", \"p2\", \"\", \"p4\", \"p5\") hrefs \u003c- c(\"https://github.com/lz100/spsComps/blob/master/img/1.jpg?raw=true\", \"https://github.com/lz100/spsComps/blob/master/img/2.jpg?raw=true\", \"\", \"https://github.com/lz100/spsComps/blob/master/img/4.jpg?raw=true\", \"https://github.com/lz100/spsComps/blob/master/img/5.jpg?raw=true\") images \u003c- c(\"https://github.com/lz100/spsComps/blob/master/img/1.jpg?raw=true\", \"https://github.com/lz100/spsComps/blob/master/img/2.jpg?raw=true\", \"https://github.com/lz100/spsComps/blob/master/img/3.jpg?raw=true\", \"https://github.com/lz100/spsComps/blob/master/img/4.jpg?raw=true\", \"https://github.com/lz100/spsComps/blob/master/img/5.jpg?raw=true\") gallery( texts = texts, hrefs = hrefs, images = images, enlarge = TRUE, enlarge_method = \"modal\" )  Gallery\np1\n  p2\n   \n  p4\n  p5\n   X    fixGalHeight(\"gallery2917503\")  You can show a gallery of plots you make in the Rmd and when people click it, it will be enlarged. You can also specify a link for each image.\nLogos a single one with hexLogo hexLogo( \"logo\", \"Logo\", hex_img = \"https://live.staticflickr.com/7875/46106952034_954b8775fa_b.jpg\", hex_link = \"https://www.google.com\", footer = \"Footer\", footer_link = \"https://www.google.com\" )  Logo\n    Footer   a panel of logos with hexPanel hexPanel( \"demo1\", \"\" , rep(\"https://live.staticflickr.com/7875/46106952034_954b8775fa_b.jpg\", 2) )              Buttons Some colorful buttons hrefTab hrefTab( title = \"Different background and text colors\", label_texts = c(\"Go top\", \"Disabled\", \"Email me\"), hrefs = c(\"#\", \"\", \"mailto:xxx@abc.com\"), bg_colors = c(\"green\", \"#eee\", \"orange\"), text_colors = c(\"#caffc1\", \"black\", \"blue\") )  Different background and text colors\nGo top  Disabled  Email me    A table colorful buttons hrefTable hrefTable( title = \"Change button color and text color\", item_titles = c(\"workflow 1\", \"No links\"), item_labels = list(c(\"tab 1\"), c(\"tab 3\", \"tab 4\")), item_hrefs = list(c(\"https://www.google.com/\"), c(\"\", \"\")), item_bg_colors = list(c(\"blue\"), c(\"red\", \"orange\")), item_text_colors = list(c(\"black\"), c(\"yellow\", \"green\")), style = \"display: table;\" )  Change button color and text color  Category Options     workflow 1 tab 1    No links tab 3 tab 4     hrefTable( title = \"Change row name colors and width\", item_titles = c(\"Green\", \"Red\", \"Orange\"), item_labels = list(c(\"tab 1\"), c(\"tab 3\", \"tab 4\"), c(\"tab 5\", \"tab 6\", \"tab 7\")), item_hrefs = list( c(\"https://www.google.com/\"), c(\"\", \"\"), c(\"https://www.google.com/\", \"https://www.google.com/\", \"\") ), item_title_colors = c(\"green\", \"red\", \"orange\"), style = \"display: table;\" )  Change row name colors and width  Category Options     Green tab 1    Red tab 3 tab 4    Orange tab 5 tab 6 tab 7     The table caption is on top in Shiny but on bottom in Rmd. You may also want to add the style = \"display: table;\" in Rmd to make the table occupy full length of the document in R markdown.\nshow tips with bsHoverPopover Space in a document is valuable. Sometimes you do not want to explain too much in the main text but still want to give readers some additional information. Use a popover to hide your additional text can be useful.\nOn a button actionButton(\"a_btn\", \"A button\", class = \"btn-primary\") %\u003e% bsHoverPopover( title = \"title a\", content = \"popover works on a button\", placement = \"bottom\" )  A button\nOn a link tags$a(href=\"mailto:xxx@abc.com\", \"Email\") %\u003e% bsHoverPopover( title = \"Email me\", content = \"popover works on a link\", placement = \"bottom\" )  Email\nOn a plot png(\"random_plot.png\") plot(1:10, 10:1) dev.off()  ## png ## 2  tags$img(src = \"../random_plot.png\") %\u003e% bsHoverPopover( title = \"My plot\", content = \"popover works on a plot\", placement = \"right\" )  Animations animateUI Add animations to existing components with animateUI\nTo buttons tags$button(id = \"btn1\", \"random button\")  random button\nanimateUI(\"btn1\", animation = \"ring\")  addSpsAnimation(\"#btn1\", \"sps-animation faa-ring animated \") To some text p(id = \"mytext\", class = \"text-red\", \"some move text\")  some move text\nanimateUI(\"mytext\", animation = \"horizontal\", speed = \"fast\")  addSpsAnimation(\"#mytext\", \"sps-animation faa-horizontal animated faa-fast\") On hover, move mouse on the red thumb tags$button( id = \"btn2\", icon(id = \"myicon\", \"thumbs-o-up\"), style = \"color: red; boarder: initial; border-color: transparent;\" )    animateUI(\"btn2\", animation = \"bounce\", speed = \"fast\", hover = TRUE)  addSpsAnimation(\"#btn2\", \"sps-animation faa-bounce animated-hover faa-fast\") Inline animation You can add animations to inline Rmarkdown text by giving it a HTML tag and id, like following:\nsome text some text \u003cspan id=\"some-text\" style=\"display: inline-block\"\u003esome text\u003c/span\u003e some text some text  some text some text some text some text some text\nanimateUI(selector = \"some-text\", animation = \"ring\")  addSpsAnimation(\"#some-text\", \"sps-animation faa-ring animated \") Most animations required the target tag to have CSS display “block” or “inline-block”, you can append this by adding style=\"display: inline-block\" to the tag as shown above or check examples below.\n animateAppend Add animations with pipe %\u003e% by animateAppend\nicon(\"home\") %\u003e% animateAppend(\"ring\")  \ntags$p(\"Append animation\", class = \"text-primary\", style=\"display: inline-block\") %\u003e% animateAppend(\"pulse\")  Append animation\n animateAppendNested Apply multiple animations to the same component\ntags$b(\"Nested animations\", class = \"text-primary\") %\u003e% animateAppendNested(\"ring\") %\u003e% animateAppendNested(\"pulse\") %\u003e% animateAppendNested(\"passing\")  Nested animations    tags$b(\"Nested animations display changed\", class = \"text-primary\") %\u003e% animateAppendNested(\"ring\") %\u003e% animateAppendNested(\"pulse\", display = \"block\", style = \"width: 30%\")  Nested animations display changed    animateIcon Here is a convenient function that allows you to create font-awesome icons with animations and customize, color, size, etc, an enhanced version of original shiny::icon and can also be used in Rmarkdown.\nDefault Default is the same as original icon\nanimateIcon(\"home\")  \nAnimation and color animateIcon(name = \"home\", animation = \"horizontal\", speed = \"slow\", color =\"red\")  \nAdd to a button tags$button(animateIcon(\"spinner\", \"spin\", \"fast\"), \"A button\")    A button  on hover animateIcon(name = \"wrench\", animation = \"wrench\", hover = TRUE, color =\"green\")  \nChange size animateIcon(\"home\", size = \"xs\")  \nanimateIcon(\"home\", size = \"sm\")  \nanimateIcon(\"home\", size = \"lg\")  \nanimateIcon(\"home\", size = \"2x\")  \nanimateIcon(\"home\", size = \"3x\")  \nanimateIcon(\"home\", size = \"5x\")  \nanimateIcon(\"home\", size = \"7x\")  \nanimateIcon(\"home\", size = \"10x\")  \n Loaders Add loaders to indicate busy status. Most cases, loaders are added by a backend server to show the busy processing status and are removed when the process is done. Rmarkdown documents does not have a server, but you can still add some loaders.\ncssLoader Default loaders There are 12 different default loaders: “circle,” “dual-ring,” “facebook,” “heart,” “ring,” “roller,” “default,” “ellipsis,” “grid,” “hourglass,” “ripple,” “spinner.”\ncssLoader(height = \"100px\")   $(function(){ $(\"#spsloader-857314629\").prepend(chooseLoader(\"spsloader-857314629\", \"default\", \"\", \"#337ab7\", \"100px\", \"100px\")); });   customize it:\ncssLoader(type = \"grid\", height = \"150px\", color = \"orange\")   $(function(){ $(\"#spsloader-481753296\").prepend(chooseLoader(\"spsloader-481753296\", \"grid\", \"\", \"orange\", \"150px\", \"150px\")); });   Add to a button:\ntags$button( ## `inline = TRUE` is important if you want loader and ## text in the same line. cssLoader(is_icon = TRUE, inline = TRUE, color = \"#3a7bd5\"), \"A button\" )    $(function(){ $(\"#spsloader-369175824\").prepend(chooseLoader(\"spsloader-369175824\", \"default\", \"\", \"#3a7bd5\", \"1.5rem\", \"1.5rem\")); });   A button  Other components Other components are either performed the best in a Shiny app or requires a server. Please see the demo\n","categories":"","description":"","excerpt":"                     For most of the UI components, you can view them …","ref":"/sps/dev/spscomps/ui/","tags":"","title":"UI components"},{"body":"Workflow management The workflow management module in SPS allows one to modify or create the configuration files required for running data analysis workflows in systemPipeR (SPR). This includes three types of important files: a sample metadata (targets) file, a workflow file (in R Markdown format) defining the workflow steps, and workflow running files in Common Workflow Language (CWL){blk} format. In SPS, one can easily create these files under the “Workflow Management” module, located in navigation bar on the left of the dashboard.\nThe current version of SPS allows to:\n create a workflow environment; create and/or check the format of targets / workflow / CWL files; download the prepared workflow files to run elsewhere, like a cluster; directly execute the workflow from SPS.  1. setup a workflow   Figure 3. A. Workflow Management - Targets File\n In the workflow module, read the instructions and choose step 1. Step 1 should be automatically opened for you on start. Choose a folder where you want to create the workflow environment. Choose a workflow template. These are SPR workflows and mainly are next-generation sequencing workflows. Click “Gen workflow” to create the workflow project. You should see a pop-up saying about the project path and other information. Clicking the pop-up will jump you to the step 2. The status tracker and banner for step 1 should all turn green.  2. Prepare a target file The targets file defines all input file paths and other sample information of analysis workflows. To better undertand the structure of this file, one can consult the “Structure of targets file” section in the SPR vignette. Essentially, this is the tabular file representation of the colData slot in an SummarizedExperiment object which stores sample IDs and other meta information.\nThe following step-by-step instructions explain how to create and/or modify targets files using RNA-Seq as an example (Fig.3 A):\n Your project targets file is loaded for you, but you can choose to upload a different one. You can edit, right click to add/remove rows/columns (The first row is treated as column names). SPR target file includes a header block, that can also be edited in the SPS app. Each headers needs to start with a “#”. Header is only useful for RNA-Seq workflow in current SPR. You can define sample comparison groups here. Leave it as default for other projects. The section on the left provides sample statistics and information whether files exist inside the workflow project’s data directory. Choose any column you want from the dropdown to check and watch the statistics bar change in this section. statistic status bar. Clicking on “Add to task” can help you to check if your target file has any formatting problem. You should see a green success pop-up if everything is right. Now your target file is ready and you can click “save” to download it and later use in a SPR project.    Figure 3. A. Workflow Management - Targets File\n3. Prepare a workflow file In SPR, workflows are defined in Rmarkdown files, you can read details and obtain them here.\nNow let us follow the order below to see how SPS helps you to prepare a workflow file for a RNAseq project (Fig.3 B):\n Your project workflow file is loaded for you, but you can choose to upload a different one. The workflow structure is displayed in a tree-leaf-like plot. Check all steps in the workflow that you want to include. You can skip (uncheck) some steps but it may cause the workflow to fail. Read more SPR instructions before do so. Clicking on the “Plot steps” will show a flow chart of what the step execution orders will be when you run the workflow in SPR. Clicking “Report preview” generates a preview of what the final report will look like for current RNAseq workflow (hidden in Fig 3.B), but in the preview, no code is evaluated. If you are satisfied with your workflow file, click “Add to task” to save your workflow file.    Figure 3. B. Workflow Management - Workflow File\n4. Prepare CWL files (optional) In the new version of SPR, all individual system workflow steps are called by the CWL files. Each SPR workflow has a set of CWL files and normally users do not need to make any change. In case you want to learn more about CWL and create some new CWL files, Step 4 is a good place to practice.\nTo run a CWL step in SPR, 3 files are required:\n targets: to determine how many samples will be run and sample names. CWL running file: can be translated to bash code; CWL input: variables to inject into the running file  SPR is the parser between R and CWL by injecting sample information from targets to CWL input file and then CWL parser translates it to bash code.\n Most people are not familiar this part, so read instructions carefully. Your project targets has been loaded for you, and an example CWL running and input for hisat2 is also loaded for you. Directly parse the code. See what commandline code you get. Change the targets injecting column, and parse again, see what has changed. You can edit the CWL running and input files Try to parse the new file and see what has changed. If new CWL files has been created, you can edit workflow Rmd files by adding your new steps.    Figure 3. C. Workflow Management - CWL File\n5. Run or finish workflow preparation Up until this step, congratulations, the workflow is prepared. You can choose to download the workflow project files as a bundle or continue to run the workflow.\n  Figure 4.A.B Workflow Runner\n On step 5 you can choose to download the prepared workflow or directly run the workflow. However, if you do not have the required commandline tools, workflow will most likely fail. Make sure you system has these tools (Read about these tools). Open up the runner. It is a “Rstudio-like” interface. Code editor. Required workflow running code is pre-entered for you. You can simply hit “Run” to start. Of course, you can delete the default code and run random R code. Output R console. Workflow running log. View any plot output. and send a copy of your current plot to SPS Canvas tab or download it.  App security Running the workflow may introduce some security concerns. Read App Security for more details.\nis_demo option The is_demo option will impact the workflow module.\n TRUE: you are not deploying the app for production purpose. It is just a show case. Under this option, users who are trying to create a workflow will be locked inside a temp directory and every time they start a new session, they will be assigned to a new temp directory. This is useful is many people want to try the app the same time, so they will have private different environment, and the temp directory will be removed afterwards. FALSE: you are using the app on your own local computer or use it in a production environment. WF module will have full access to the sever storage system and users can choose any place they have permission to create the workflow session.  ","categories":"","description":"","excerpt":"Workflow management The workflow management module in SPS allows one …","ref":"/sps/modules/workflow/","tags":"","title":"Workflow"},{"body":"pre code { white-space: pre !important; overflow-x: scroll !important; word-break: keep-all !important; word-wrap: initial !important; }  document.addEventListener(\"DOMContentLoaded\", function() { document.querySelector(\"h1\").className = \"title\"; });  document.addEventListener(\"DOMContentLoaded\", function() { var links = document.links; for (var i = 0, linksLength = links.length; i Note: the most recent version of this vignette can be found here.\nNote: if you use systemPipeR and systemPipeRdata in published research, please cite:\nBackman, T.W.H and Girke, T. (2016). systemPipeR: Workflow and Report Generation Environment. BMC Bioinformatics, 17: 388. 10.1186/s12859-016-1241-0.\nIntroduction systemPipeRdata is a helper package to generate with a single command workflow templates that are intended to be used by its parent package systemPipeR (H Backman and Girke 2016). The systemPipeR project provides a suite of R/Bioconductor packages for designing, building and running end-to-end analysis workflows on local machines, HPC clusters and cloud systems, while generating at the same time publication quality analysis reports.\nTo test workflows quickly or design new ones from existing templates, users can generate with a single command workflow instances fully populated with sample data and parameter files required for running a chosen workflow. Pre-configured directory structure of the workflow environment and the sample data used by systemPipeRdata are described here.\nGetting started Installation The systemPipeRdata package is available at Bioconductor and can be installed from within R as follows:\nif (!requireNamespace(\"BiocManager\", quietly = TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"systemPipeRdata\")  Also, it is possible to install the development version from Bioconductor.\nBiocManager::install(\"systemPipeRdata\", version = \"devel\", build_vignettes = TRUE, dependencies = TRUE) # Installs Devel version from Bioconductor  Loading package and documentation library(\"systemPipeRdata\") # Loads the package  library(help = \"systemPipeRdata\") # Lists package info vignette(\"systemPipeRdata\") # Opens vignette  Starting with pre-configured workflow templates Load one of the available workflows into your current working directory. The following does this for the varseq workflow template. The name of the resulting workflow directory can be specified under the mydirname argument. The default NULL uses the name of the chosen workflow. An error is issued if a directory of the same name and path exists already.\ngenWorkenvir(workflow = \"systemPipeR/SPvarseq\", mydirname = \"varseq\") setwd(\"varseq\")  On Linux and OS X systems the same can be achieved from the command-line of a terminal with the following commands.\n$ Rscript -e \"systemPipeRdata::genWorkenvir(workflow='systemPipeR/SPvarseq', mydirname='varseq')\"  Workflow templates collection A collection of workflow templates are available, and it is possible to browse the current availability, as follows:\navailableWF(github = TRUE)  This function returns the list of workflow templates available within the package and systemPipeR Organization on GitHub. Each one listed template can be created as described above.\nThe workflow template choose from Github will be installed as an R package, and also it creates the environment with all the settings and files to run the demo analysis.\ngenWorkenvir(workflow=\"systemPipeR/SPrnaseq\", mydirname=\"NULL\") setwd(\"SPrnaseq\")  Besides, it is possible to choose different versions of the workflow template, defined through other branches on the GitHub Repository. By default, the master branch is selected, however, it is possible to define a different branch with the ref argument.\ngenWorkenvir(workflow=\"systemPipeR/SPrnaseq\", ref = \"singleMachine\") setwd(\"SPrnaseq\")  Download a specific R Markdown file Also, it is possible to download a specific workflow script for your analysis. The URL can be specified under url argument and the R Markdown file name in the urlname argument. The default NULL copies the current version available in the chose template.\ngenWorkenvir(workflow=\"systemPipeR/SPrnaseq\", url = \"https://raw.githubusercontent.com/systemPipeR/systemPipeRNAseq/cluster/vignettes/systemPipeRNAseq.Rmd\", urlname = \"rnaseq_V-cluster.Rmd\") setwd(\"rnaseq\")  Dynamic generation of workflow template It is possible to create a new workflow structure from RStudio menu File -\u003e New File -\u003e R Markdown -\u003e From Template -\u003e systemPipeR New WorkFlow. This interactive option creates the same environment as demonstrated above.\nFigure 1: Selecting workflow template within RStudio.\nDirectory Structure The workflow templates generated by genWorkenvir contain the following preconfigured directory structure:\n workflow/ (e.g. rnaseq/)  This is the root directory of the R session running the workflow. Run script ( *.Rmd) and sample annotation (targets.txt) files are located here. Note, this directory can have any name (e.g. rnaseq, varseq). Changing its name does not require any modifications in the run script(s). Important subdirectories:  param/  Stores non-CWL parameter files such as: *.param, *.tmpl and *.run.sh. These files are only required for backwards compatibility to run old workflows using the previous custom command-line interface. param/cwl/: This subdirectory stores all the CWL parameter files. To organize workflows, each can have its own subdirectory, where all CWL param and input.yml files need to be in the same subdirectory.   data/   FASTQ files FASTA file of reference (e.g. reference genome) Annotation files etc.   results/  Analysis results are usually written to this directory, including: alignment, variant and peak files (BAM, VCF, BED); tabular result files; and image/plot files Note, the user has the option to organize results files for a given sample and analysis step in a separate subdirectory.        Note: Directory names are indicated in green. Users can change this structure as needed, but need to adjust the code in their workflows accordingly.\n  Figure 2: systemPipeR’s preconfigured directory structure.\nRun workflows Next, run from within R the chosen sample workflow by executing the code provided in the corresponding *.Rmd* template file. Much more detailed information on running and customizing systemPipeR* workflows is available in its overview vignette here. This vignette can also be opened from R with the following command.\nlibrary(\"systemPipeR\") # Loads systemPipeR which needs to be installed via BiocManager::install() from Bioconductor  vignette(\"systemPipeR\", package = \"systemPipeR\")  Return paths to sample data The location of the sample data provided by systemPipeRdata can be returned as a list.\npathList()  ## $targets ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/param/targets.txt\" ## ## $targetsPE ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/param/targetsPE.txt\" ## ## $annotationdir ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/annotation/\" ## ## $fastqdir ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/fastq/\" ## ## $bamdir ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/bam/\" ## ## $paramdir ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/param/\" ## ## $workflows ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/workflows/\" ## ## $chipseq ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/workflows/chipseq/\" ## ## $rnaseq ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/workflows/rnaseq/\" ## ## $riboseq ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/workflows/riboseq/\" ## ## $varseq ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/workflows/varseq/\" ## ## $new ## [1] \"/home/dcassol/src/R-devel/library/systemPipeRdata/extdata/workflows/new/\"  Version information sessionInfo()  ## R Under development (unstable) (2021-02-04 r79940) ## Platform: x86_64-pc-linux-gnu (64-bit) ## Running under: Ubuntu 20.04.2 LTS ## ## Matrix products: default ## BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 ## LAPACK: /home/dcassol/src/R-devel/lib/libRlapack.so ## ## locale: ## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C ## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 ## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 ## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C ## [9] LC_ADDRESS=C LC_TELEPHONE=C ## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C ## ## attached base packages: ## [1] stats graphics grDevices utils datasets ## [6] methods base ## ## other attached packages: ## [1] systemPipeRdata_1.19.2 BiocStyle_2.19.2 ## ## loaded via a namespace (and not attached): ## [1] knitr_1.33 magrittr_2.0.1 ## [3] BiocGenerics_0.37.4 R6_2.5.0 ## [5] rlang_0.4.11 stringr_1.4.0 ## [7] tools_4.1.0 parallel_4.1.0 ## [9] xfun_0.22 jquerylib_0.1.4 ## [11] htmltools_0.5.1.1 remotes_2.3.0 ## [13] yaml_2.2.1 digest_0.6.27 ## [15] bookdown_0.22 formatR_1.9 ## [17] BiocManager_1.30.12 sass_0.3.1 ## [19] codetools_0.2-18 evaluate_0.14 ## [21] rmarkdown_2.7.12 blogdown_1.3 ## [23] stringi_1.5.3 compiler_4.1.0 ## [25] bslib_0.2.4 jsonlite_1.7.2  Funding This project was supported by funds from the National Institutes of Health (NIH) and the National Science Foundation (NSF).\nReferences H Backman, Tyler W, and Thomas Girke. 2016. “systemPipeR: NGS workflow and report generation environment.” BMC Bioinformatics 17 (1): 388. https://doi.org/10.1186/s12859-016-1241-0.\n  ","categories":"","description":"","excerpt":"pre code { white-space: pre !important; overflow-x: scroll !important; …","ref":"/sp/sprdata/systempiperdata/","tags":"","title":"Workflow templates and sample data"},{"body":"Demo    Demo demo Source code     shinyapps.io github    Instructions Instructions on how to use {drawer} and the capture buttons are provided in the demo Shiny app in details. Click on the show code button on each tab in the demo app for code to reproduce.\n","categories":"","description":"","excerpt":"Demo    Demo demo Source code     shinyapps.io github    Instructions …","ref":"/sps/dev/drawer/shiny/","tags":"","title":"drawer in Shiny"},{"body":" document.addEventListener(\"DOMContentLoaded\", function() { document.querySelector(\"h1\").className = \"title\"; });  document.addEventListener(\"DOMContentLoaded\", function() { var links = document.links; for (var i = 0, linksLength = links.length; i Getting Started Installation The R software for running systemPipeR can be downloaded from CRAN. The systemPipeR* environment can be installed from the R console using the BiocManager::install* command. The associated data package systemPipeRdata can be installed the same way. The latter is a helper package for generating systemPipeR workflow environments with a single command containing all parameter files and sample data required to quickly test and run workflows.\nif (!requireNamespace(\"BiocManager\", quietly = TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"systemPipeR\") BiocManager::install(\"systemPipeRdata\")  Please note that if you desire to use a third-party command line tool, the particular tool and dependencies need to be installed and exported in your PATH. See details.\nLoading package and documentation library(\"systemPipeR\") # Loads the package library(help = \"systemPipeR\") # Lists package info vignette(\"systemPipeR\") # Opens vignette  Load sample data and workflow templates The mini sample FASTQ files used by this overview vignette as well as the associated workflow reporting vignettes can be loaded via the systemPipeRdata package as shown below. The chosen data set SRP010938 obtains 18 paired-end (PE) read sets from Arabidposis thaliana (Howard et al. 2013). To minimize processing time during testing, each FASTQ file has been subsetted to 90,000-100,000 randomly sampled PE reads that map to the first 100,000 nucleotides of each chromosome of the A. thalina genome. The corresponding reference genome sequence (FASTA) and its GFF annotation files (provided in the same download) have been truncated accordingly. This way the entire test sample data set requires less than 200MB disk storage space. A PE read set has been chosen for this test data set for flexibility, because it can be used for testing both types of analysis routines requiring either SE (single-end) reads or PE reads.\nThe following generates a fully populated systemPipeR workflow environment (here for RNA-Seq) in the current working directory of an R session. At this time the package includes workflow templates for RNA-Seq, ChIP-Seq, VAR-Seq, and Ribo-Seq. Templates for additional NGS applications will be provided in the future.\nlibrary(systemPipeRdata) genWorkenvir(workflow = \"rnaseq\") setwd(\"rnaseq\")  If you desire run this tutorial with your data set, please follow the instruction here:\nlibrary(systemPipeRdata) genWorkenvir(workflow = \"new\", mydirname = \"FEB_project\")  Workflow template from an individual’s package The package provides pre-configured workflows and reporting templates for a wide range of NGS applications that are listed here. Additional workflow templates will be provided in the future. If you desire to use an individual package and version, follow the instruction below:\nlibrary(systemPipeRdata) genWorkenvir(workflow = NULL, package_repo = \"systemPipeR/systemPipeRIBOseq\", ref = \"master\", subdir = NULL)  library(systemPipeRdata) genWorkenvir(workflow = NULL, package_repo = \"systemPipeR/systemPipeRNAseq\", ref = \"singleMachine\", subdir = NULL)  Directory Structure The working environment of the sample data loaded in the previous step contains the following pre-configured directory structure (Figure 4). Directory names are indicated in green. Users can change this structure as needed, but need to adjust the code in their workflows accordingly.\n workflow/ (e.g. rnaseq/)  This is the root directory of the R session running the workflow. Run script ( *.Rmd) and sample annotation (targets.txt) files are located here. Note, this directory can have any name (e.g. rnaseq, varseq). Changing its name does not require any modifications in the run script(s). Important subdirectories:  param/  Stores non-CWL parameter files such as: *.param, *.tmpl and *.run.sh. These files are only required for backwards compatibility to run old workflows using the previous custom command-line interface. param/cwl/: This subdirectory stores all the CWL parameter files. To organize workflows, each can have its own subdirectory, where all CWL param and input.yml files need to be in the same subdirectory.   data/   FASTQ files FASTA file of reference (e.g. reference genome) Annotation files etc.   results/  Analysis results are usually written to this directory, including: alignment, variant and peak files (BAM, VCF, BED); tabular result files; and image/plot files Note, the user has the option to organize results files for a given sample and analysis step in a separate subdirectory.          Figure 5: systemPipeR’s preconfigured directory structure.\nThe following parameter files are included in each workflow template:\n targets.txt: initial one provided by user; downstream targets_*.txt files are generated automatically *.param/cwl: defines parameter for input/output file operations, e.g.:  hisat2-se/hisat2-mapping-se.cwl hisat2-se/hisat2-mapping-se.yml   *_run.sh: optional bash scripts Configuration files for computer cluster environments (skip on single machines):  .batchtools.conf.R: defines the type of scheduler for batchtools pointing to template file of cluster, and located in user’s home directory *.tmpl: specifies parameters of scheduler used by a system, e.g. Torque, SGE, Slurm, etc.    Structure of targets file The targets file defines all input files (e.g. FASTQ, BAM, BCF) and sample comparisons of an analysis workflow. The following shows the format of a sample targets file included in the package. It also can be viewed and downloaded from systemPipeR’s GitHub repository here. In a target file with a single type of input files, here FASTQ files of single-end (SE) reads, the first three columns are mandatory including their column names, while it is four mandatory columns for FASTQ files of PE reads. All subsequent columns are optional and any number of additional columns can be added as needed.\nUsers should note here, the usage of targets files is optional when using systemPipeR’s new CWL interface. They can be replaced by a standard YAML input file used by CWL. Since for organizing experimental variables targets files are extremely useful and user-friendly. Thus, we encourage users to keep using them.\nStructure of targets file for single-end (SE) samples library(systemPipeR) targetspath \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") read.delim(targetspath, comment.char = \"#\")[1:4, ]  ## FileName SampleName Factor SampleLong Experiment ## 1 ./data/SRR446027_1.fastq.gz M1A M1 Mock.1h.A 1 ## 2 ./data/SRR446028_1.fastq.gz M1B M1 Mock.1h.B 1 ## 3 ./data/SRR446029_1.fastq.gz A1A A1 Avr.1h.A 1 ## 4 ./data/SRR446030_1.fastq.gz A1B A1 Avr.1h.B 1 ## Date ## 1 23-Mar-2012 ## 2 23-Mar-2012 ## 3 23-Mar-2012 ## 4 23-Mar-2012  To work with custom data, users need to generate a targets file containing the paths to their own FASTQ files and then provide under targetspath the path to the corresponding targets file.\nStructure of targets file for paired-end (PE) samples For paired-end (PE) samples, the structure of the targets file is similar, where users need to provide two FASTQ path columns: FileName1 and FileName2 with the paths to the PE FASTQ files.\ntargetspath \u003c- system.file(\"extdata\", \"targetsPE.txt\", package = \"systemPipeR\") read.delim(targetspath, comment.char = \"#\")[1:2, 1:6]  ## FileName1 FileName2 SampleName Factor ## 1 ./data/SRR446027_1.fastq.gz ./data/SRR446027_2.fastq.gz M1A M1 ## 2 ./data/SRR446028_1.fastq.gz ./data/SRR446028_2.fastq.gz M1B M1 ## SampleLong Experiment ## 1 Mock.1h.A 1 ## 2 Mock.1h.B 1  Sample comparisons Sample comparisons are defined in the header lines of the targets file starting with ‘# \u003cCMP\u003e.’\nreadLines(targetspath)[1:4]  ## [1] \"# Project ID: Arabidopsis - Pseudomonas alternative splicing study (SRA: SRP010938; PMID: 24098335)\" ## [2] \"# The following line(s) allow to specify the contrasts needed for comparative analyses, such as DEG identification. All possible comparisons can be specified with 'CMPset: ALL'.\" ## [3] \"# \u003cCMP\u003e CMPset1: M1-A1, M1-V1, A1-V1, M6-A6, M6-V6, A6-V6, M12-A12, M12-V12, A12-V12\" ## [4] \"# \u003cCMP\u003e CMPset2: ALL\"  The function readComp imports the comparison information and stores it in a list. Alternatively, readComp can obtain the comparison information from the corresponding SYSargs object (see below). Note, these header lines are optional. They are mainly useful for controlling comparative analyses according to certain biological expectations, such as identifying differentially expressed genes in RNA-Seq experiments based on simple pair-wise comparisons.\nreadComp(file = targetspath, format = \"vector\", delim = \"-\")  ## $CMPset1 ## [1] \"M1-A1\" \"M1-V1\" \"A1-V1\" \"M6-A6\" \"M6-V6\" \"A6-V6\" \"M12-A12\" ## [8] \"M12-V12\" \"A12-V12\" ## ## $CMPset2 ## [1] \"M1-A1\" \"M1-V1\" \"M1-M6\" \"M1-A6\" \"M1-V6\" \"M1-M12\" \"M1-A12\" ## [8] \"M1-V12\" \"A1-V1\" \"A1-M6\" \"A1-A6\" \"A1-V6\" \"A1-M12\" \"A1-A12\" ## [15] \"A1-V12\" \"V1-M6\" \"V1-A6\" \"V1-V6\" \"V1-M12\" \"V1-A12\" \"V1-V12\" ## [22] \"M6-A6\" \"M6-V6\" \"M6-M12\" \"M6-A12\" \"M6-V12\" \"A6-V6\" \"A6-M12\" ## [29] \"A6-A12\" \"A6-V12\" \"V6-M12\" \"V6-A12\" \"V6-V12\" \"M12-A12\" \"M12-V12\" ## [36] \"A12-V12\"  Structure of the new param files and construct SYSargs2 container SYSargs2 stores all the information and instructions needed for processing a set of input files with a single or many command-line steps within a workflow (i.e. several components of the software or several independent software tools). The SYSargs2 object is created and fully populated with the loadWF and renderWF functions, respectively.\nIn CWL, files with the extension .cwl define the parameters of a chosen command-line step or workflow, while files with the extension .yml define the input variables of command-line steps. Note, input variables provided by a targets file can be passed on to a SYSargs2 instance via the inputvars argument of the renderWF function.\nThe following imports a .cwl file (here hisat2-mapping-se.cwl) for running the short read aligner HISAT2 (Kim, Langmead, and Salzberg 2015). The loadWorkflow and renderWF functions render the proper command-line strings for each sample and software tool.\nlibrary(systemPipeR) targets \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/hisat2/hisat2-se\", package = \"systemPipeR\") WF \u003c- loadWF(targets = targets, wf_file = \"hisat2-mapping-se.cwl\", input_file = \"hisat2-mapping-se.yml\", dir_path = dir_path) WF \u003c- renderWF(WF, inputvars = c(FileName = \"_FASTQ_PATH1_\", SampleName = \"_SampleName_\"))  Several accessor methods are available that are named after the slot names of the SYSargs2 object.\nnames(WF)  ## [1] \"targets\" \"targetsheader\" \"modules\" \"wf\" ## [5] \"clt\" \"yamlinput\" \"cmdlist\" \"input\" ## [9] \"output\" \"cwlfiles\" \"inputvars\"  Of particular interest is the cmdlist() method. It constructs the system commands for running command-line software as specified by a given .cwl file combined with the paths to the input samples (e.g. FASTQ files) provided by a targets file. The example below shows the cmdlist() output for running HISAT2 on the first SE read sample. Evaluating the output of cmdlist() can be very helpful for designing and debugging .cwl files of new command-line software or changing the parameter settings of existing ones.\ncmdlist(WF)[1]  ## $M1A ## $M1A$`hisat2-mapping-se` ## [1] \"hisat2 -S ./results/M1A.sam -x ./data/tair10.fasta -k 1 --min-intronlen 30 --max-intronlen 3000 -U ./data/SRR446027_1.fastq.gz --threads 4\"  The output components of SYSargs2 define the expected output files for each step in the workflow; some of which are the input for the next workflow step, here next SYSargs2 instance (see Figure 2).\noutput(WF)[1]  ## $M1A ## $M1A$`hisat2-mapping-se` ## [1] \"./results/M1A.sam\"  modules(WF)  ## module1 ## \"hisat2/2.1.0\"  targets(WF)[1]  ## $M1A ## $M1A$FileName ## [1] \"./data/SRR446027_1.fastq.gz\" ## ## $M1A$SampleName ## [1] \"M1A\" ## ## $M1A$Factor ## [1] \"M1\" ## ## $M1A$SampleLong ## [1] \"Mock.1h.A\" ## ## $M1A$Experiment ## [1] 1 ## ## $M1A$Date ## [1] \"23-Mar-2012\"  targets.as.df(targets(WF))[1:4, 1:4]  ## FileName SampleName Factor SampleLong ## 1 ./data/SRR446027_1.fastq.gz M1A M1 Mock.1h.A ## 2 ./data/SRR446028_1.fastq.gz M1B M1 Mock.1h.B ## 3 ./data/SRR446029_1.fastq.gz A1A A1 Avr.1h.A ## 4 ./data/SRR446030_1.fastq.gz A1B A1 Avr.1h.B  output(WF)[1]  ## $M1A ## $M1A$`hisat2-mapping-se` ## [1] \"./results/M1A.sam\"  cwlfiles(WF)  ## $cwl ## [1] \"/home/dcassol/src/R-devel/library/systemPipeR/extdata/cwl/hisat2/hisat2-se/hisat2-mapping-se.cwl\" ## ## $yml ## [1] \"/home/dcassol/src/R-devel/library/systemPipeR/extdata/cwl/hisat2/hisat2-se/hisat2-mapping-se.yml\" ## ## $steps ## [1] \"hisat2-mapping-se\" ## ## $targets ## [1] \"/home/dcassol/src/R-devel/library/systemPipeR/extdata/targets.txt\"  inputvars(WF)  ## $FileName ## [1] \"_FASTQ_PATH1_\" ## ## $SampleName ## [1] \"_SampleName_\"  In an ‘R-centric’ rather than a ‘CWL-centric’ workflow design the connectivity among workflow steps is established by writing all relevant output with the writeTargetsout function to a new targets file that serves as input to the next loadWorkflow and renderWF call. By chaining several SYSargs2 steps together one can construct complex workflows involving many sample-level input/output file operations with any combination of command-line or R-based software. Alternatively, a CWL-centric workflow design can be used that defines all/most workflow steps with CWL workflow and parameter files. Due to time and space restrictions, the CWL-centric approach is not covered by this tutorial.\nThird-party software tools Current, systemPipeR provides the param file templates for third-party software tools. Please check the listed software tools.\n  Tool Name  Description  Step      bwa  BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome.  Alignment    Bowtie2  Bowtie 2 is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences.  Alignment    FASTX-Toolkit  FASTX-Toolkit is a collection of command line tools for Short-Reads FASTA/FASTQ files preprocessing.  Read Preprocessing    TransRate  Transrate is software for de-novo transcriptome assembly quality analysis.  Quality    Gsnap  GSNAP is a genomic short-read nucleotide alignment program.  Alignment    Samtools  Samtools is a suite of programs for interacting with high-throughput sequencing data.  Post-processing    Trimmomatic  Trimmomatic is a flexible read trimming tool for Illumina NGS data.  Read Preprocessing    Rsubread  Rsubread is a Bioconductor software package that provides high-performance alignment and read counting functions for RNA-seq reads.  Alignment    Picard  Picard is a set of command line tools for manipulating high-throughput sequencing (HTS) data and formats such as SAM/BAM/CRAM and VCF.  Manipulating HTS data    Busco  BUSCO assesses genome assembly and annotation completeness with Benchmarking Universal Single-Copy Orthologs.  Quality    Hisat2  HISAT2 is a fast and sensitive alignment program for mapping NGS reads (both DNA and RNA) to reference genomes.  Alignment    Tophat2  TopHat is a fast splice junction mapper for RNA-Seq reads.  Alignment    GATK  Variant Discovery in High-Throughput Sequencing Data.  Variant Discovery    STAR  STAR is an ultrafast universal RNA-seq aligner.  Alignment    Trim\\_galore  Trim Galore is a wrapper around Cutadapt and FastQC to consistently apply adapter and quality trimming to FastQ files.  Read Preprocessing    TransDecoder  TransDecoder identifies candidate coding regions within transcript sequences.  Find Coding Regions    Trinity  Trinity assembles transcript sequences from Illumina RNA-Seq data.  denovo Transcriptome Assembly    Trinotate  Trinotate is a comprehensive annotation suite designed for automatic functional annotation of transcriptomes.  Transcriptome Functional Annotation    MACS2  MACS2 identifies transcription factor binding sites in ChIP-seq data.  Peak calling    Kallisto  kallisto is a program for quantifying abundances of transcripts from RNA-Seq data.  Read counting    BCFtools  BCFtools is a program for variant calling and manipulating files in the Variant Call Format (VCF) and its binary counterpart BCF.  Variant Discovery    Bismark  Bismark is a program to map bisulfite treated sequencing reads to a genome of interest and perform methylation calls in a single step.  Bisulfite mapping    Fastqc  FastQC is a quality control tool for high throughput sequence data.  Quality    Blast  BLAST finds regions of similarity between biological sequences.  Blast      Remember, if you desire to run any of these tools, make sure to have the respective software installed on your system and configure in the PATH. You can check as follows:\ntryCL(command = \"grep\")  Structure of param file and SYSargs container (Previous version) The param file defines the parameters of a chosen command-line software. The following shows the format of a sample param file provided by this package.\nparampath \u003c- system.file(\"extdata\", \"tophat.param\", package = \"systemPipeR\") read.delim(parampath, comment.char = \"#\")  ## PairSet Name Value ## 1 modules \u003cNA\u003e bowtie2/2.2.5 ## 2 modules \u003cNA\u003e tophat/2.0.14 ## 3 software \u003cNA\u003e tophat ## 4 cores -p 4 ## 5 other \u003cNA\u003e -g 1 --segment-length 25 -i 30 -I 3000 ## 6 outfile1 -o \u003cFileName1\u003e ## 7 outfile1 path ./results/ ## 8 outfile1 remove \u003cNA\u003e ## 9 outfile1 append .tophat ## 10 outfile1 outextension .tophat/accepted_hits.bam ## 11 reference \u003cNA\u003e ./data/tair10.fasta ## 12 infile1 \u003cNA\u003e \u003cFileName1\u003e ## 13 infile1 path \u003cNA\u003e ## 14 infile2 \u003cNA\u003e \u003cFileName2\u003e ## 15 infile2 path \u003cNA\u003e  The systemArgs function imports the definitions of both the param file and the targets file, and stores all relevant information in a SYSargs object (S4 class). To run the pipeline without command-line software, one can assign NULL to sysma instead of a param file. In addition, one can start systemPipeR workflows with pre-generated BAM files by providing a targets file where the FileName column provides the paths to the BAM files. Note, in the following example the usage of suppressWarnings() is only relevant for building this vignette. In typical workflows it should be removed.\ntargetspath \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") args \u003c- suppressWarnings(systemArgs(sysma = parampath, mytargets = targetspath)) args  ## An instance of 'SYSargs' for running 'tophat' on 18 samples  Several accessor methods are available that are named after the slot names of the SYSargs object.\nnames(args)  ## [1] \"targetsin\" \"targetsout\" \"targetsheader\" \"modules\" ## [5] \"software\" \"cores\" \"other\" \"reference\" ## [9] \"results\" \"infile1\" \"infile2\" \"outfile1\" ## [13] \"sysargs\" \"outpaths\"  Of particular interest is the sysargs() method. It constructs the system commands for running command-lined software as specified by a given param file combined with the paths to the input samples (e.g. FASTQ files) provided by a targets file. The example below shows the sysargs() output for running TopHat2 on the first PE read sample. Evaluating the output of sysargs() can be very helpful for designing and debugging param files of new command-line software or changing the parameter settings of existing ones.\nsysargs(args)[1]  ## M1A ## \"tophat -p 4 -g 1 --segment-length 25 -i 30 -I 3000 -o /home/dcassol/danielac@ucr.edu/projects/SP/SPR_WF_org/systemPipeR.github.io_docsy/content/en/sp/spr/results/SRR446027_1.fastq.gz.tophat /home/dcassol/danielac@ucr.edu/projects/SP/SPR_WF_org/systemPipeR.github.io_docsy/content/en/sp/spr/data/tair10.fasta ./data/SRR446027_1.fastq.gz \"  modules(args)  ## [1] \"bowtie2/2.2.5\" \"tophat/2.0.14\"  cores(args)  ## [1] 4  outpaths(args)[1]  ## M1A ## \"/home/dcassol/danielac@ucr.edu/projects/SP/SPR_WF_org/systemPipeR.github.io_docsy/content/en/sp/spr/results/SRR446027_1.fastq.gz.tophat/accepted_hits.bam\"  The content of the param file can also be returned as JSON object as follows (requires rjson package).\nsystemArgs(sysma = parampath, mytargets = targetspath, type = \"json\")  ## [1] \"{\\\"modules\\\":{\\\"n1\\\":\\\"\\\",\\\"v2\\\":\\\"bowtie2/2.2.5\\\",\\\"n1\\\":\\\"\\\",\\\"v2\\\":\\\"tophat/2.0.14\\\"},\\\"software\\\":{\\\"n1\\\":\\\"\\\",\\\"v1\\\":\\\"tophat\\\"},\\\"cores\\\":{\\\"n1\\\":\\\"-p\\\",\\\"v1\\\":\\\"4\\\"},\\\"other\\\":{\\\"n1\\\":\\\"\\\",\\\"v1\\\":\\\"-g 1 --segment-length 25 -i 30 -I 3000\\\"},\\\"outfile1\\\":{\\\"n1\\\":\\\"-o\\\",\\\"v2\\\":\\\"\u003cFileName1\u003e\\\",\\\"n3\\\":\\\"path\\\",\\\"v4\\\":\\\"./results/\\\",\\\"n5\\\":\\\"remove\\\",\\\"v1\\\":\\\"\\\",\\\"n2\\\":\\\"append\\\",\\\"v3\\\":\\\".tophat\\\",\\\"n4\\\":\\\"outextension\\\",\\\"v5\\\":\\\".tophat/accepted_hits.bam\\\"},\\\"reference\\\":{\\\"n1\\\":\\\"\\\",\\\"v1\\\":\\\"./data/tair10.fasta\\\"},\\\"infile1\\\":{\\\"n1\\\":\\\"\\\",\\\"v2\\\":\\\"\u003cFileName1\u003e\\\",\\\"n1\\\":\\\"path\\\",\\\"v2\\\":\\\"\\\"},\\\"infile2\\\":{\\\"n1\\\":\\\"\\\",\\\"v2\\\":\\\"\u003cFileName2\u003e\\\",\\\"n1\\\":\\\"path\\\",\\\"v2\\\":\\\"\\\"}}\"  References Howard, Brian E, Qiwen Hu, Ahmet Can Babaoglu, Manan Chandra, Monica Borghi, Xiaoping Tan, Luyan He, et al. 2013. “High-Throughput RNA Sequencing of Pseudomonas-Infected Arabidopsis Reveals Hidden Transcriptome Complexity and Novel Splice Variants.” PLoS One 8 (10): e74183. https://doi.org/10.1371/journal.pone.0074183.\n Kim, Daehwan, Ben Langmead, and Steven L Salzberg. 2015. “HISAT: A Fast Spliced Aligner with Low Memory Requirements.” Nat. Methods 12 (4): 357–60.\n  ","categories":"","description":"","excerpt":" document.addEventListener(\"DOMContentLoaded\", function() { …","ref":"/sp/spr/gettingstarted/","tags":"","title":"Getting Started"},{"body":"Full if (!requireNamespace(\"BiocManager\", quietly=TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"systemPipeShiny\", dependencies=TRUE)  This will install all required packages including suggested packages that are required by the core modules. Be aware, it will take quite some time if you are installing on Linux where only source installation is available. Windows and Mac binary installations will be much faster.\nMinimum To install the package, please use the BiocManager::install command:\nif (!requireNamespace(\"BiocManager\", quietly=TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"systemPipeShiny\")  By the minimum installation, all the 3 core modules are not installed. You can still start the app, and When you start the app and click on these modules, it will tell to enable these modules, what packages and command you need to run. Just follow the instructions. So, install as you need.\nMost recent To obtain the most recent updates immediately, one can install it directly from GitHub{blk} as follow:\nif (!requireNamespace(\"remotes\", quietly=TRUE)) install.packages(\"remotes\") remotes::install(\"systemPipeR/systemPipeShiny\", dependencies=TRUE)  Similarly, remotes::install(\"systemPipeR/systemPipeShiny\") for the minimum develop version.\nLinux If you are on Linux, you may also need the following libraries before installing SPS. Different distributions may have different commands, but the following commands are examples for Ubuntu:\nsudo apt-get install libcurl4-openssl-dev sudo apt-get install libv8-dev sudo apt-get install libxm12-dev sudo apt-get install libssl-dev  On other Linux distributions, the install commands may be slightly different.\n","categories":"","description":"","excerpt":"Full if (!requireNamespace(\"BiocManager\", quietly=TRUE)) …","ref":"/sps/install/","tags":"","title":"Installation"},{"body":" SPS tabs In SPS, tab is the basic component of a functionality unit. For example, all the Modules in SPS are complex tabs with many small sub-tabs, the Canvas is another independent tab providing image editing features and an user custom tab is also a SPS tab.\nFrom the developer’s view, all SPS tabs are Shiny Modules{blk}.\nTo understand how SPS tabs work, we will demonstrate with a SPS project. For demo purpose, we are using the /tmp folder but one should use a regular location instead of the temp in a real case.\nsuppressPackageStartupMessages(library(systemPipeShiny)) spsInit(app_path = tempdir(), project_name = \"tab_demo\", overwrite = TRUE, change_wd = FALSE) ## [SPS-INFO] 2021-04-19 23:16:52 Start to create a new SPS project ## [SPS-INFO] 2021-04-19 23:16:52 Create project under /tmp/RtmpjAYUc0/tab_demo ## [SPS-INFO] 2021-04-19 23:16:52 Now copy files ## [SPS-INFO] 2021-04-19 23:16:52 Create SPS database ## [SPS-INFO] 2021-04-19 23:16:52 Created SPS database method container ## [SPS-INFO] 2021-04-19 23:16:52 Creating SPS db... ## [SPS-DANGER] 2021-04-19 23:16:52 Done, Db created at '/tmp/RtmpjAYUc0/tab_demo/config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-19 23:16:52 Key md5 ecdf3312953a6bcdaea49e6d85954b80 ## [SPS-INFO] 2021-04-19 23:16:52 SPS project setup done! ## save project path (sps_dir \u003c- file.path(tempdir(), \"tab_demo\")) ## [1] \"/tmp/RtmpjAYUc0/tab_demo\"  To reproduce code locally, run the following chunk instead.\nlibrary(systemPipeShiny) spsInit() sps_dir \u003c- normalizePath(\".\")  Tab registration In SPS, all tabs are controlled by the config/tabs.csv file. To see what kind of tabs you have with current project. use the spsTabInfo function. It returns a tibble of current tab information.\nspsTabInfo(app_path = sps_dir)  ## # A tibble: 11 x 8 ## tab_id display_label type type_sub image displayed tab_file_name plugin ## \u003cchr\u003e \u003cchr\u003e \u003cchr\u003e \u003cchr\u003e \u003cchr\u003e \u003cchr\u003e \u003cchr\u003e \u003cchr\u003e ## 1 core_ab… About this App core \"\" \"\" 1 No file for … \"core\" ## 2 core_ca… Canvas core \"\" \"\" 1 No file for … \"core\" ## 3 core_we… Home core \"\" \"\" 1 No file for … \"core\" ## 4 core_ri… Internal use on… core \"\" \"\" 1 No file for … \"core\" ## 5 core_top Top push bars core \"\" \"\" 1 No file for … \"core\" ## 6 module_… Module Main Page core \"\" \"\" 1 No file for … \"core\" ## 7 wf Workflow module modu… \"\" \"\" 1 No file for … \"core\" ## 8 vs_rnas… RNAseq module modu… \"\" \"\" 1 No file for … \"core\" ## 9 vs_esq Quick ggplot mo… modu… \"\" \"\" 1 No file for … \"core\" ## 10 vs_main custom tabs mai… core \"\" \"\" 1 No file for … \"core\" ## 11 vs_exam… My custom plott… vs \"plot\" \"\" 1 tab_vs_examp… \"\"   tab_id: A unique string ID display_label: for type is “core” or “module”, this is only some description, but for you own custom tabs, this value will be used as a display tab name on left sidebar on SPS UI. type: tab category, “core”, “module” and “vs” (visualization). type_sub: more specific category, current only “plot” (plotting) image: If this is an user custom tab, providing an image path will display the image in visualization main tab gallery. If it not provided, a warning will be given on app starts and an “No image” image will be used like the following:   displayed: Internal use only tab_file_name: where the tab file is relative to the R folder. plugin: Internal use only  Add a new custom tab SPS provides templates to help developers to create a small SPS tab that\n The main purpose is to generate some plots can be loaded into SPS framework easily can interact with other SPS tabs (components), like the Canvas tab.  When you initiate a SPS project, an example custom tab is created for you already. You can find the tab file inside the R folder of the project root. When you start the app for the first time, you can easily find it from the left sidebar, or click on the “Custom tabs” to go to custom tab main tab.\n Example custom tab  Simple template Under current SPS version, users are able to add custom tabs with the spsNewTab function. This function:\n creates the tab file. provides a nice template. Helps you to register tab information to tabs.csv  spsNewTab(tab_id = \"vs_new\", tab_displayname = \"New tab demo\", app_path = sps_dir) ## [SPS-INFO] 2021-04-19 23:16:52 Write to file /tmp/RtmpjAYUc0/tab_demo/R/tab_vs_new.R ## [SPS-INFO] 2021-04-19 23:16:52 Now register your new tab to config/tab.csv ## [SPS-SUCCESS] 2021-04-19 23:16:52 New tab created! ## [SPS] 2021-04-19 23:16:52 To load this new tab: `sps(tabs = c(\"vs_new\")`  If you are running the code locally, and are using Rstudio, the tab file will be opened automatically for you.\nIn your global.R, add the new tab to confirm you want to load it, and then restart you app. Scroll down to the bottom, you should see:\n# add \"vs_new\" in `tabs` sps_app \u003c- sps( tabs = c(\"vs_example\", \"vs_new\"), server_expr = { msg(\"Custom expression runs -- Hello World\", \"GREETING\", \"green\") } )   Load new custom tabs  By default, it uses the simple template, which contains the spsEzUI and spsEzServer functions. We have provided commented instructions on how to fill each argument.\nUI spsEzUI( desc = \"xxx\", tab_title = \"xxx\", plot_title = \"xxx\", plot_control = shiny::tagList( xxx ) )  The only augment new users need to take some time to learn is adding plot_control. Developers need to add some Shiny UI components to let users control how the plotting is done with interactive options. Basic use can be learned in 5 minutes on Shiny website{blk} and Shiny book{blk}\nServer spsEzServer( plot_code = { # data passed from data loading is a reactiveValues object, data stored in `mydata$data` plot_data \u003c- mydata$data # some validations, make sure users give you the right data format spsValidate({ stopifnot(inherits(plot_data, \"data.frame\")) # require a dataframe stopifnot(nrow(plot_data) \u003e 1) # has least one row if (!all(c(\"Sepal.Length\", \"Sepal.Width\") %in% colnames(plot_data)))# has two required columns stop(\"Require column 'Sepal.Length' and 'Sepal.Width'\") TRUE # give it a TRUE if all checks passed. }, verbose = FALSE # only show messages when fail ) # actual plot code ggplot2::ggplot(plot_data) + ggplot2::geom_point(ggplot2::aes(x = Sepal.Length, y = Sepal.Width)) + # grab user defined title from plot control by `input$+control_ID`, # no need to add `ns()` on server end. ggplot2::ggtitle(input$plot_title) }, other_server_code = {} )  For the server code, users only need to focus on the plotting code. The only very important thing developers need to remember is that the plotting data been passed to this function is stored in a reactiveValues{blk} object and it is called mydata$data. Usually we assign it to a new value so it can be used easily downstream, like plot_data \u003c- mydata$data.\nSome validation is recommended before reaching the plotting code. You would never know what kind of dataset users upload. It is always good to check if users' uploads meet the requirements. In SPS you can use the spsValidate function or use Shiny’s default validate{blk} function (spsValidate is discussed in developer tools).\nFull template For some developers who already has experience with Shiny, and would like to make more complex customization, using the full template enables you to change every detail on the new tab. Simply add the template = \"full\" argument.\nspsNewTab( tab_id = \"vs_new_full\", tab_displayname = \"New tab demo\", template = \"full\", app_path = sps_dir) ## [SPS-INFO] 2021-04-19 23:16:52 Write to file /tmp/RtmpjAYUc0/tab_demo/R/tab_vs_new_full.R ## [SPS-INFO] 2021-04-19 23:16:52 Now register your new tab to config/tab.csv ## [SPS-SUCCESS] 2021-04-19 23:16:52 New tab created! ## [SPS] 2021-04-19 23:16:52 To load this new tab: `sps(tabs = c(\"vs_new_full\")`  You can see the full template is a lot longer than the simple template:\nsimple_len \u003c- R.utils::countLines(file.path(sps_dir, \"R\", \"tab_vs_new.R\")) full_len \u003c- R.utils::countLines(file.path(sps_dir, \"R\", \"tab_vs_new_full.R\")) spsinfo(glue::glue(\"Simple template has {simple_len} lines\"), TRUE)  ## [SPS-INFO] 2021-04-19 23:16:52 Simple template has 66 lines  spsinfo(glue::glue(\"Full template has {full_len} lines\"), TRUE)  ## [SPS-INFO] 2021-04-19 23:16:52 Full template has 281 lines  Tab registeration In your global.R, scroll down to the bottom, you should see:\nsps_app \u003c- sps( tabs = c(\"vs_example\"), server_expr = { msg(\"Custom expression runs -- Hello World\", \"GREETING\", \"green\") } )  This is the SPS main function. You can load/unload custom tabs by providing tab IDs in tabs argument, like c(\"tab1\", \"tab2). Open config/tabs.csv or use spsTabInfo() to see what tabs IDs can be load and other tab information. Essential framework tabs (core) and built-in modules (modules) are loaded automatically. However, you can choose to unload core and modules tabs, and overwrite core tabs by changing some SPS options.\nTab naming Once a tab ID is provided in the sps function tabs argument, when the function runs, it is looking for tab information inside this tabs.csv, like the display name and tab image. Then it will search for the UI and server function in the enviornment. It is expecting a UI function named tab_id + UI -\u003e tab_idUI and the server tab_id + Server -\u003e tab_idServer. If you did not use the spsNewTab function to generate the new tab, make sure you name your tab UI and server in this pattern.\n","categories":"","description":"","excerpt":" SPS tabs In SPS, tab is the basic component of a functionality unit. …","ref":"/sps/adv_features/tabs/","tags":"","title":"Manage tabs"},{"body":" As a framework, default SPS tabs like the welcome tab, the module main tab, the visualization main tab, the Canvas tab and the About tab can be user-defined.\n Default UI and tabs you can overwrite  Overwrite SPS default tabs To start create a SPS project:\nsuppressPackageStartupMessages(library(systemPipeShiny))  app_path \u003c- \".\" spsInit(app_path = app_path, overwrite = TRUE, open_files = FALSE)  ## [SPS-DANGER] 2021-04-22 15:45:11 Done, Db created at '/tmp/Rtmp2YLDmY/overwrite/config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-22 15:45:11 Key md5 6886f6bf17229ffb646102abad92fa31 ## [SPS-INFO] 2021-04-22 15:45:11 SPS project setup done!  use the function spsCoreTabReplace to create a template of which tab you would like to overwrite. Here we use the “Welcome” tab as example to demo how to write your own welcome tab.\nspsCoreTabReplace(replace_tab = \"welcome\")  ## [SUCCESS] 2021-04-22 15:45:11 File /tmp/Rtmp2YLDmY/overwrite/R/tab_core_welcome.R created  If you are using Rstudio the template will be opened for you like following:\n########################## Overwrite the welcome tab ########################### ## UI core_welcomeUI \u003c- function(id){ ns \u003c- NS(id) tagList( # add your UI code below ) } ## server core_welcomeServer \u003c- function(id, shared){ module \u003c- function(input, output, session, shared){ ns \u003c- session$ns # add your server code below } moduleServer(id, module) }  Then you can write your own UI and server for this tab, for example, we can add some UI to the tab:\ncore_welcomeUI \u003c- function(id){ ns \u003c- NS(id) tagList( # add your UI code below h2(\"This is my welcome page\"), h4(\"some UI\"), sliderInput(ns(\"num\"), \"Select a number\", 0, 10, 5) ) }  When you restart the app, you should see your custom Welcome tab:\n Custom Welcome tab UI  ","categories":"","description":"","excerpt":" As a framework, default SPS tabs like the welcome tab, the module …","ref":"/sps/adv_features/overwritetabs/","tags":"","title":"Overwrite tabs"},{"body":"RNA-Seq Module This is a module which takes a raw count table to do normalization, Differential gene expression (DEG) analysis, and finally helps users to generate different plots to visualize the results.\nPrepare metadata and count table To start, we require two files, the metadata file (targets) and a raw count table (Fig. 5).\n  Figure 5 RNAseq\n This is the RNAseq module UI when you first click it. All sub-tabs are disbled at the beginning. Other tabs will enabled as you proceed with different options. First, we need a metadata file to tell SPS what samples and conditions to use. Here, we use the metadata file from SPR, which is also known as “targets” file. If you are not familiar with the targets file, we suggest to use the workflow module step 2 to practice creating and checking the format. You can also use the example to see how it looks like. The loaded targets table is display here. You can use the box below each column name to filter what samples to include/exclude. Only the “SampleName” and “Factor” columns are used, other columns are ignored. SampleName should be a unique character string without space for each row. Factor is the experiment design factors, or conditions, or treatments. If you want to DEG analysis, DEG comparison groups are defined in the targets file header. The header will be parsed into comparison groups which contain individual comparisons. If the parsed comparison is not what you want, edit the header lines and reupload. If everything is expected, confirm to use this table. You should see the progress timeline of step 1 becomes green if your targets and header pass the format checking. (Not on figure) Similarly, use example or upload a count table and confirm to use it.  Note: For the count table, the first column will be used as gene names. Other column names will be treated as sample names, and values in these columns are treated as raw counts. Make sure columns except the first one are numeric, and replace NA with 0.\nUpon successfully confirm targets and count table, you should see the “Normalize Data” subtab is enabled. You can click on the top navigation or click the pop-up for the next step.\nProcess raw count If this UI is displayed, that means your targets and count table are accepted by SPS (Fig 6). On this sub-tab, you can choose:\n Transform your count data with “raw”, “rlog” or “VST” and visualize the results in other sub-tabs. Do DEG analysis.  These two options are independent.\n  Figure 6 RNAseq Normalization\n At step 1 panel, choose how SPS can help you, count transformation or DEG analysis. The former will jump you to step 2, latter will jump to step 3. There are many options. If you are not clear, hover your mouse on the option, and some tips will show up. To start data transformation or DEG analysis. A gallery of different plot options will show up when the data process is done. When the data process is done, you can download results from the right side panel. Check all items you want and SPS will help you to zip it into one file to download. If at least one item is checked, downloading is enabled. Progress timeline will also change upon successful data process. Different visualization options will be enabled depending on the data process options.  Plot options SPS RNAseq module provides 6 different plot options to cluster transformed count table.\n  Figure 6 RNAseq plots\n Change plot options to customize your plots. Most plots are Plotly plots, which means you can interact with these plots, like hiding/show groups, zoom in/out, etc. All SPS plots are resizable. Drag the bottom-right corner icon to resize your plot. Click “To canvas” to take a screenshot of current plot and edit it in SPS Canvas tab. Or clicking the down-arrow button to directly save current plot to a png or jpg.  DEG report This is a special sub-tab designed to filter and visualize DEG results. This sub-tab can be accessed once the DEG is calculated on the “Normalize Data” sub-tab.\n  Figure 7 RNAseq DEG\n DEG summary plot. You can view what are the DEG results across different comparision groups. Switch to view a ggplot friendly table. Different from the table you could download from “Normalize Data” subtab, this DEG table is rearranged so you can easily make a ggplot from it. You can change the filter settings here, so DEGs will be re-filtered and you do not need to go back to “Normalize Data” subtab to recalculate DEG. DEG plotting options. Choose from a volcano plot, an upset plot (intersection), a MA plot or a heatmap.  Interact with other bioconductor packages. Locally If you are familiar with R and want to continue other analysis after these, simple stop SPS:\n After count transformation, there is a spsRNA_trans object stored in your R environment. raw method gives you a normalized count table. Other two methods give you a DESeq2 class object. You can use it for other analysis. After DEG analysis, SPS stores a global object called spsDEG. It is a summerizedExperiment object which has all individual tables from all DEG comparisons. You can use it for other downstream analysis.  Remotely If you are using SPS from a remote server, you can choose to download results from “Normalize Data” sub-tab. Choose results in tabular format or summerizedExperiment format which is saved in a .rds file.\n","categories":"","description":"","excerpt":"RNA-Seq Module This is a module which takes a raw count table to do …","ref":"/sps/modules/rnaseq/","tags":"","title":"RNAseq"},{"body":"How to create a new Workflow Template SPRthis package expand usethis package, providing automation to create systemPipeR workflows templates.\nInstallation To install SPRthis using from BiocManager the following code:\nif (!requireNamespace(\"BiocManager\", quietly = TRUE)) { install.packages(\"BiocManager\") BiocManager::install(\"dcassol/SPRthis\")  Quick start to using to SPRthis ## Load the package library(SPRthis) ## create Package sprthis(wfName=\"SPRtest\", analysis=\"SPRtest\", path=tempdir())  ## ✓ Setting active project to '/tmp/RtmpunjAF9'  ## ✓ Creating 'R/'  ## ✓ Writing 'DESCRIPTION'  ## Package: SPRtest ## Title: SPRtest ## Version: 0.9.0 ## Authors@R (parsed): ## * First Last \u003cfirst.last@example.com\u003e [aut, cre] (YOUR-ORCID-ID) ## Description: This package provides a pre-configured workflow and reporting ## template for SPRtest. ## License: Artistic-2.0 ## URL: https://github.com/systemPipeR/SPRtest ## Imports: ## systemPipeR (\u003e= 1.25.0) ## Suggests: ## BiocStyle, ## knitr, ## rmarkdown ## VignetteBuilder: ## knitr ## biocViews: Infrastructure, ... ## Encoding: UTF-8 ## LazyData: true ## Roxygen: list(markdown = TRUE) ## RoxygenNote: 7.1.1 ## SystemRequirements: SPRtest can be used to run external command-line ## software, but the corresponding tool needs to be installed on a ## system.  ## ✓ Writing 'NAMESPACE'  ## ✓ Setting active project to '\u003cno active project\u003e'  ## [1] \"/tmp/RtmpunjAF9\"   SPRtest/ ├── DESCRIPTION ├── NAMESPACE ├── README.md ├── SPRtest.Rproj ├── .gitignore ├── .Rbuildignore ├── .Rproj.user/ ├── R/ │ ├── functions.R ├── vignettes │ ├── bibtex.bib │ ├── SPRtest.Rmd └── inst ├── rmarkdown │ └── templates │ └── SPRtest │ ├── template.yml │ └── skeleton │ ├── batchtools.slurm.tmpl │ ├── .batchtools.conf.R │ ├── bibtex.bib │ ├── NEWS │ ├── SPRconfig.yml │ ├── skeleton.Rmd │ ├── targetsPE.txt │ ├── data/ │ ├── param/ │ └── results/  Help functions to create the package Create the webiste for the package with pkgdown Edit the _pkgdown.yml file and run:\npkgdown::build_site()  Documentation with roxygen2 roxygen2::roxygenise()  Testing the code with testthat To test the code, you can run\ndevtools::test()  Update R Markdown template on skeleton path \u003c- file.path(\"vignettes/SPRtest.Rmd\") skeleton_update(path)  Package available to genWorkenvir Function After creating the new repository on GitHub systemPipeR Organization, please follow:\n Rules:  The Workflow Template need to be available under systemPipeR Organization; The repository needs to be public; About setting:  Description: keywords in the description are required: “Workflow Template”; Topics: we expected “systempiper” and “release” or “development” words on Topics section;   Branch name: To make simple, please name the branch as “master”.    Check availability of workflow templates A collection of workflow templates are available, and it is possible to browse the current availability, as follows:\nsystemPipeRdata::availableWF(github = TRUE)  ## $systemPipeRdata ## [1] \"chipseq\" \"new\" \"riboseq\" \"rnaseq\" \"varseq\" ## ## $github ## workflow branches version ## 1 systemPipeR/SPchipseq master release ## 2 systemPipeR/SPriboseq master release ## 3 systemPipeR/SPrnaseq cluster, master, singleMachine release ## 4 systemPipeR/SPvarseq master release ## 5 systemPipeR/SPclipseq master devel ## 6 systemPipeR/SPdenovo master devel ## 7 systemPipeR/SPmetatrans master devel ## 8 systemPipeR/SPmethylseq master devel ## 9 systemPipeR/SPmirnaseq master devel ## html description ## 1 https://github.com/systemPipeR/SPchipseq Workflow Template ## 2 https://github.com/systemPipeR/SPriboseq Workflow Template ## 3 https://github.com/systemPipeR/SPrnaseq Workflow Template ## 4 https://github.com/systemPipeR/SPvarseq Workflow Template ## 5 https://github.com/systemPipeR/SPclipseq Workflow Template ## 6 https://github.com/systemPipeR/SPdenovo Workflow Template ## 7 https://github.com/systemPipeR/SPmetatrans Workflow Template ## 8 https://github.com/systemPipeR/SPmethylseq Workflow Template ## 9 https://github.com/systemPipeR/SPmirnaseq Workflow Template  This function returns the list of workflow templates available within the package and systemPipeR Project Organization on GitHub. Each one listed template can be created as described above.\n","categories":"","description":"","excerpt":"How to create a new Workflow Template SPRthis package expand usethis …","ref":"/sp/sprdata/newwf/","tags":"","title":"Rules to create a new Workflow Template"},{"body":"{spsComps} has some useful functions for exception catch, expression validation, and more. Even though we say they are Shiny server functions, but in fact most of them can be run without a Shiny server. We have designed the functions to detect whether there is a Shiny server, if not, they will work only in R console as well.\nload package library(spsComps)  ## Loading required package: shiny  ## Loading required package: spsUtil  library(magrittr)  Server components shinyCatch basic The shinyCatch function is useful to capture exception. What we mean exception can be message, warning or error. For example\nshinyCatch({ message(\"This is a message\") warning(\"This is a warning\") stop(\"This is an error\") })  ## [SPS-INFO] 2021-04-14 17:31:28 This is a message ## ## [SPS-WARNING] 2021-04-14 17:31:28 This is a warning ## [SPS-ERROR] 2021-04-14 17:31:28 This is an error  ## NULL  You can see all 3 levels are captured inside the [SPS-XX] log on your console. If you run this in your Shiny app, a pop-up message with the corresponding log level message will be displayed in in app, like following:\nSo the message on both UI and console is called dual-end logging in SPS.\nServer only Of course, if you do not want users to see the message, you can hide it by shiny = FALSE, but the message will be still logged on R console. Run the following on your own computer and watch the difference.\nlibrary(shiny) ui \u003c- fluidPage( spsDepend(\"toastr\") ) server \u003c- function(input, output, session) { shinyCatch({ stop(\"This is an error\") }, shiny = FALSE) } shinyApp(ui, server)  get return shinyCatch is able to return you values if your expression has any. Imagine we have a function addNum that gives message, warning or error depeend on the input.\naddNum \u003c- function(num){ if (num \u003e 0) {message(num)} else if (num == 0) {warning(\"Num is 0\")} else {stop(\"less than 0\")} return(num + num) } value_a \u003c- shinyCatch({ addNum(1) })  ## [SPS-INFO] 2021-04-14 17:31:28 1  value_a  ## [1] 2  value_b \u003c- shinyCatch({ addNum(0) })  ## [SPS-WARNING] 2021-04-14 17:31:28 Num is 0  value_b  ## [1] 0  value_c \u003c- shinyCatch({ addNum(-1) })  ## [SPS-ERROR] 2021-04-14 17:31:28 less than 0  value_c  ## NULL  You can see at message and warning level, the expected value returned, and at error level, the return is NULL. So the following code value_c still runs and is not blocked by the error occurred in shinyCatch.\nBlocking level More often, if there is an error, we do want take the log in R console, inform the Shinyapp user and then stop the following code. In this case, we need to specify the blocking_level. So, default is \"none\", do not block and return NULL if there is an error, and you can choose \"error\", \"warning\" or \"message\".\n error: block downstream if the first error detected in shinyCatch warning: block downstream if the first error or warning detected in shinyCatch message: block downstream if the first error, warning or message detected in shinyCatch  You can see the stringency becomes tighter: message \u003e warning \u003e error\nBlocking code will generate error, in order to have the Rmd rendered, we wrap the expression in try \ntry({ shinyCatch({ stop(\"error level is the most commonly used level\") }, blocking_level = \"error\") print(\"This will not be evaluated\") })  ## [SPS-ERROR] 2021-04-14 17:31:28 error level is the most commonly used level ## Error :  try({ shinyCatch({ message(\"error level is the most commonly used level\") }, blocking_level = \"message\") print(\"This will not be evaluated either\") })  ## [SPS-INFO] 2021-04-14 17:31:28 error level is the most commonly used level ## ## Error :  You can see the following print in both cases are not got evaluated.\nBlock reative The most useful case for shinyCatch is to use it in the Shiny reactive context. Most errors in shiny::reactive, shiny::observer, shiny::observeEvent, or shiny::renderXXX series function will crash the app. With shinyCatch, it will not. It “dual-logs” the error and stop downstream code.\nThe following example use shiny::reactiveConsole() to mock a Shiny server session\nshiny::reactiveConsole(TRUE) y \u003c- observe({ stop(\"an error from a function\") print(\"some other process\") })  ## Warning: Error in \u003cobserver\u003e: The value of x is ## 38: stop ## 37: \u003cobserver\u003e [#2] ## 35: contextFunc ## 34: env$runWith ## 27: ctx$run ## 26: run ## 7: flushCallback ## 6: FUN ## 5: lapply ## 4: ctx$executeFlushCallbacks ## 3: .getReactiveEnvironment()$flush ## 2: flushReact ## 1: \u003cAnonymous\u003e  It crashes the app. However, if you use shinyCatch\nshiny::reactiveConsole(TRUE) y \u003c- observe({ shinyCatch({ stop(\"an error from a function\") }, blocking_level = \"error\") print(\"some other process\") })  ## [SPS-ERROR] 2021-03-02 22:13:05 an error from a function  It only logs the error and prevent the downstream print to run. Now try following real Shiny apps and watch the difference:\n# with shinyCatch library(shiny) server \u003c- function(input, output, session) { observe({ shinyCatch({ stop(\"an error from a function\") }, blocking_level = \"error\") print(\"some other process\") }) } shinyApp(fluidPage(spsDepend(\"toastr\")), server)  # without shinyCatch library(shiny) server \u003c- function(input, output, session) { observe({ stop(\"an error from a function\") print(\"some other process\") }) } shinyApp(fluidPage(spsDepend(\"toastr\")), server)  spsValidate In data analysis, it is important we do some validations before the downstream process, like make a plot. It is epecially the case in Shiny apps. We cannot predict what the user inputs will be, like what kind of data they will use. Similar to shinyCatch, spsValidate is able to catch exceptions but more useful to handle validations. In addtion to shinyCatch functionalities, it will give users a success message if the expression goes through and return TRUE (shinyCatch returns the final expression value).\nshiny::reactiveConsole(TRUE) x \u003c- reactiveVal(10) y \u003c- observe({ spsValidate({ # have multiple validations in one expression if (x() == 1) stop(\"cannot be 1\") if (x() == 0) stop(\"cannot be 0\") if (x() \u003c 0) stop(\"less than 0\") }) message(\"The value of x is \", x()) }) x(0) x(-10)  ## The value of x is 10 ## [ ERROR] 2021-03-02 22:36:16 cannot be 0 ## [ ERROR] 2021-03-02 22:36:16 less than 0  Try this real Shiny app:\nlibrary(shiny) ui \u003c- fluidPage( spsDepend(\"toastr\"), shiny::sliderInput( \"num\", \"change number\", min = -1, max = 2, value = 2, step = 1 ) ) server \u003c- function(input, output, session) { x \u003c- reactive(as.numeric(input$num)) y \u003c- observe({ spsValidate(vd_name = \"check numbers\", verbose = TRUE, { # have multiple validations in one expression if (x() == 1) stop(\"cannot be 1\") if (x() == 0) stop(\"cannot be 0\") if (x() \u003c 0) stop(\"less than 0\") }) message(\"The value of x is \", x()) }) } shinyApp(ui, server)  You should see the success message like this:\nshinyCheckPkg Sometimes we want the app behave differently if users have certain packages installed. For example, if some packages are installed, we open up additional tabs on UI to allow more features. This can be done with the shinyCheckPkg function. This function has to run inside Shiny server, an alternative version to use without Shiny is from the spsUtil package, spsUtil::checkNameSpace\nUse it in Shiny server, specify the packages you want to check by different sources, like CRAN, Bioconductor, or github.\nshinyCheckPkg(session, cran_pkg = c(\"pkg1\", \"pkg2\"), bioc_pkg = \"bioxxx\", github = \"user1/pkg1\")  It will return TRUE if all packages are installed, otherwise FALSE\nNow try this real example. We check if the ggplot99 package is installed, if yes we make a plot. It also combines the spsValidate function. You can have a better idea how these functions work.\nlibrary(shiny) ui \u003c- fluidPage( tags$label('Check if package \"pkg1\", \"pkg2\", \"bioxxx\", github package \"user1/pkg1\" are installed'), br(), actionButton(\"check_random_pkg\", \"check random_pkg\"), br(), spsHr(), tags$label('We can combine `spsValidate` to block server code to prevent crash if some packages are not installed.'), br(), tags$label('If \"shiny\" is installed, make a plot.'), br(), actionButton(\"check_shiny\", \"check shiny\"), br(), tags$label('If \"ggplot99\" is installed, make a plot.'), br(), actionButton(\"check_gg99\", \"check ggplot99\"), br(), plotOutput(\"plot_pkg\") ) server \u003c- function(input, output, session) { observeEvent(input$check_random_pkg, { shinyCheckPkg(session, cran_pkg = c(\"pkg1\", \"pkg2\"), bioc_pkg = \"bioxxx\", github = \"user1/pkg1\") }) observeEvent(input$check_shiny, { spsValidate(verbose = FALSE, { if(!shinyCheckPkg(session, cran_pkg = c(\"shiny\"))) stop(\"Install packages\") }) output$plot_pkg \u003c- renderPlot(plot(1)) }) observeEvent(input$check_gg99, { spsValidate({ if(!shinyCheckPkg(session, cran_pkg = c(\"ggplot99\"))) stop(\"Install packages\") }) output$plot_pkg \u003c- renderPlot(plot(99)) }) } shinyApp(ui, server)  You should see something like this if there is any missing package:\n","categories":"","description":"","excerpt":"{spsComps} has some useful functions for exception catch, expression …","ref":"/sps/dev/spscomps/server/","tags":"","title":"Server functions"},{"body":"","categories":"","description":"","excerpt":"","ref":"/sp/spt/","tags":"","title":"systemPipeTools"},{"body":"The {drawer} package offers an interactive image editing tool that can be added as part of the HTML in Shiny, R markdown or any type of HTML document. Often times, plots, photos are embedded in the web application/file. {drawer} can take screenshots of these image-like elements, or any part of the HTML document and send to an image editing space called “canvas” to allow users immediately edit the screenshot(s) within the same document. Users can quickly combine, compare different screenshots, upload their own images and maybe make a scientific figure.\nFeatures  {drawer} is built with 99% javascript + HTML + CSS, there is no need to have a Shiny server or any other types of server in the back-end. That’s why you can use it in any HTML document. All you need is a modern web browser, like Chrome or Firefox (IE will not work). Shiny and R markdown compatible. Screenshot any element in the page and edit in canvas or download it png or jpg Drag and upload your own images.  Drawer UI screenshot Installation Read the developer tools main page, not repeating here.\nFunctions reference manual Please read the reference manuals for details of every function.\nSome video demos Basic looking and options  Capture Use the capture buttons to capture plots, images and other elements in the same document.\n Add text  Upload your own images You can upload one or multiple your own images at once.\n Browser support {drawer} only works on recent browsers versions, like Chrome, latest Edge, Firefox. IE is not supported (IE is not my friend). Also, some browser privacy extensions will block javascript and HTML5 canvas fingerprint. This will cause the screenshot to be blank. {drawer} does not collect any user information from you.\n","categories":"","description":"","excerpt":"The {drawer} package offers an interactive image editing tool that can …","ref":"/sps/dev/drawer/","tags":"","title":"drawer"},{"body":"\n{drawer} is compatible with R markdown. Instructions and details are all included in these Rmd demos:\n   Demo Source code     With instructions link   Pure drawer link    ","categories":"","description":"","excerpt":"\n{drawer} is compatible with R markdown. Instructions and details are …","ref":"/sps/dev/drawer/rmd/","tags":"","title":"drawer in Rmarkdown"},{"body":"document.addEventListener(\"DOMContentLoaded\", function() { document.querySelector(\"h1\").className = \"title\"; });  document.addEventListener(\"DOMContentLoaded\", function() { var links = document.links; for (var i = 0, linksLength = links.length; i How to run a Workflow This tutorial introduces the basic ideas and tools needed to build a specific workflow from preconfigured templates.\nLoad sample data and workflow templates library(systemPipeRdata) genWorkenvir(workflow = \"rnaseq\") setwd(\"rnaseq\")  Setup and Requirements To go through this tutorial, you need the following software installed:\n R/\u003e=3.6.2 systemPipeR R package (version 1.22) Hisat2/2.1.0  If you desire to build your pipeline with any different software, make sure to have the respective software installed and configured in your PATH. To make sure if the configuration is right, you always can test as follow:\ntryCL(command = \"hisat2\") ## 'All set up, proceed!'  Project Initialization The Project management structure is essential, especially for reproducibility and efficiency in the analysis. Here we show how to construct an instance of this S4 object class by the initWF function. The object of class SYSarsgsList storing all the configuration information for the project and allows management and control at a high level.\nscript \u003c- \"systemPipeRNAseq.Rmd\" targetspath \u003c- \"targets.txt\" sysargslist \u003c- initWF(script = script, targets = targetspath)  Project Initialization in a Temporary Directory library(systemPipeRdata) script \u003c- system.file(\"extdata/workflows/rnaseq\", \"systemPipeRNAseq.Rmd\", package = \"systemPipeRdata\") targets \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") dir_path \u003c- tempdir() SYSconfig \u003c- initProject(projPath = dir_path, targets = targets, script = script, overwrite = TRUE) sysargslist_temp \u003c- initWF(sysconfig = \"SYSconfig.yml\")  Configuration and run of the project sysargslist \u003c- configWF(x = sysargslist, input_steps = \"1:3\") sysargslist \u003c- runWF(sysargslist = sysargslist, steps = \"ALL\") sysargslist \u003c- runWF(sysargslist = sysargslist, steps = \"1:2\")  How to Use Pipes with systemPipeR At first encounter, you may wonder whether an operator such as %\u003e% can really be all that beneficial; but as you may notice, it semantically changes your code in a way that makes it more intuitive to both read and write.\nConsider the following example, in which the steps are the initialization, configuration and running the entire workflow.\nlibrary(systemPipeR) sysargslist \u003c- initWF(script = \"systemPipeRNAseq.Rmd\", overwrite = T) %\u003e% configWF(input_steps = \"1:3\") %\u003e% runWF(steps = \"1:2\")  How to run the workflow on a cluster This section of the tutorial provides an introduction to the usage of the systemPipeR features on a cluster.\nNow open the R markdown script *.Rmdin your R IDE (_e.g._vim-r or RStudio) and run the workflow as outlined below. If you work under Vim-R-Tmux, the following command sequence will connect the user in an interactive session with a node on the cluster. The code of the Rmd script can then be sent from Vim on the login (head) node to an open R session running on the corresponding computer node. This is important since Tmux sessions should not be run on the computer nodes.\nq(\"no\") # closes R session on head node  srun --x11 --partition=short --mem=2gb --cpus-per-task 4 --ntasks 1 --time 2:00:00 --pty bash -l module load R/3.4.2 R  Now check whether your R session is running on a computer node of the cluster and not on a head node.\nsystem(\"hostname\") # should return name of a compute node starting with i or c getwd() # checks current working directory of R session dir() # returns content of current working directory  Parallelization on clusters Alternatively, the computation can be greatly accelerated by processing many files in parallel using several compute nodes of a cluster, where a scheduling/queuing system is used for load balancing. For this the clusterRun function submits the computing requests to the scheduler using the run specifications defined by runCommandline.\nTo avoid over-subscription of CPU cores on the compute nodes, the value from yamlinput(args)['thread'] is passed on to the submission command, here ncpus in the resources list object. The number of independent parallel cluster processes is defined under the Njobs argument. The following example will run 18 processes in parallel using for each 4 CPU cores. If the resources available on a cluster allow running all 18 processes at the same time then the shown sample submission will utilize in total 72 CPU cores. Note, clusterRun can be used with most queueing systems as it is based on utilities from the batchtools package which supports the use of template files (*.tmpl) for defining the run parameters of different schedulers. To run the following code, one needs to have both a conf file (see .batchtools.conf.R samples here) and a template file (see *.tmpl samples here) for the queueing available on a system. The following example uses the sample conf and template files for the Slurm scheduler provided by this package.\nlibrary(batchtools) resources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) reg \u003c- clusterRun(args, FUN = runCommandline, more.args = list(args = args, make_bam = TRUE, dir = FALSE), conffile = \".batchtools.conf.R\", template = \"batchtools.slurm.tmpl\", Njobs = 18, runid = \"01\", resourceList = resources) getStatus(reg = reg) waitForJobs(reg = reg)  References ","categories":"","description":"","excerpt":"document.addEventListener(\"DOMContentLoaded\", function() { …","ref":"/sp/spr/spr_run/","tags":"","title":"How to run a Workflow"},{"body":"Quick {ggplot} module This module enables you to quickly upload datasets and make a {ggplot{blk}} in a second by using some functionalities from {Esquisse{blk}}.\nUpload data  Upload data   The first thing you come to this module is to upload a tabular data file. You can choose to use the example or upload your own. The example is just the iris data.  1.1. If you choose to upload, there will be a upload button where you need to choose your own file.   By default, it assumes you upload a “.csv” file with “#” as comments. If not you can choose the file delimiter and comment character. You can view your uploaded data and use the boxes below each column name to perform some filters, but you are not allowed to edit the data. If everything looks good, you can submit to proceed to the plot making panel.  Make a plot  Make a plot  Figure 8 Quick ggplot\n Provide a tabular data table by uploading or use example. Drag variables from into different ggplot aesthetic boxes to make a ggplot. Change to different plot types. Customize other different plotting options.  For a more specific guide, read Esquisse official guide{blk}.\n","categories":"","description":"","excerpt":"Quick {ggplot} module This module enables you to quickly upload …","ref":"/sps/modules/ggplot/","tags":"","title":"Quick ggplot"},{"body":"SPS example usage To start with SPS after installation:\nLoad package Load the systemPipeShiny package in your R session.\nlibrary(systemPipeShiny)  ## Loading required package: shiny  ## Loading required package: spsUtil  ## Loading required package: spsComps  ## Loading required package: drawer  Initialize SPS project Before launching the SPS application, a project environment needs to be created with the following command.\nspsInit()  For this demo, the project directory structure is written to a temporary directory on a user’s system. For a real project, it should be written to a defined and user controlled location on a system rather than a temporary directory.\nsps_tmp_dir \u003c- tempdir() spsInit(app_path = sps_tmp_dir, change_wd = FALSE, project_name = \"SPSProject\")  ## [SPS-INFO] 2021-04-12 11:31:39 Start to create a new SPS project  ## [SPS-INFO] 2021-04-12 11:31:39 Create project under /tmp/RtmpgROsL7/SPSProject  ## [SPS-INFO] 2021-04-12 11:31:39 Now copy files  ## [SPS-INFO] 2021-04-12 11:31:39 Create SPS database  ## [SPS-INFO] 2021-04-12 11:31:39 Created SPS database method container  ## [SPS-INFO] 2021-04-12 11:31:39 Creating SPS db...  ## [SPS-DANGER] 2021-04-12 11:31:39 Done, Db created at '/tmp/RtmpgROsL7/SPSProject/config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-12 11:31:39 Key md5 ae88518aa6cce9a5af24d37c4c3c1b16 ## [SPS-INFO] 2021-04-12 11:31:39 SPS project setup done!  sps_dir \u003c- file.path(sps_tmp_dir, \"SPSProject\")  SPS project structure The file and directory structure of an SPS project is organized as follows.\n SPS_xx/ ├── server.R | ├── global.R | Most important server, UI and global files, unless special needs, `global.R` is the only file you need to edit manually ├── ui.R | ├── deploy.R | Deploy helper file ├── config | Important app config files. Do not edit them if you don't know │ ├── sps.db | SPS database │ ├── sps_options.yaml | SPS default option list │ └── tabs.csv | SPS tab information ├── data | App example data files │ ├── xx.csv ├── R | All SPS additional tab files and helper R function files │ ├── tab_xx.R ├── README.md ├── results | Not in use for this current version, you can store some data been generated from the app │ └── README.md └── www | Internet resources ├── about | About tab information │ └── xx.md ├── css | CSS files │ └── sps.css ├── img | App image resources │ └── xx.png ├── js | Javascripts │ └── xx.js ├── loading_themes | Loading screen files │ └── xx.html └── plot_list | Image files for plot gallery └── plot_xx.jpg  Launch SPS By default, the working directory will be set inside the project folder automatically. To launch the SPS Shiny application, one only needs to execute the following command.\nshiny::runApp()  After the SPS app has been launched, clicking the “Continue to app” button on the welcome screen will open the main dashboard (Fig.2).\n Figure 2: Snapshot of SPS' UI.\n Welcome screen. Module tabs. User defined custom tabs. The Canvas tab. All SPS tabs has this description on top. It is highly recommend to click here to expand and read the full the description for the first time.  Alternatively, when using RStudio one can click the Run App button in the top right corner.\nIn addition, in Rstudio the global.R file will be automatically opened when the SPS project is created. Custom changes can be made inside this file before the app launches. Later we will discuss how to change and create new custom tabs and change other settings.\n","categories":"","description":"","excerpt":"SPS example usage To start with SPS after installation:\nLoad package …","ref":"/sps/quick_start/","tags":"","title":"Quick start"},{"body":" SPS framework come with a plenty of useful general R utility functions, like pretty logging, package namespace checking, URL checking, and more.\nSince SPS 1.1, these functions are separated into a supporting package called spsUtil (systemPipeShiny Utility). You can install it from CRAN.\nInstallation Read the developer tools main page, not repeating here.\nFunctions reference manual In documents, we only highlight some important functions. Please read the reference manuals for details of every function.\nFunction highlights library(spsUtil)  logging with msg basic Often times in an R function, we want to use some text to inform users the status and message. We can use functions like message, warning, stop to generate different levels of information.\n{spsUtil} provides some more informative and prettier ways to generate these kind of messages.\nmsg(\"my message\")  ## [INFO] 2021-04-12 11:49:35 my message  You can see it starts with a level information, then a time stamp, and follows the actual message. By default, it uses the INFO level, and you can change to whatever level you want. However, there are 3 keywords that have special meaning.\nLevels  INFO: equals message method in native R WARNING: generates warnings the same as warning function ERROR: generates error the same as stop function and will prevent downstream code get evaluated.  If the level is other than these 3, there is no special meaning in R, just cat the message out.\nmsg(\"I am info\", level = \"INFO\")  ## [INFO] 2021-04-12 11:49:35 I am info  msg(\"I am warning\", level = \"warning\") # not case sensitive  ## Warning: [WARNING] 2021-04-12 11:49:35 I am warning  msg(\"I am error\", level = \"ERROR\")  ## Error: [ERROR] 2021-04-12 11:49:35 I am error  msg(\"I am random level\", level = \"MY LEVEL\")  ## [MY LEVEL] 2021-04-12 11:49:35 I am random level  Prefix For the 3 key levels, you can specify the prefix in front of the level text to over write the default level text INFO, WARNING, or ERROR\nmsg(\"I am info\", level = \"INFO\", info_text = \"NEW-INFO\")  ## [NEW-INFO] 2021-04-12 11:49:35 I am info  msg(\"I am warning\", level = \"warning\", warning_text = \"MY-WARNING\")  ## Warning: [MY-WARNING] 2021-04-12 11:49:35 I am warning  msg(\"I am error\", level = \"ERROR\", error_text = \"STOP\")  ## Error: [STOP] 2021-04-12 11:49:35 I am error  Colors Colors are automatically enabled if it is supported. If you try all code above in your terminal or Rstudio, they all have colors. In Rmd, to enable the color, you need to add the following code chunk. You also need to install the fansi package.\n```{r echo=FALSE, results='asis'} options(crayon.enabled = TRUE) old_hooks \u003c- fansi::set_knit_hooks(knitr::knit_hooks, which = c(\"output\", \"message\", \"error\", \"warning\")) ```  PRE.fansi SPAN {padding-top: .25em; padding-bottom: .25em};  msg(\"I am info\", level = \"INFO\", info_text = \"NEW-INFO\")  ## [NEW-INFO] 2021-04-12 11:49:35 I am info  The 3 key levels has default colors:\n INFO: blue WARNING: orange ERROR: red  You can specify colors for your own levels\nmsg(\"I am warning\", level = \"warning\") ## not super orange in Rmd translation -_-=  ## Warning: [WARNING] 2021-04-12 11:49:35 I am warning  msg(\"I am error\", level = \"error\")  ## Error: [ERROR] 2021-04-12 11:49:35 I am error  msg(\"oh yeah\", level = \"SUCCESS\", .other_color = \"green\")  ## [SUCCESS] 2021-04-12 11:49:35 oh yeah  msg(\"oh no\", level = \"FAIL\", .other_color = \"purple\")  ## [FAIL] 2021-04-12 11:49:35 oh no  Wrapper You can use this logging function in your own projects by wrapping it inside a upper level function, like what we do for spsinfo, spswarn, spserror. They have SPS- prefix added, and have some SPS global settings appended.\nspsOption('use_crayon', TRUE) spsinfo(\"info\", verbose = TRUE) ## default `verbose` mute the message  ## [SPS-INFO] 2021-04-12 11:49:35 info  spswarn(\"warning\")  ## Warning: [SPS-WARNING] 2021-04-12 11:49:35 warning  spserror(\"stop\")  ## Error: [SPS-ERROR] 2021-04-12 11:49:35 stop  To create a simple one for project is very easy. Assume your project is named “My Project”. You can create logging as:\nmpInfo \u003c- function(text){ spsUtil::msg(text, info_text = \"MP-INFO\") } mpWarn \u003c- function(text){ spsUtil::msg(text, level = \"warning\", warning_text = \"MP-WARNING\") } mpErr \u003c- function(text){ spsUtil::msg(text, level = \"error\", error_text = \"MP-ERROR\") } mpInfo(\"info\")  ## [MP-INFO] 2021-04-12 11:49:35 info  mpWarn(\"warning\")  ## Warning: [MP-WARNING] 2021-04-12 11:49:35 warning  mpErr(\"error\")  ## Error: [MP-ERROR] 2021-04-12 11:49:35 error  mute message with quiet In R, you can easily mute message and warnings with suppressMessages(), and suppressWarnings(), but not so easy with print or cat methods. spsUtil::quiet enables you to mute all these methods or choose what to mute.\n{ # muted quiet(warning(123)) quiet(message(123)) quiet(print(123)) quiet(cat(123)) # not muted quiet(warning(123), warning = FALSE) quiet(message(123), message = FALSE) quiet(print(123), print_cat = FALSE) quiet(cat(123), print_cat = FALSE) } ## Warning in force(x): 123 ## 123 ## [1] 123 ## 123  check “empty” values with emptyIsFalse In R, values like NA, \"\", NULL, length(0) is not very meaningful in condition judgment and will give you errors. Yet, R does not have a native method to handle these “empty” values in if like other languages. They are meaningful in other ways, but in conditions, we may want to turn them to FALSE.\nif(\"\") TRUE else FALSE ## Error in if (\"\") TRUE else FALSE: argument is not interpretable as logical if(NULL) TRUE else FALSE ## Error in if (NULL) TRUE else FALSE: argument is of length zero if(character(0)) TRUE else FALSE ## Error in if (character(0)) TRUE else FALSE: argument is of length zero if(NA) TRUE else FALSE ## Error in if (NA) TRUE else FALSE: missing value where TRUE/FALSE needed  You can see they all give errors. In other languages (javascript in this example), these values are often treated as FALSE.\nif (NaN) true; else false //\u003e false if (undefined) true; else false //\u003e false if (\"\") true; else false //\u003e false if (null) true; else false //\u003e false  if (NaN) true; else false // false if (undefined) true; else false // false if (\"\") true; else false // false if (null) true; else false // false  This is how emptyIsFalse work. If the input is one of these values, return FALSE, else TRUE\nif(emptyIsFalse(\"\")) TRUE else FALSE ## [1] FALSE if(emptyIsFalse(NULL)) TRUE else FALSE ## [1] FALSE if(emptyIsFalse(character(0))) TRUE else FALSE ## [1] FALSE if(emptyIsFalse(NA)) TRUE else FALSE ## [1] FALSE  check missing packages checkNameSpace In our functions, sometimes we want to have the users to install certain packages to enable more functionalities, like the DESeq2::lfcShrink function. Or like in a Rmd source code, before other people can rerender the document, they must install certain packages. checkNameSpace checks all required packages and returns the missing names.\ncheckNameSpace(\"random_pkg\")  ## Warning: [WARNING] 2021-04-12 11:49:35 These packages are missing from ## CRAN: random_pkg  ## [1] \"random_pkg\"  You can add it to your function to or on the top of your Rmd document to inform your users the missing packages and where to install.\npkgs \u003c- list( CRAN = c(\"pkg1\", \"pkg2\"), Bioconductor = c(\"bio_pkg1\", \"bio_pkg2\") ) missing_pkg \u003c- checkNameSpace(pkgs[[1]], from = names(pkgs)[1])  ## Warning: [WARNING] 2021-04-12 11:49:35 These packages are missing from ## CRAN: pkg1,pkg2  missing_pkg \u003c- c(missing_pkg, checkNameSpace(pkgs[[2]], from = names(pkgs)[2]))  ## Warning: [WARNING] 2021-04-12 11:49:35 These packages are missing from ## Bioconductor: bio_pkg1,bio_pkg2  if(emptyIsFalse(missing_pkg)) stop(\"Install packages\")  ## Error in eval(expr, envir, enclos): Install packages  Or write your custom warning message:\n{ missing_pkg \u003c- mapply(function(pkg, from) { checkNameSpace(pkg, quietly = TRUE, from) }, pkg = pkgs, from = names(pkgs), SIMPLIFY = FALSE) cat( \"Use `install.packages(c('\", paste0(missing_pkg[['CRAN']], collapse = \"','\"), \"'))` to install CRAN packages\\n\", sep = \"\" ) cat( \"Use `BiocManager::install(c('\", paste0(missing_pkg[['Bioconductor']], collapse = \"','\"), \"'))` to install Bioconductor packages\\n\", sep = \"\" ) if(emptyIsFalse(unlist(missing_pkg))) stop(\"Install packages\") }  ## Use `install.packages(c('pkg1','pkg2'))` to install CRAN packages ## Use `BiocManager::install(c('bio_pkg1','bio_pkg2'))` to install Bioconductor packages  ## Error in eval(expr, envir, enclos): Install packages  check a URL is reachable with checkUrl Useful if you need make some big HTTP requests.\ncheckUrl(\"https://google.com\")  ## [1] TRUE  checkUrl(\"https://randomwebsite123.com\", timeout = 1)  ## Warning: [WARNING] 2021-04-12 11:49:37 Bad url https:// ## randomwebsite123.com ## Warning: [WARNING] 2021-04-12 11:49:37 Timeout was reached: [randomwebsite123.com] Connection timed out after 1001 milliseconds \n## [1] FALSE  ","categories":"","description":"","excerpt":" SPS framework come with a plenty of useful general R utility …","ref":"/sps/dev/spsutil/","tags":"","title":"spsUtil"},{"body":" Find here all the documentation!\n systemPipeRdata: Workflow templates and sample data systemPipeRdata is a helper package to generate with a single command workflow templates that are intended to be used by its parent package systemPipeR. The systemPipeR project provides a suite of R/Bioconductor packages for designing, building and running end-to-end analysis workflows on local machines, HPC clusters and cloud systems, while generating at the same time publication quality analysis reports.\nTo test workflows quickly or design new ones from existing templates, users can generate with a single command workflow instances fully populated with sample data and parameter files required for running a chosen workflow. Pre-configured directory structure of the workflow environment and the sample data used by systemPipeRdata are described here.\nInstallation To install the package, please use the BiocManager::install command:\nif (!requireNamespace(\"BiocManager\", quietly=TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"systemPipeRdata\")  To obtain the most recent updates immediately, one can install it directly from github as follow:\nif (!requireNamespace(\"BiocManager\", quietly=TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"tgirke/systemPipeRdata\", build_vignettes=TRUE, dependencies=TRUE)  Due to the large size of the sample data (~320 MB) provided by systemPipeRdata, its download/install may take some time.\nTo install the parent package systemPipeR itself, please use the BiocManager::install method as instructed here.\nUsage Detailed user manuals are available here:\n systemPipeRdata Vignette systemPipeR Overview Vignette   ","categories":"","description":"","excerpt":" Find here all the documentation!\n systemPipeRdata: Workflow templates …","ref":"/sp/sprdata/","tags":"","title":"systemPipeRdata"},{"body":" Load and unload tabs In SPS, all tabs including tabs for modules and other default tabs can be loaded and unloaded. This is controlled by SPS options\nUnder current version, these options are:\n   Option Description Default Other     module_wf load workflow module? TRUE FALSE   module_rnaseq load RNAseq module? TRUE FALSE   module_ggplot load quick ggplot module? TRUE FALSE   tab_welcome load welcome tab? TRUE FALSE   tab_vs_main load custom visualization main tab? TRUE FALSE   tab_canvas load Canvas tab? TRUE FALSE   tab_about load about tab? TRUE FALSE    Each of them controls whether to load or unload a tab. By default, all tabs are loaded, but you can unload them by turn them to FALSE.\nThe original UI look like this:\n Default UI  To unload some tabs, scroll to the option lines in global.R file:\noptions(sps = list( ... tab_welcome = TRUE, tab_vs_main = TRUE, tab_canvas = FALSE, tab_about = FALSE, module_wf = FALSE, module_rnaseq = FALSE, module_ggplot = TRUE, ... ))  We unload the “Canvas tab”, “workflow module tab”, “RNASeq module tab” and “About tab”. When you restart the app, you should see some tabs are gone:\nThe original UI look like this:\n Unload some tabs  Exception for module main page You may have noticed, there is no option to unload the module main tab, which is named “Modules” on the left sidebar. This is because this tab is controlled by its sub-tabs, the module tab options. To unload this tab, all the module tabs have to be unloaded the same time like following. If any module is loaded, this module main tab cannot be unloaded.\noptions(sps = list( ... module_wf = FALSE, module_rnaseq = FALSE, module_ggplot = FALSE, ... ))   No module loaded  ","categories":"","description":"","excerpt":" Load and unload tabs In SPS, all tabs including tabs for modules and …","ref":"/sps/adv_features/displaytabs/","tags":"","title":"Toggle tabs"},{"body":" In this section, we will discuss the pre-defined modules in SPS\n ","categories":"","description":"","excerpt":" In this section, we will discuss the pre-defined modules in SPS\n ","ref":"/sps/modules/","tags":"","title":"Modules"},{"body":" SPS notification system In SPS, there is a notification dropdown where developers can broadcast new messages to users. The dropdown is located on the top-right corner.\n Notification Dropdown  \nWhen a notification item is clicked, details of the notification will be displayed in a modal.\n Notification Detail Moadal  Official notification If you only use the original SPS, we will send out new notifications every time we update the package or other important things that we want to inform you. You should see the icon of the dropdown becomes.  + the message number. If there is no message or you have clicked the dropdown, it will become  + 0.\nCustom notification If you do not want to receive the official notification or want to write your own note to your users, first let us understand how it works.\nMechanism Every time when you run the sps() main function, it will look for a remote URL that stores the notification information in yaml{blk} format. If this file can be successfully parsed, you will see the notification dropdown menu on SPS UI, otherwise no dropdown displayed.\nTo define your own notification URL, you need to change the option note_url in the global.R file. Read more about changing SPS options. The default value is a file on Github, which also can be used as your template to write custom notification messages:\nhttps://raw.githubusercontent.com/systemPipeR/systemPipeShiny/master/inst/remote_resource/notifications.yaml{blk}\nNotification template If you download the link above, you should see something like this:\n############ Create remote messages to notify users in the app ################# ## When app starts, it will first try to load this file from online. ## You should place this file somewhere publically reachable online, like Github. ## This file should not be included in your app deployment. ## Add the url of this file to the SPS option `note_url` in \"global.R\" file # type: one of 'package' or 'general', required # expire: note will be displayed before the date, required, YYYY-MM-DD format # title: string, required # icon: any font-awesome icon name, default is the \"info-circle\" # status: one of primary, success, info, warning, danger, default is \"primary\" # pkg_name: string, required if type == 'package', such as \"systemPipeShiny\" # version: string, required if type == 'package', such as \"1.0.0\" # message: string, optional, the text body of the notification. Be careful with indentations. - note: type: general pkg_name: version: title: Notification broadcasting expire: 2099-01-01 icon: status: message: | ## SPS notifications What you are looking at is the SPS notification broadcasting system. It display messages to your users by reading a remote `yaml` file stored online. SPS will fetch the content of this file and translate it to different notes you can see here. So you do not need to re-deploy the app every time there is a new notification. 1. You can customize your own notifications by using [this file as template](https://raw.githubusercontent.com/systemPipeR/systemPipeShiny/master/inst/remote_resource/notifications.yaml). 2. After the modification, place this file in public accessible location, like Github, do not inlcude this file in app deployment. 3. During app deployment, indicate the URL of this file in `global.R` file, `note_url:` option.  Template details Most entries are easy-to-understand. Here are some key points.\nIndentation Indentation is very important in a yaml file. In the template, we use 4 spaces as 1 level of indentation.\nNotification start and end Always start with a - note: to define a notification item. After you finish typing the message body, leave at least one line blank before starting another notification.\ntype general: Use this type to create a general notification. It will ignore pkg_name and version information.\npackage: A notification that is related to a package updates. This type of note will first check if the user has installed the package (single one) with a version that is higher than the specified version number in pkg_name and version entries. If so, the notification will not be displayed. If not the user will see the notification before expiration date.\nexpire The expire decides how long to show users the notification. If current date has passed the date in expire, the notification will not be displayed.\nMessage body Use | to start a new line and put the markdown format text body in the next indentation level.\n","categories":"","description":"","excerpt":" SPS notification system In SPS, there is a notification dropdown …","ref":"/sps/adv_features/notification/","tags":"","title":"Notification system"},{"body":" Guidelines from bioconductor_docker.\n Running the systemPipeR with Docker Get a copy of the public docker image docker pull systempipe/systempipe_docker:latest  To run RStudio Server: docker run -e PASSWORD=systemPipe -p 8787:8787 \\ systempipe/systempipe_docker:latest  You can then open a web browser pointing to your docker host on port 8787. If you’re on Linux and using default settings, the docker host is 127.0.0.1 (or localhost, so the full URL to RStudio would be http://localhost:8787). If you are on Mac or Windows and running Docker Toolbox, you can determine the docker host with the docker-machine ip default command.\nIn the above command, -e PASSWORD= is setting the RStudio password and is required by the RStudio Docker image. It can be whatever you like except it cannot be rstudio. Log in to RStudio with the username rstudio and whatever password was specified, in this example systemPipe.\nTo run R from the command line: docker run -it --user rstudio systempipe/systempipe_docker:latest R  To open a Bash shell on the container: docker run -it --user rstudio systempipe/systempipe_docker:latest bash   Install Prerequisites: Linux Mac Windows\nInstructions here on how to install Docker Engine on Ubuntu.\nsudo apt-get update sudo apt-get install \\ apt-transport-https \\ ca-certificates \\ curl \\ gnupg curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg echo \\ \"deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \\ $(lsb_release -cs) stable\" | sudo tee /etc/apt/sources.list.d/docker.list \u003e /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io ## Verify that Docker Engine is installed correctly by running the hello-world image. sudo docker run hello-world  Uninstall sudo apt-get remove docker docker-engine docker.io containerd runc   Docker Hub Account To be able to share a custom image, please go to https://hub.docker.com and create a free account.\n Log in to the Docker Hub locally Login with your Docker ID to push and pull images from Docker Hub. If you don’t have a Docker ID, head over to https://hub.docker.com to create one.\ndocker login # Username: XXXX # Password: xxx # Login Succeeded   Run Docker docker run-dP systempipe/systempipe_docker:latest  Make sure the container is running:\ndocker ps # CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS # NAMES #5d007f66a7b3 systempipe/systempipe_docker:latest \"/init\" 5 minutes ago Up 5 minutes 0.0.0.0:49153-\u003e8787/tcp determined_easle  Login to the container Please check the NAMES in this example, determined_easle, to login into the container.\ndocker exec -it determined_easle /bin/bash  Other alternatives to run the container To run RStudio Server: docker run -e PASSWORD=systemPipe -p 8787:8787 \\ systempipe/systempipe_docker:latest  To run R from the command line: docker run -it --user rstudio systempipe/systempipe_docker:latest R  To open a Bash shell on the container: docker run -it --user rstudio systempipe/systempipe_docker:latest bash  Check R Version into the container R --version  Stop Docker docker stop determined_easle   Create your first repository Link Create a repository:  Sign in to Docker Hub. Click Create a Repository on the Docker Hub welcome page: Name it /my-repo. Click Create.  Build and push a container image to Docker Hub from your computer Start by creating a Dockerfile to specify your application mkdir docker_test cd docker_test touch Dockerfile  # Docker inheritance FROM systempipe/systempipe_docker:latest ## Install BiocStyle RUN R -e 'BiocManager::install(\"BiocStyle\")' # Install required Bioconductor package from devel version RUN R -e 'BiocManager::install(\"tgirke/systemPipeR\")' RUN R -e 'BiocManager::install(\"tgirke/systemPipeRdata\")' WORKDIR /home/rstudio/SPRojects COPY --chown=rstudio:rstudio . /home/rstudio/SPRojects # Metadata LABEL name=\"systempipe/systempipe_docker\" \\ version=$BIOCONDUCTOR_DOCKER_systempipe \\ url=\"https://github.com/systemPipeR/systempipe/systempipe_docker\" \\ vendor=\"systemPipeR Project\" \\ maintainer=\"email@gmail.com\" \\ description=\"Bioconductor docker image containing the systemPipeR Project\" \\ license=\"Artistic-2.0\"  Run docker build to build your Docker image docker build -t systempipe/systempipe_docker .  Run docker run to test your Docker image locally docker run -e PASSWORD=systemPipe -p 8787:8787 systempipe/systempipe_docker:latest  Run docker push to push your Docker image to Docker Hub docker push systempipe/systempipe_docker   Your repository in Docker Hub should now display a new latest tag under Tags   Make changes to the container and Create the new image Create a folder, for example:\ndocker run -dP systempipe/systempipe_docker docker ps ## To check the NAME \u003clucid_grothendieck\u003e docker exec -it lucid_grothendieck /bin/bash root@33c758eb1626:/# R  setwd(\"home/rstudio/\") systemPipeRdata::genWorkenvir(\"rnaseq\")  exit docker commit -m \"Added rnaseq template\" -a \"Dani Cassol\" lucid_grothendieck dcassol/systempipeworkshop2021:rnaseq docker push systempipe/systempipe_docker:rnaseq  Run the new image:\ndocker run -e PASSWORD=systemPipe -p 8787:8787 systempipe/systempipe_docker:rnaseq   Commands List which docker machines are available locally {bash, eval=FALSE}docker images\nList running containers {bash, eval=FALSE}docker ps\nList all containers {bash, eval=FALSE}docker ps -a\nResume a stopped container {bash, eval=FALSE}docker start \u003cCONTAINER ID\u003e\nShell into a running container {bash, eval=FALSE}docker exec -it \u003cCONTAINER ID\u003e /bin/bash\nStop OR remove a cointainer {bash, eval=FALSE}docker stop \u003cCONTAINER ID\u003e {bash, eval=FALSE}docker rm \u003cCONTAINER ID\u003e\nRemove a image {bash, eval=FALSE}docker rmi dcassol/systempipeworkshop2021:rnaseq\n Docker and GitHub Actions  To create a new token, go to Docker Hub Settings  1.1. Account Settings » Security » New Access Token 1.2. Add Access Token Description » Create 1.3. Copy the Access Token » Copy and Close\nGo to the Repository at GitHub  2.1. Settings \u003e Secrets \u003e New repository secret 2.2. Create a new secret with the name DOCKER_HUB_USERNAME and your Docker ID as value 2.3. Click at Add secret 2.4. Create a new secret with the name DOCKER_HUB_ACCESS_TOKEN and your Personal Access Token (PAT) as value (generated in the previous step)\nSet up the GitHub Actions workflow  steps: - name: Checkout Repo uses: actions/checkout@v2 - name: Login to Docker Hub uses: docker/login-action@v1 with: username: ${{ secrets.DOCKER_HUB_USERNAME }} password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}   Common Problems ## Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.24/auth: dial unix /var/run/docker.sock: connect: permission denied  Solution:\nsudo chmod 666 /var/run/docker.sock   Singularity Container Please download the Docker image of systemPipe, as follow:\nsingularity pull docker://systempipe/systempipe_docker:latest  You can also use the build command to download pre-built images from Docker. Unlike pull, build will convert the image to the latest Singularity image format after downloading it.\nsingularity build systempipe_docker_latest.sif docker://systempipe/systempipe_docker:latest  To run the container:\nsingularity shell systempipe_docker_latest.sif   Resources  Docker Run: How to create images from an application Docker Hub Quickstart Configure GitHub Actions Singularity  ","categories":"","description":"","excerpt":" Guidelines from bioconductor_docker.\n Running the systemPipeR with …","ref":"/sp/sp_docker/","tags":"","title":"SPR Docker container"},{"body":"document.addEventListener(\"DOMContentLoaded\", function() { document.querySelector(\"h1\").className = \"title\"; });  document.addEventListener(\"DOMContentLoaded\", function() { var links = document.links; for (var i = 0, linksLength = links.length; i Define environment settings and samples A typical workflow starts with generating the expected working environment containing the proper directory structure, input files, and parameter settings. To simplify this task, one can load one of the existing NGS workflows templates provided by systemPipeRdata into the current working directory. The following does this for the rnaseq template. The name of the resulting workflow directory can be specified under the mydirname argument. The default NULL uses the name of the chosen workflow. An error is issued if a directory of the same name and path exists already. On Linux and OS X systems one can also create new workflow instances from the command-line of a terminal as shown here. To apply workflows to custom data, the user needs to modify the targets file and if necessary update the corresponding .cwl and .yml files. A collection of pre-generated .cwl and .yml files are provided in the param/cwl subdirectory of each workflow template. They are also viewable in the GitHub repository of systemPipeRdata (see here).\nlibrary(systemPipeR) library(systemPipeRdata) genWorkenvir(workflow = \"rnaseq\", mydirname = NULL) setwd(\"rnaseq\")  Read Preprocessing Preprocessing with preprocessReads function The function preprocessReads allows to apply predefined or custom read preprocessing functions to all FASTQ files referenced in a SYSargs2 container, such as quality filtering or adaptor trimming routines. The paths to the resulting output FASTQ files are stored in the output slot of the SYSargs2 object. Internally, preprocessReads uses the FastqStreamer function from the ShortRead package to stream through large FASTQ files in a memory-efficient manner. The following example performs adaptor trimming with the trimLRPatterns function from the Biostrings package. After the trimming step a new targets file is generated (here targets_trimPE.txt) containing the paths to the trimmed FASTQ files. The new targets file can be used for the next workflow step with an updated SYSargs2 instance, e.g. running the NGS alignments with the trimmed FASTQ files.\nConstruct SYSargs2 object from cwl and yml param and targets files.\ntargetsPE \u003c- system.file(\"extdata\", \"targetsPE.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/preprocessReads/trim-pe\", package = \"systemPipeR\") trim \u003c- loadWorkflow(targets = targetsPE, wf_file = \"trim-pe.cwl\", input_file = \"trim-pe.yml\", dir_path = dir_path) trim \u003c- renderWF(trim, inputvars = c(FileName1 = \"_FASTQ_PATH1_\", FileName2 = \"_FASTQ_PATH2_\", SampleName = \"_SampleName_\")) trim output(trim)[1:2]  preprocessReads(args = trim, Fct = \"trimLRPatterns(Rpattern='GCCCGGGTAA', subject=fq)\", batchsize = 1e+05, overwrite = TRUE, compress = TRUE)  The following example shows how one can design a custom read preprocessing function using utilities provided by the ShortRead package, and then run it in batch mode with the ‘preprocessReads’ function (here on paired-end reads).\nfilterFct \u003c- function(fq, cutoff = 20, Nexceptions = 0) { qcount \u003c- rowSums(as(quality(fq), \"matrix\") \u003c= cutoff, na.rm = TRUE) # Retains reads where Phred scores are \u003e= cutoff with N exceptions fq[qcount \u003c= Nexceptions] } preprocessReads(args = trim, Fct = \"filterFct(fq, cutoff=20, Nexceptions=0)\", batchsize = 1e+05)  Preprocessing with TrimGalore! TrimGalore! is a wrapper tool to consistently apply quality and adapter trimming to fastq files, with some extra functionality for removing Reduced Representation Bisulfite-Seq (RRBS) libraries.\ntargets \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/trim_galore/trim_galore-se\", package = \"systemPipeR\") trimG \u003c- loadWorkflow(targets = targets, wf_file = \"trim_galore-se.cwl\", input_file = \"trim_galore-se.yml\", dir_path = dir_path) trimG \u003c- renderWF(trimG, inputvars = c(FileName = \"_FASTQ_PATH1_\", SampleName = \"_SampleName_\")) trimG cmdlist(trimG)[1:2] output(trimG)[1:2] ## Run Single Machine Option trimG \u003c- runCommandline(trimG[1], make_bam = FALSE)  Preprocessing with Trimmomatic targetsPE \u003c- system.file(\"extdata\", \"targetsPE.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/trimmomatic/trimmomatic-pe\", package = \"systemPipeR\") trimM \u003c- loadWorkflow(targets = targetsPE, wf_file = \"trimmomatic-pe.cwl\", input_file = \"trimmomatic-pe.yml\", dir_path = dir_path) trimM \u003c- renderWF(trimM, inputvars = c(FileName1 = \"_FASTQ_PATH1_\", FileName2 = \"_FASTQ_PATH2_\", SampleName = \"_SampleName_\")) trimM cmdlist(trimM)[1:2] output(trimM)[1:2] ## Run Single Machine Option trimM \u003c- runCommandline(trimM[1], make_bam = FALSE)  FASTQ quality report The following seeFastq and seeFastqPlot functions generate and plot a series of useful quality statistics for a set of FASTQ files including per cycle quality box plots, base proportions, base-level quality trends, relative k-mer diversity, length and occurrence distribution of reads, number of reads above quality cutoffs and mean quality distribution. The function seeFastq computes the quality statistics and stores the results in a relatively small list object that can be saved to disk with save() and reloaded with load() for later plotting. The argument klength specifies the k-mer length and batchsize the number of reads to a random sample from each FASTQ file.\nfqlist \u003c- seeFastq(fastq = infile1(trim), batchsize = 10000, klength = 8) pdf(\"./results/fastqReport.pdf\", height = 18, width = 4 * length(fqlist)) seeFastqPlot(fqlist) dev.off()    Figure 5: FASTQ quality report\n  Parallelization of FASTQ quality report on a single machine with multiple cores.\nf \u003c- function(x) seeFastq(fastq = infile1(trim)[x], batchsize = 1e+05, klength = 8) fqlist \u003c- bplapply(seq(along = trim), f, BPPARAM = MulticoreParam(workers = 4)) seeFastqPlot(unlist(fqlist, recursive = FALSE))  Parallelization of FASTQ quality report via scheduler (e.g. Slurm) across several compute nodes.\nlibrary(BiocParallel) library(batchtools) f \u003c- function(x) { library(systemPipeR) targetsPE \u003c- system.file(\"extdata\", \"targetsPE.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/preprocessReads/trim-pe\", package = \"systemPipeR\") trim \u003c- loadWorkflow(targets = targetsPE, wf_file = \"trim-pe.cwl\", input_file = \"trim-pe.yml\", dir_path = dir_path) trim \u003c- renderWF(trim, inputvars = c(FileName1 = \"_FASTQ_PATH1_\", FileName2 = \"_FASTQ_PATH2_\", SampleName = \"_SampleName_\")) seeFastq(fastq = infile1(trim)[x], batchsize = 1e+05, klength = 8) } resources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) param \u003c- BatchtoolsParam(workers = 4, cluster = \"slurm\", template = \"batchtools.slurm.tmpl\", resources = resources) fqlist \u003c- bplapply(seq(along = trim), f, BPPARAM = param) seeFastqPlot(unlist(fqlist, recursive = FALSE))  NGS Alignment software After quality control, the sequence reads can be aligned to a reference genome or transcriptome database. The following sessions present some NGS sequence alignment software. Select the most accurate aligner and determining the optimal parameter for your custom data set project.\nFor all the following examples, it is necessary to install the respective software and export the PATH accordingly. If it is available Environment Module in the system, you can load all the request software with moduleload(args) function.\nAlignment with HISAT2 using SYSargs2 The following steps will demonstrate how to use the short read aligner Hisat2 (Kim, Langmead, and Salzberg 2015) in both interactive job submissions and batch submissions to queuing systems of clusters using the systemPipeR's new CWL command-line interface.\nThe parameter settings of the aligner are defined in the hisat2-mapping-se.cwl and hisat2-mapping-se.yml files. The following shows how to construct the corresponding SYSargs2 object, here args.\ntargets \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/hisat2/hisat2-se\", package = \"systemPipeR\") args \u003c- loadWorkflow(targets = targets, wf_file = \"hisat2-mapping-se.cwl\", input_file = \"hisat2-mapping-se.yml\", dir_path = dir_path) args \u003c- renderWF(args, inputvars = c(FileName = \"_FASTQ_PATH1_\", SampleName = \"_SampleName_\")) args  ## Instance of 'SYSargs2': ## Slot names/accessors: ## targets: 18 (M1A...V12B), targetsheader: 4 (lines) ## modules: 1 ## wf: 0, clt: 1, yamlinput: 7 (components) ## input: 18, output: 18 ## cmdlist: 18 ## WF Steps: ## 1. hisat2-mapping-se (rendered: TRUE)  cmdlist(args)[1:2]  ## $M1A ## $M1A$`hisat2-mapping-se` ## [1] \"hisat2 -S ./results/M1A.sam -x ./data/tair10.fasta -k 1 --min-intronlen 30 --max-intronlen 3000 -U ./data/SRR446027_1.fastq.gz --threads 4\" ## ## ## $M1B ## $M1B$`hisat2-mapping-se` ## [1] \"hisat2 -S ./results/M1B.sam -x ./data/tair10.fasta -k 1 --min-intronlen 30 --max-intronlen 3000 -U ./data/SRR446028_1.fastq.gz --threads 4\"  output(args)[1:2]  ## $M1A ## $M1A$`hisat2-mapping-se` ## [1] \"./results/M1A.sam\" ## ## ## $M1B ## $M1B$`hisat2-mapping-se` ## [1] \"./results/M1B.sam\"  Subsetting SYSargs2 class slots for each workflow step.\nsubsetWF(args, slot = \"input\", subset = \"FileName\")[1:2] ## Subsetting the input files for this particular workflow  ## M1A M1B ## \"./data/SRR446027_1.fastq.gz\" \"./data/SRR446028_1.fastq.gz\"  subsetWF(args, slot = \"output\", subset = 1, index = 1)[1:2] ## Subsetting the output files for one particular step in the workflow  ## M1A M1B ## \"./results/M1A.sam\" \"./results/M1B.sam\"  subsetWF(args, slot = \"step\", subset = 1)[1] ## Subsetting the command-lines for one particular step in the workflow  ## M1A ## \"hisat2 -S ./results/M1A.sam -x ./data/tair10.fasta -k 1 --min-intronlen 30 --max-intronlen 3000 -U ./data/SRR446027_1.fastq.gz --threads 4\"  subsetWF(args, slot = \"output\", subset = 1, index = 1, delete = TRUE)[1] ## DELETING specific output files  ## The subset cannot be deleted: no such file ## M1A ## \"./results/M1A.sam\"  Build Hisat2 index.\ndir_path \u003c- system.file(\"extdata/cwl/hisat2/hisat2-idx\", package = \"systemPipeR\") idx \u003c- loadWorkflow(targets = NULL, wf_file = \"hisat2-index.cwl\", input_file = \"hisat2-index.yml\", dir_path = dir_path) idx \u003c- renderWF(idx) idx cmdlist(idx) ## Run runCommandline(idx, make_bam = FALSE)  Interactive job submissions in a single machine To simplify the short read alignment execution for the user, the command-line can be run with the runCommandline function. The execution will be on a single machine without submitting to a queuing system of a computer cluster. This way, the input FASTQ files will be processed sequentially. By default runCommandline auto detects SAM file outputs and converts them to sorted and indexed BAM files, using internally the Rsamtools package (Morgan et al. 2019). Besides, runCommandline allows the user to create a dedicated results folder for each workflow and a sub-folder for each sample defined in the targets file. This includes all the output and log files for each step. When these options are used, the output location will be updated by default and can be assigned to the same object.\nrunCommandline(args, make_bam = FALSE) ## generates alignments and writes *.sam files to ./results folder args \u003c- runCommandline(args, make_bam = TRUE) ## same as above but writes files and converts *.sam files to sorted and indexed BAM files. Assigning the new extention of the output files to the object args.  If available, multiple CPU cores can be used for processing each file. The number of CPU cores (here 4) to use for each process is defined in the *.yml file. With yamlinput(args)['thread'] one can return this value from the SYSargs2 object.\nParallelization on clusters Alternatively, the computation can be greatly accelerated by processing many files in parallel using several compute nodes of a cluster, where a scheduling/queuing system is used for load balancing. For this the clusterRun function submits the computing requests to the scheduler using the run specifications defined by runCommandline.\nTo avoid over-subscription of CPU cores on the compute nodes, the value from yamlinput(args)['thread'] is passed on to the submission command, here ncpus in the resources list object. The number of independent parallel cluster processes is defined under the Njobs argument. The following example will run 18 processes in parallel using for each 4 CPU cores. If the resources available on a cluster allow running all 18 processes at the same time then the shown sample submission will utilize in total 72 CPU cores. Note, clusterRun can be used with most queueing systems as it is based on utilities from the batchtools package which supports the use of template files (*.tmpl) for defining the run parameters of different schedulers. To run the following code, one needs to have both a conf file (see .batchtools.conf.R samples here) and a template file (see *.tmpl samples here) for the queueing available on a system. The following example uses the sample conf and template files for the Slurm scheduler provided by this package.\nlibrary(batchtools) resources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) reg \u003c- clusterRun(args, FUN = runCommandline, more.args = list(args = args, make_bam = TRUE, dir = FALSE), conffile = \".batchtools.conf.R\", template = \"batchtools.slurm.tmpl\", Njobs = 18, runid = \"01\", resourceList = resources) getStatus(reg = reg) waitForJobs(reg = reg)  Check and update the output location if necessary.\nargs \u003c- output_update(args, dir = FALSE, replace = TRUE, extension = c(\".sam\", \".bam\")) ## Updates the output(args) to the right location in the subfolders output(args)  Create new targets file To establish the connectivity to the next workflow step, one can write a new targets file with the writeTargetsout function. The new targets file serves as input to the next loadWorkflow and renderWF call.\nnames(clt(args)) writeTargetsout(x = args, file = \"default\", step = 1, new_col = \"FileName\", new_col_output_index = 1, overwrite = TRUE)  Alignment with HISAT2 and SAMtools Alternatively, it possible to build an workflow with HISAT2 and SAMtools.\ntargets \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/workflow-hisat2/workflow-hisat2-se\", package = \"systemPipeR\") WF \u003c- loadWorkflow(targets = targets, wf_file = \"workflow_hisat2-se.cwl\", input_file = \"workflow_hisat2-se.yml\", dir_path = dir_path) WF \u003c- renderWF(WF, inputvars = c(FileName = \"_FASTQ_PATH1_\", SampleName = \"_SampleName_\")) WF cmdlist(WF)[1:2] output(WF)[1:2]  Alignment with Tophat2 The NGS reads of this project can also be aligned against the reference genome sequence using Bowtie2/TopHat2 (Kim et al. 2013; Langmead and Salzberg 2012).\nBuild Bowtie2 index.\ndir_path \u003c- system.file(\"extdata/cwl/bowtie2/bowtie2-idx\", package = \"systemPipeR\") idx \u003c- loadWorkflow(targets = NULL, wf_file = \"bowtie2-index.cwl\", input_file = \"bowtie2-index.yml\", dir_path = dir_path) idx \u003c- renderWF(idx) idx cmdlist(idx) ## Run in single machine runCommandline(idx, make_bam = FALSE)  The parameter settings of the aligner are defined in the tophat2-mapping-pe.cwl and tophat2-mapping-pe.yml files. The following shows how to construct the corresponding SYSargs2 object, here tophat2PE.\ntargetsPE \u003c- system.file(\"extdata\", \"targetsPE.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/tophat2/tophat2-pe\", package = \"systemPipeR\") tophat2PE \u003c- loadWorkflow(targets = targetsPE, wf_file = \"tophat2-mapping-pe.cwl\", input_file = \"tophat2-mapping-pe.yml\", dir_path = dir_path) tophat2PE \u003c- renderWF(tophat2PE, inputvars = c(FileName1 = \"_FASTQ_PATH1_\", FileName2 = \"_FASTQ_PATH2_\", SampleName = \"_SampleName_\")) tophat2PE cmdlist(tophat2PE)[1:2] output(tophat2PE)[1:2] ## Run in single machine tophat2PE \u003c- runCommandline(tophat2PE[1], make_bam = TRUE)  Parallelization on clusters.\nresources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) reg \u003c- clusterRun(tophat2PE, FUN = runCommandline, more.args = list(args = tophat2PE, make_bam = TRUE, dir = FALSE), conffile = \".batchtools.conf.R\", template = \"batchtools.slurm.tmpl\", Njobs = 18, runid = \"01\", resourceList = resources) waitForJobs(reg = reg)  Create new targets file\nnames(clt(tophat2PE)) writeTargetsout(x = tophat2PE, file = \"default\", step = 1, new_col = \"tophat2PE\", new_col_output_index = 1, overwrite = TRUE)  Alignment with Bowtie2 (e.g. for miRNA profiling) The following example runs Bowtie2 as a single process without submitting it to a cluster.\nBuilding the index:\ndir_path \u003c- system.file(\"extdata/cwl/bowtie2/bowtie2-idx\", package = \"systemPipeR\") idx \u003c- loadWorkflow(targets = NULL, wf_file = \"bowtie2-index.cwl\", input_file = \"bowtie2-index.yml\", dir_path = dir_path) idx \u003c- renderWF(idx) idx cmdlist(idx) ## Run in single machine runCommandline(idx, make_bam = FALSE)  Building all the command-line:\ntargetsPE \u003c- system.file(\"extdata\", \"targetsPE.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/bowtie2/bowtie2-pe\", package = \"systemPipeR\") bowtiePE \u003c- loadWorkflow(targets = targetsPE, wf_file = \"bowtie2-mapping-pe.cwl\", input_file = \"bowtie2-mapping-pe.yml\", dir_path = dir_path) bowtiePE \u003c- renderWF(bowtiePE, inputvars = c(FileName1 = \"_FASTQ_PATH1_\", FileName2 = \"_FASTQ_PATH2_\", SampleName = \"_SampleName_\")) bowtiePE cmdlist(bowtiePE)[1:2] output(bowtiePE)[1:2]  Running all the jobs to computing nodes.\nresources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) reg \u003c- clusterRun(bowtiePE, FUN = runCommandline, more.args = list(args = bowtiePE, dir = FALSE), conffile = \".batchtools.conf.R\", template = \"batchtools.slurm.tmpl\", Njobs = 18, runid = \"01\", resourceList = resources) getStatus(reg = reg)  Alternatively, it possible to run all the jobs in a single machine.\nbowtiePE \u003c- runCommandline(bowtiePE)  Create new targets file.\nnames(clt(bowtiePE)) writeTargetsout(x = bowtiePE, file = \"default\", step = 1, new_col = \"bowtiePE\", new_col_output_index = 1, overwrite = TRUE)  Alignment with BWA-MEM (e.g. for VAR-Seq) The following example runs BWA-MEM as a single process without submitting it to a cluster. ##TODO: add reference\nBuild the index:\ndir_path \u003c- system.file(\"extdata/cwl/bwa/bwa-idx\", package = \"systemPipeR\") idx \u003c- loadWorkflow(targets = NULL, wf_file = \"bwa-index.cwl\", input_file = \"bwa-index.yml\", dir_path = dir_path) idx \u003c- renderWF(idx) idx cmdlist(idx) # Indexes reference genome ## Run runCommandline(idx, make_bam = FALSE)  Running the alignment:\ntargetsPE \u003c- system.file(\"extdata\", \"targetsPE.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/bwa/bwa-pe\", package = \"systemPipeR\") bwaPE \u003c- loadWorkflow(targets = targetsPE, wf_file = \"bwa-pe.cwl\", input_file = \"bwa-pe.yml\", dir_path = dir_path) bwaPE \u003c- renderWF(bwaPE, inputvars = c(FileName1 = \"_FASTQ_PATH1_\", FileName2 = \"_FASTQ_PATH2_\", SampleName = \"_SampleName_\")) bwaPE cmdlist(bwaPE)[1:2] output(bwaPE)[1:2] ## Single Machine bwaPE \u003c- runCommandline(args = bwaPE, make_bam = FALSE) ## Cluster library(batchtools) resources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) reg \u003c- clusterRun(bwaPE, FUN = runCommandline, more.args = list(args = bwaPE, dir = FALSE), conffile = \".batchtools.conf.R\", template = \"batchtools.slurm.tmpl\", Njobs = 18, runid = \"01\", resourceList = resources) getStatus(reg = reg)  Create new targets file.\nnames(clt(bwaPE)) writeTargetsout(x = bwaPE, file = \"default\", step = 1, new_col = \"bwaPE\", new_col_output_index = 1, overwrite = TRUE)  Alignment with Rsubread (e.g. for RNA-Seq) The following example shows how one can use within the environment the R-based aligner , allowing running from R or command-line.\n## Build the index: dir_path \u003c- system.file(\"extdata/cwl/rsubread/rsubread-idx\", package = \"systemPipeR\") idx \u003c- loadWorkflow(targets = NULL, wf_file = \"rsubread-index.cwl\", input_file = \"rsubread-index.yml\", dir_path = dir_path) idx \u003c- renderWF(idx) idx cmdlist(idx) runCommandline(args = idx, make_bam = FALSE) ## Running the alignment: targets \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/rsubread/rsubread-se\", package = \"systemPipeR\") rsubread \u003c- loadWorkflow(targets = targets, wf_file = \"rsubread-mapping-se.cwl\", input_file = \"rsubread-mapping-se.yml\", dir_path = dir_path) rsubread \u003c- renderWF(rsubread, inputvars = c(FileName = \"_FASTQ_PATH1_\", SampleName = \"_SampleName_\")) rsubread cmdlist(rsubread)[1] ## Single Machine rsubread \u003c- runCommandline(args = rsubread[1])  Create new targets file.\nnames(clt(rsubread)) writeTargetsout(x = rsubread, file = \"default\", step = 1, new_col = \"rsubread\", new_col_output_index = 1, overwrite = TRUE)  Alignment with gsnap (e.g. for VAR-Seq and RNA-Seq) Another R-based short read aligner is gsnap from the gmapR package (Wu and Nacu 2010). The code sample below introduces how to run this aligner on multiple nodes of a compute cluster.\n## Build the index: dir_path \u003c- system.file(\"extdata/cwl/gsnap/gsnap-idx\", package = \"systemPipeR\") idx \u003c- loadWorkflow(targets = NULL, wf_file = \"gsnap-index.cwl\", input_file = \"gsnap-index.yml\", dir_path = dir_path) idx \u003c- renderWF(idx) idx cmdlist(idx) runCommandline(args = idx, make_bam = FALSE) ## Running the alignment: targetsPE \u003c- system.file(\"extdata\", \"targetsPE.txt\", package = \"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/gsnap/gsnap-pe\", package = \"systemPipeR\") gsnap \u003c- loadWorkflow(targets = targetsPE, wf_file = \"gsnap-mapping-pe.cwl\", input_file = \"gsnap-mapping-pe.yml\", dir_path = dir_path) gsnap \u003c- renderWF(gsnap, inputvars = c(FileName1 = \"_FASTQ_PATH1_\", FileName2 = \"_FASTQ_PATH2_\", SampleName = \"_SampleName_\")) gsnap cmdlist(gsnap)[1] output(gsnap)[1] ## Cluster library(batchtools) resources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) reg \u003c- clusterRun(gsnap, FUN = runCommandline, more.args = list(args = gsnap, make_bam = FALSE), conffile = \".batchtools.conf.R\", template = \"batchtools.slurm.tmpl\", Njobs = 18, runid = \"01\", resourceList = resources) getStatus(reg = reg) gsnap \u003c- output_update(gsnap, dir = FALSE, replace = TRUE, extension = c(\".sam\", \".bam\"))  Create new targets file.\nnames(clt(gsnap)) writeTargetsout(x = gsnap, file = \"default\", step = 1, new_col = \"gsnap\", new_col_output_index = 1, overwrite = TRUE)  Create symbolic links for viewing BAM files in IGV The genome browser IGV supports reading of indexed/sorted BAM files via web URLs. This way it can be avoided to create unnecessary copies of these large files. To enable this approach, an HTML directory with Http access needs to be available in the user account (e.g. home/publichtml) of a system. If this is not the case then the BAM files need to be moved or copied to the system where IGV runs. In the following, htmldir defines the path to the HTML directory with http access where the symbolic links to the BAM files will be stored. The corresponding URLs will be written to a text file specified under the _urlfile_ argument.\nsymLink2bam(sysargs = args, htmldir = c(\"~/.html/\", \"somedir/\"), urlbase = \"http://myserver.edu/~username/\", urlfile = \"IGVurl.txt\")  Read counting for mRNA profiling experiments Create txdb (needs to be done only once).\nlibrary(GenomicFeatures) txdb \u003c- makeTxDbFromGFF(file = \"data/tair10.gff\", format = \"gff\", dataSource = \"TAIR\", organism = \"Arabidopsis thaliana\") saveDb(txdb, file = \"./data/tair10.sqlite\")  The following performs read counting with summarizeOverlaps in parallel mode with multiple cores.\nlibrary(BiocParallel) txdb \u003c- loadDb(\"./data/tair10.sqlite\") eByg \u003c- exonsBy(txdb, by = \"gene\") outpaths \u003c- subsetWF(args, slot = \"output\", subset = 1, index = 1) bfl \u003c- BamFileList(outpaths, yieldSize = 50000, index = character()) multicoreParam \u003c- MulticoreParam(workers = 4) register(multicoreParam) registered() counteByg \u003c- bplapply(bfl, function(x) summarizeOverlaps(eByg, x, mode = \"Union\", ignore.strand = TRUE, inter.feature = TRUE, singleEnd = TRUE)) # Note: for strand-specific RNA-Seq set 'ignore.strand=FALSE' and for PE data set # 'singleEnd=FALSE' countDFeByg \u003c- sapply(seq(along = counteByg), function(x) assays(counteByg[[x]])$counts) rownames(countDFeByg) \u003c- names(rowRanges(counteByg[[1]])) colnames(countDFeByg) \u003c- names(bfl) rpkmDFeByg \u003c- apply(countDFeByg, 2, function(x) returnRPKM(counts = x, ranges = eByg)) write.table(countDFeByg, \"results/countDFeByg.xls\", col.names = NA, quote = FALSE, sep = \"\\t\") write.table(rpkmDFeByg, \"results/rpkmDFeByg.xls\", col.names = NA, quote = FALSE, sep = \"\\t\")  Please note, in addition to read counts this step generates RPKM normalized expression values. For most statistical differential expression or abundance analysis methods, such as edgeR or DESeq2, the raw count values should be used as input. The usage of RPKM values should be restricted to specialty applications required by some users, e.g. manually comparing the expression levels of different genes or features.\nRead counting with summarizeOverlaps using multiple nodes of a cluster.\nlibrary(BiocParallel) f \u003c- function(x) { library(systemPipeR) library(BiocParallel) library(GenomicFeatures) txdb \u003c- loadDb(\"./data/tair10.sqlite\") eByg \u003c- exonsBy(txdb, by = \"gene\") args \u003c- systemArgs(sysma = \"param/tophat.param\", mytargets = \"targets.txt\") outpaths \u003c- subsetWF(args, slot = \"output\", subset = 1, index = 1) bfl \u003c- BamFileList(outpaths, yieldSize = 50000, index = character()) summarizeOverlaps(eByg, bfl[x], mode = \"Union\", ignore.strand = TRUE, inter.feature = TRUE, singleEnd = TRUE) } resources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) param \u003c- BatchtoolsParam(workers = 4, cluster = \"slurm\", template = \"batchtools.slurm.tmpl\", resources = resources) counteByg \u003c- bplapply(seq(along = args), f, BPPARAM = param) countDFeByg \u003c- sapply(seq(along = counteByg), function(x) assays(counteByg[[x]])$counts) rownames(countDFeByg) \u003c- names(rowRanges(counteByg[[1]])) colnames(countDFeByg) \u003c- names(outpaths)  Useful commands for monitoring the progress of submitted jobs\ngetStatus(reg = reg) outpaths \u003c- subsetWF(args, slot = \"output\", subset = 1, index = 1) file.exists(outpaths) sapply(1:length(outpaths), function(x) loadResult(reg, id = x)) # Works after job completion  Read and alignment count stats Generate a table of read and alignment counts for all samples.\nread_statsDF \u003c- alignStats(args) write.table(read_statsDF, \"results/alignStats.xls\", row.names = FALSE, quote = FALSE, sep = \"\\t\")  The following shows the first four lines of the sample alignment stats file provided by the systemPipeR package. For simplicity the number of PE reads is multiplied here by 2 to approximate proper alignment frequencies where each read in a pair is counted.\nread.table(system.file(\"extdata\", \"alignStats.xls\", package = \"systemPipeR\"), header = TRUE)[1:4, ]  ## FileName Nreads2x Nalign Perc_Aligned Nalign_Primary Perc_Aligned_Primary ## 1 M1A 192918 177961 92.24697 177961 92.24697 ## 2 M1B 197484 159378 80.70426 159378 80.70426 ## 3 A1A 189870 176055 92.72397 176055 92.72397 ## 4 A1B 188854 147768 78.24457 147768 78.24457  Parallelization of read/alignment stats on single machine with multiple cores.\nf \u003c- function(x) alignStats(args[x]) read_statsList \u003c- bplapply(seq(along = args), f, BPPARAM = MulticoreParam(workers = 8)) read_statsDF \u003c- do.call(\"rbind\", read_statsList)  Parallelization of read/alignment stats via scheduler (e.g. Slurm) across several compute nodes.\nlibrary(BiocParallel) library(batchtools) f \u003c- function(x) { library(systemPipeR) targets \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") dir_path \u003c- \"param/cwl/hisat2/hisat2-se\" ## TODO: replace path to system.file args \u003c- loadWorkflow(targets = targets, wf_file = \"hisat2-mapping-se.cwl\", input_file = \"hisat2-mapping-se.yml\", dir_path = dir_path) args \u003c- renderWF(args, inputvars = c(FileName = \"_FASTQ_PATH1_\", SampleName = \"_SampleName_\")) args \u003c- output_update(args, dir = FALSE, replace = TRUE, extension = c(\".sam\", \".bam\")) alignStats(args[x]) } resources \u003c- list(walltime = 120, ntasks = 1, ncpus = 4, memory = 1024) param \u003c- BatchtoolsParam(workers = 4, cluster = \"slurm\", template = \"batchtools.slurm.tmpl\", resources = resources) read_statsList \u003c- bplapply(seq(along = args), f, BPPARAM = param) read_statsDF \u003c- do.call(\"rbind\", read_statsList)  Read counting for miRNA profiling experiments Download miRNA genes from miRBase.\nsystem(\"wget ftp://mirbase.org/pub/mirbase/19/genomes/My_species.gff3 -P ./data/\") gff \u003c- import.gff(\"./data/My_species.gff3\") gff \u003c- split(gff, elementMetadata(gff)$ID) bams \u003c- names(bampaths) names(bams) \u003c- targets$SampleName bfl \u003c- BamFileList(bams, yieldSize = 50000, index = character()) countDFmiR \u003c- summarizeOverlaps(gff, bfl, mode = \"Union\", ignore.strand = FALSE, inter.feature = FALSE) # Note: inter.feature=FALSE important since pre and mature miRNA ranges overlap rpkmDFmiR \u003c- apply(countDFmiR, 2, function(x) returnRPKM(counts = x, gffsub = gff)) write.table(assays(countDFmiR)$counts, \"results/countDFmiR.xls\", col.names = NA, quote = FALSE, sep = \"\\t\") write.table(rpkmDFmiR, \"results/rpkmDFmiR.xls\", col.names = NA, quote = FALSE, sep = \"\\t\")  Correlation analysis of samples The following computes the sample-wise Spearman correlation coefficients from the rlog (regularized-logarithm) transformed expression values generated with the DESeq2 package. After transformation to a distance matrix, hierarchical clustering is performed with the hclust function and the result is plotted as a dendrogram (sample_tree.pdf).\nlibrary(DESeq2, warn.conflicts = FALSE, quietly = TRUE) library(ape, warn.conflicts = FALSE) countDFpath \u003c- system.file(\"extdata\", \"countDFeByg.xls\", package = \"systemPipeR\") countDF \u003c- as.matrix(read.table(countDFpath)) colData \u003c- data.frame(row.names = targets.as.df(targets(args))$SampleName, condition = targets.as.df(targets(args))$Factor) dds \u003c- DESeqDataSetFromMatrix(countData = countDF, colData = colData, design = ~condition)  ## Warning in DESeqDataSet(se, design = design, ignoreRank): some variables in ## design formula are characters, converting to factors  d \u003c- cor(assay(rlog(dds)), method = \"spearman\") hc \u003c- hclust(dist(1 - d)) plot.phylo(as.phylo(hc), type = \"p\", edge.col = 4, edge.width = 3, show.node.label = TRUE, no.margin = TRUE)  Figure 6: Correlation dendrogram of samples for rlog values.\n  Alternatively, the clustering can be performed with RPKM normalized expression values. In combination with Spearman correlation the results of the two clustering methods are often relatively similar.\nrpkmDFeBygpath \u003c- system.file(\"extdata\", \"rpkmDFeByg.xls\", package = \"systemPipeR\") rpkmDFeByg \u003c- read.table(rpkmDFeBygpath, check.names = FALSE) rpkmDFeByg \u003c- rpkmDFeByg[rowMeans(rpkmDFeByg) \u003e 50, ] d \u003c- cor(rpkmDFeByg, method = \"spearman\") hc \u003c- hclust(as.dist(1 - d)) plot.phylo(as.phylo(hc), type = \"p\", edge.col = \"blue\", edge.width = 2, show.node.label = TRUE, no.margin = TRUE)  DEG analysis with edgeR The following *run_edgeR* function is a convenience wrapper for identifying differentially expressed genes (DEGs) in batch mode with *edgeR’s GML method (Robinson, McCarthy, and Smyth 2010) for any number of pairwise sample comparisons specified under the cmp* argument. Users are strongly encouraged to consult the edgeR vignette for more detailed information on this topic and how to properly run edgeR on data sets with more complex experimental designs.\ntargetspath \u003c- system.file(\"extdata\", \"targets.txt\", package = \"systemPipeR\") targets \u003c- read.delim(targetspath, comment = \"#\") cmp \u003c- readComp(file = targetspath, format = \"matrix\", delim = \"-\") cmp[[1]]  ## [,1] [,2] ## [1,] \"M1\" \"A1\" ## [2,] \"M1\" \"V1\" ## [3,] \"A1\" \"V1\" ## [4,] \"M6\" \"A6\" ## [5,] \"M6\" \"V6\" ## [6,] \"A6\" \"V6\" ## [7,] \"M12\" \"A12\" ## [8,] \"M12\" \"V12\" ## [9,] \"A12\" \"V12\"  countDFeBygpath \u003c- system.file(\"extdata\", \"countDFeByg.xls\", package = \"systemPipeR\") countDFeByg \u003c- read.delim(countDFeBygpath, row.names = 1) edgeDF \u003c- run_edgeR(countDF = countDFeByg, targets = targets, cmp = cmp[[1]], independent = FALSE, mdsplot = \"\")  ## Disp = 0.21829 , BCV = 0.4672  Filter and plot DEG results for up and down-regulated genes. Because of the small size of the toy data set used by this vignette, the FDR value has been set to a relatively high threshold (here 10%). More commonly used FDR cutoffs are 1% or 5%. The definition of ‘up’ and ‘down’ is given in the corresponding help file. To open it, type ?filterDEGs in the R console.\nDEG_list \u003c- filterDEGs(degDF = edgeDF, filter = c(Fold = 2, FDR = 10))  Figure 7: Up and down regulated DEGs identified by edgeR.\n  names(DEG_list)  ## [1] \"UporDown\" \"Up\" \"Down\" \"Summary\"  DEG_list$Summary[1:4, ]  ## Comparisons Counts_Up_or_Down Counts_Up Counts_Down ## M1-A1 M1-A1 0 0 0 ## M1-V1 M1-V1 1 1 0 ## A1-V1 A1-V1 1 1 0 ## M6-A6 M6-A6 0 0 0  DEG analysis with DESeq2 The following *run_DESeq2* function is a convenience wrapper for identifying DEGs in batch mode with DESeq2* (Love, Huber, and Anders 2014) for any number of pairwise sample comparisons specified under the cmp* argument. Users are strongly encouraged to consult the DESeq2 vignette for more detailed information on this topic and how to properly run DESeq2 on data sets with more complex experimental designs.\ndegseqDF \u003c- run_DESeq2(countDF = countDFeByg, targets = targets, cmp = cmp[[1]], independent = FALSE)  ## Warning in DESeqDataSet(se, design = design, ignoreRank): some variables in ## design formula are characters, converting to factors  Filter and plot DEG results for up and down-regulated genes.\nDEG_list2 \u003c- filterDEGs(degDF = degseqDF, filter = c(Fold = 2, FDR = 10))  Figure 8: Up and down regulated DEGs identified by DESeq2.\n  Venn Diagrams The function overLapper can compute Venn intersects for large numbers of sample sets (up to 20 or more) and vennPlot can plot 2-5 way Venn diagrams. A useful feature is the possibility to combine the counts from several Venn comparisons with the same number of sample sets in a single Venn diagram (here for 4 up and down DEG sets).\nvennsetup \u003c- overLapper(DEG_list$Up[6:9], type = \"vennsets\") vennsetdown \u003c- overLapper(DEG_list$Down[6:9], type = \"vennsets\") vennPlot(list(vennsetup, vennsetdown), mymain = \"\", mysub = \"\", colmode = 2, ccol = c(\"blue\", \"red\"))  Figure 9: Venn Diagram for 4 Up and Down DEG Sets.\n  GO term enrichment analysis of DEGs Obtain gene-to-GO mappings The following shows how to obtain gene-to-GO mappings from biomaRt (here for A. thaliana) and how to organize them for the downstream GO term enrichment analysis. Alternatively, the gene-to-GO mappings can be obtained for many organisms from Bioconductor’s *.db genome annotation packages or GO annotation files provided by various genome databases. For each annotation, this relatively slow preprocessing step needs to be performed only once. Subsequently, the preprocessed data can be loaded with the load function as shown in the next subsection.\nlibrary(\"biomaRt\") listMarts() # To choose BioMart database listMarts(host = \"plants.ensembl.org\") m \u003c- useMart(\"plants_mart\", host = \"plants.ensembl.org\") listDatasets(m) m \u003c- useMart(\"plants_mart\", dataset = \"athaliana_eg_gene\", host = \"plants.ensembl.org\") listAttributes(m) # Choose data types you want to download go \u003c- getBM(attributes = c(\"go_id\", \"tair_locus\", \"namespace_1003\"), mart = m) go \u003c- go[go[, 3] != \"\", ] go[, 3] \u003c- as.character(go[, 3]) go[go[, 3] == \"molecular_function\", 3] \u003c- \"F\" go[go[, 3] == \"biological_process\", 3] \u003c- \"P\" go[go[, 3] == \"cellular_component\", 3] \u003c- \"C\" go[1:4, ] dir.create(\"./data/GO\") write.table(go, \"data/GO/GOannotationsBiomart_mod.txt\", quote = FALSE, row.names = FALSE, col.names = FALSE, sep = \"\\t\") catdb \u003c- makeCATdb(myfile = \"data/GO/GOannotationsBiomart_mod.txt\", lib = NULL, org = \"\", colno = c(1, 2, 3), idconv = NULL) save(catdb, file = \"data/GO/catdb.RData\")  Batch GO term enrichment analysis Apply the enrichment analysis to the DEG sets obtained in the above differential expression analysis. Note, in the following example the FDR filter is set here to an unreasonably high value, simply because of the small size of the toy data set used in this vignette. Batch enrichment analysis of many gene sets is performed with the GOCluster_Report function. When method=\"all\", it returns all GO terms passing the p-value cutoff specified under the cutoff arguments. When method=\"slim\", it returns only the GO terms specified under the myslimv argument. The given example shows how one can obtain such a GO slim vector from BioMart for a specific organism.\nload(\"data/GO/catdb.RData\") DEG_list \u003c- filterDEGs(degDF = edgeDF, filter = c(Fold = 2, FDR = 50), plot = FALSE) up_down \u003c- DEG_list$UporDown names(up_down) \u003c- paste(names(up_down), \"_up_down\", sep = \"\") up \u003c- DEG_list$Up names(up) \u003c- paste(names(up), \"_up\", sep = \"\") down \u003c- DEG_list$Down names(down) \u003c- paste(names(down), \"_down\", sep = \"\") DEGlist \u003c- c(up_down, up, down) DEGlist \u003c- DEGlist[sapply(DEGlist, length) \u003e 0] BatchResult \u003c- GOCluster_Report(catdb = catdb, setlist = DEGlist, method = \"all\", id_type = \"gene\", CLSZ = 2, cutoff = 0.9, gocats = c(\"MF\", \"BP\", \"CC\"), recordSpecGO = NULL) library(\"biomaRt\") m \u003c- useMart(\"plants_mart\", dataset = \"athaliana_eg_gene\", host = \"plants.ensembl.org\") goslimvec \u003c- as.character(getBM(attributes = c(\"goslim_goa_accession\"), mart = m)[, 1]) BatchResultslim \u003c- GOCluster_Report(catdb = catdb, setlist = DEGlist, method = \"slim\", id_type = \"gene\", myslimv = goslimvec, CLSZ = 10, cutoff = 0.01, gocats = c(\"MF\", \"BP\", \"CC\"), recordSpecGO = NULL)  Plot batch GO term results The data.frame generated by GOCluster_Report can be plotted with the goBarplot function. Because of the variable size of the sample sets, it may not always be desirable to show the results from different DEG sets in the same bar plot. Plotting single sample sets is achieved by subsetting the input data frame as shown in the first line of the following example.\ngos \u003c- BatchResultslim[grep(\"M6-V6_up_down\", BatchResultslim$CLID), ] gos \u003c- BatchResultslim pdf(\"GOslimbarplotMF.pdf\", height = 8, width = 10) goBarplot(gos, gocat = \"MF\") dev.off() goBarplot(gos, gocat = \"BP\") goBarplot(gos, gocat = \"CC\")  Figure 10: GO Slim Barplot for MF Ontology.\n  Clustering and heat maps The following example performs hierarchical clustering on the rlog transformed expression matrix subsetted by the DEGs identified in the above differential expression analysis. It uses a Pearson correlation-based distance measure and complete linkage for cluster join.\nlibrary(pheatmap) geneids \u003c- unique(as.character(unlist(DEG_list[[1]]))) y \u003c- assay(rlog(dds))[geneids, ] pdf(\"heatmap1.pdf\") pheatmap(y, scale = \"row\", clustering_distance_rows = \"correlation\", clustering_distance_cols = \"correlation\") dev.off()    Figure 11: Heat map with hierarchical clustering dendrograms of DEGs.\n  References Kim, Daehwan, Ben Langmead, and Steven L Salzberg. 2015. “HISAT: A Fast Spliced Aligner with Low Memory Requirements.” Nat. Methods 12 (4): 357–60.\n Kim, Daehwan, Geo Pertea, Cole Trapnell, Harold Pimentel, Ryan Kelley, and Steven L Salzberg. 2013. “TopHat2: Accurate Alignment of Transcriptomes in the Presence of Insertions, Deletions and Gene Fusions.” Genome Biol. 14 (4): R36. https://doi.org/10.1186/gb-2013-14-4-r36.\n Langmead, Ben, and Steven L Salzberg. 2012. “Fast Gapped-Read Alignment with Bowtie 2.” Nat. Methods 9 (4): 357–59. https://doi.org/10.1038/nmeth.1923.\n Love, Michael, Wolfgang Huber, and Simon Anders. 2014. “Moderated Estimation of Fold Change and Dispersion for RNA-seq Data with DESeq2.” Genome Biol. 15 (12): 550. https://doi.org/10.1186/s13059-014-0550-8.\n Morgan, Martin, Hervé Pagès, Valerie Obenchain, and Nathaniel Hayden. 2019. Rsamtools: Binary Alignment (BAM), FASTA, Variant Call (BCF), and Tabix File Import. http://bioconductor.org/packages/Rsamtools.\n Robinson, M D, D J McCarthy, and G K Smyth. 2010. “edgeR: A Bioconductor Package for Differential Expression Analysis of Digital Gene Expression Data.” Bioinformatics 26 (1): 139–40. https://doi.org/10.1093/bioinformatics/btp616.\n Wu, T D, and S Nacu. 2010. “Fast and SNP-tolerant Detection of Complex Variants and Splicing in Short Reads.” Bioinformatics 26 (7): 873–81. https://doi.org/10.1093/bioinformatics/btq057.\n  ","categories":"","description":"","excerpt":"document.addEventListener(\"DOMContentLoaded\", function() { …","ref":"/sp/spr/steps/","tags":"","title":"Workflow steps overview"},{"body":" SPS Canvas is a place to display and edit scrennshots from different plots. To start to use Canvas, you need to take some screenshots but clicking “To Canvas” buttons on different tabs/modules. After clicking, the screenshots will be automatically sent from these places to this Canvas.\nAfter SPS v1.1.0 this Canvas feature has been provided as a separate package {drawer}. If you like this feature and want to use outside of SPS, install {drawer}. It is fully compatible with both Shiny and R markdown.\nPrepare plots In other SPS tabs, adjust your plots to the optimal size by dragging the corner:\n  Then use the ‵toCanvas‵ button of that plot to send a screenshot of current plot to the Canvas. Or you can click on the “down arrow”  to save it to edit in other tools.\n  Use the Canvas   Figure 9 Canvas\n The Canvas area. Canvas drawing grids. By default, your objects are limited to these drawing grids, but you can change it from top options inside “canvas”. The grid area size is automatically calculated to fit your screen size when you start SPS. Object information. When you select any object on the Canvas, a bounding box will show to display the object’s dimensions, scale, angle and other information. You can disable them in the “View” menu To edit your screenshots, simply drag your screenshots from left to Canvas working area. You can add text or titles, and change the font color, decorations in this panel. Different Canvas options. Several menus and buttons help you to better control the Canvas. Hover your mouse on buttons will display a tooltip of their functionality.  Keyboard shortcuts are also enabled with SPS Canvas. Go to “help” menu to see these options.\nSupport Canvas only works on recent browsers versions, like Chrome, latest Edge, Firefox. IE is not supported. Also, some browser privacy extensions will block javascript and HTML5 canvas fingerprint. This will cause the screenshot to be blank.\n","categories":"","description":"","excerpt":" SPS Canvas is a place to display and edit scrennshots from different …","ref":"/sps/canvas/","tags":"","title":"Canvas"},{"body":" SPS interative guides (tutorials) SPS provides some interactive guides for users to familiarize the app. There is a very simple one-step welcome guide that will initialize everytime on app start to indicate where the guide dropdown is (top-right corner).\n Welcome guide on start  \nBy clicking the guide dropdown menu, you can open up the list of available guides. By default, we only proivde a main SPS guide.\n Guide dropdown menu  \nIf any of the guide is clicked in the dropdown, the corresponding interactive guide will start.\n Guide started  You can click “Next”, “Previous” or “Close” to navigate the guide or close the guide.\nCustom guide To build and provide your custom guides to your users, there is file guide_content.R created on SPS project initialization under the R folder: /R/guide_content.R. This is the place to define your own guide.\nThis file look like this:\n#################### Define your custom SPS tutorials ########################## # use `shinydashboardPlus::messageItem` to add your tutorials UI to this list guide_ui \u003c- list( ## An example is provided below shinydashboardPlus::messageItem( inputId = \"guide_main\", from = \"Main Guide\", icon = icon('home'), message = \"Brief introduction\" ) ) # use `cicerone::Cicerone$new()` to add your tutorials content to this list # See help `?cicerone::Cicerone` # A named list, each item's name must match the `inputId` in UI to trigger it in app. guide_content \u003c- try(list( ## An example is provided below, replace or add your own to the list guide_main = cicerone::Cicerone$new(overlay_click_next = TRUE)$ step(el = \"sidebarItemExpanded\", title = \"SPS tabs\", description = \"Browse SPS functionalities as tabs from the left\", position = \"right-center\")$ ... ))  There are two parts that you need to define: UI and actual guide content.\nUI UI is what users see inside the dropdown menu. Guides UI needs to be stored in a list and each item should be a shinydashboardPlus::messageItem.\n inputId must be unique. from is the short description of the guide. icon should be a call of shiny::icon(), the icon of the guide in dropdown menu message: short description of the guide in dropdown menu  To add multiple guides' UI, for example:\nguide_ui \u003c- list( ## An example is provided below shinydashboardPlus::messageItem( inputId = \"guide1\", ... ), shinydashboardPlus::messageItem( inputId = \"guide2\", ... ), ... )  Guide content The guide content is defined with the {cicerone{blk}} package with cicerone::Cicerone R6 class and also in a named list.\nThe name of each item in a list must match the name of inputId in guide UI. For example, we have two guides\nguide_ui \u003c- list( ## An example is provided below shinydashboardPlus::messageItem( inputId = \"guide1\", ... ), shinydashboardPlus::messageItem( inputId = \"guide2\", ... ), ... ) guide_content \u003c- try(list( guide1 = cicerone::Cicerone$new(overlay_click_next = TRUE)$ ..., guide2 = cicerone::Cicerone$new(overlay_click_next = TRUE)$ ... ))  The “guide1” of inputId in the guide_ui must match “guide1” in guide_content list.\nThe “guide2” of inputId in the guide_ui must match “guide2” in guide_content list.\nDefine guide content How to define the guide content will not be expanded here, read details in the {cicerone{blk}} package manual. Here are some key points:\n The cicerone::Cicerone is R6 class, and it is obejct oriented, so you need to use cicerone::Cicerone$new() method to create a new object before you can add any guide step. R6 methods calling can be chained together, like how to define steps: guide_content \u003c- try(list( guide1 = cicerone::Cicerone$new()$ # chain object creation with step defining step(...)$ # chain step1 to step2 step(...)$ # step2 to step3 step(...)$ # step3 to step4 step(...) # DO NOT use `$` for the last step ))    If you follow the manual of {cicerone} to define steps, the guide will look like this when users click it.\n Cicerone steps  ","categories":"","description":"","excerpt":" SPS interative guides (tutorials) SPS provides some interactive …","ref":"/sps/adv_features/guide/","tags":"","title":"SPS Guide"},{"body":"document.addEventListener(\"DOMContentLoaded\", function() { document.querySelector(\"h1\").className = \"title\"; });  document.addEventListener(\"DOMContentLoaded\", function() { var links = document.links; for (var i = 0, linksLength = links.length; i Workflow templates The intended way of running systemPipeR workflows is via *.Rmd files, which can be executed either line-wise in interactive mode or with a single command from R or the command-line. This way comprehensive and reproducible analysis reports can be generated in PDF or HTML format in a fully automated manner by making use of the highly functional reporting utilities available for R. The following shows how to execute a workflow (e.g., systemPipeRNAseq.Rmd) from the command-line.\nRscript -e \"rmarkdown::render('systemPipeRNAseq.Rmd')\"  Templates for setting up custom project reports are provided as **.Rmd* files by the helper package *systemPipeRdata* and in the vignettes subdirectory of systemPipeR. The corresponding HTML of these report templates are available here: systemPipeRNAseq, systemPipeRIBOseq, systemPipeChIPseq and systemPipeVARseq. To work with *.Rnw* or *.Rmd* files efficiently, basic knowledge of Sweave or knitr and Latex or R Markdown v2 is required.\nRNA-Seq sample Load the RNA-Seq sample workflow into your current working directory.\nlibrary(systemPipeRdata) genWorkenvir(workflow = \"rnaseq\") setwd(\"rnaseq\")  Run workflow Next, run the chosen sample workflow systemPipeRNAseq (PDF, Rmd) by executing from the command-line make -B within the rnaseq directory. Alternatively, one can run the code from the provided *.Rmd template file from within R interactively.\nThe workflow includes following steps:\n Read preprocessing  Quality filtering (trimming) FASTQ quality report   Alignments: Tophat2 (or any other RNA-Seq aligner) Alignment stats Read counting Sample-wise correlation analysis Analysis of differentially expressed genes (DEGs) GO term enrichment analysis Gene-wise clustering  ChIP-Seq sample Load the ChIP-Seq sample workflow into your current working directory.\nlibrary(systemPipeRdata) genWorkenvir(workflow = \"chipseq\") setwd(\"chipseq\")  Run workflow Next, run the chosen sample workflow systemPipeChIPseq_single (PDF, Rmd) by executing from the command-line make -B within the chipseq directory. Alternatively, one can run the code from the provided *.Rmd template file from within R interactively.\nThe workflow includes the following steps:\n Read preprocessing  Quality filtering (trimming) FASTQ quality report   Alignments: Bowtie2 or rsubread Alignment stats Peak calling: MACS2, BayesPeak Peak annotation with genomic context Differential binding analysis GO term enrichment analysis Motif analysis  VAR-Seq sample VAR-Seq workflow for the single machine Load the VAR-Seq sample workflow into your current working directory.\nlibrary(systemPipeRdata) genWorkenvir(workflow = \"varseq\") setwd(\"varseq\")  Run workflow Next, run the chosen sample workflow systemPipeVARseq_single (PDF, Rmd) by executing from the command-line make -B within the varseq directory. Alternatively, one can run the code from the provided *.Rmd template file from within R interactively.\nThe workflow includes following steps:\n Read preprocessing  Quality filtering (trimming) FASTQ quality report   Alignments: gsnap, bwa Variant calling: VariantTools, GATK, BCFtools Variant filtering: VariantTools and VariantAnnotation Variant annotation: VariantAnnotation Combine results from many samples Summary statistics of samples  VAR-Seq workflow for computer cluster The workflow template provided for this step is called systemPipeVARseq.Rmd (PDF, Rmd). It runs the above VAR-Seq workflow in parallel on multiple compute nodes of an HPC system using Slurm as the scheduler.\nRibo-Seq sample Load the Ribo-Seq sample workflow into your current working directory.\nlibrary(systemPipeRdata) genWorkenvir(workflow = \"riboseq\") setwd(\"riboseq\")  Run workflow Next, run the chosen sample workflow systemPipeRIBOseq (PDF, Rmd) by executing from the command-line make -B within the ribseq directory. Alternatively, one can run the code from the provided *.Rmd template file from within R interactively.\nThe workflow includes following steps:\n Read preprocessing  Adaptor trimming and quality filtering FASTQ quality report   Alignments: Tophat2 (or any other RNA-Seq aligner) Alignment stats Compute read distribution across genomic features Adding custom features to the workflow (e.g. uORFs) Genomic read coverage along with transcripts Read counting Sample-wise correlation analysis Analysis of differentially expressed genes (DEGs) GO term enrichment analysis Gene-wise clustering Differential ribosome binding (translational efficiency)  Version information Note: the most recent version of this tutorial can be found here.\nsessionInfo()  ## R version 4.0.3 (2020-10-10) ## Platform: x86_64-pc-linux-gnu (64-bit) ## Running under: Ubuntu 20.04.2 LTS ## ## Matrix products: default ## BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 ## LAPACK: /home/dcassol/src/R-4.0.3/lib/libRlapack.so ## ## locale: ## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C ## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 ## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 ## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C ## [9] LC_ADDRESS=C LC_TELEPHONE=C ## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C ## ## attached base packages: ## [1] stats4 parallel stats graphics grDevices utils datasets ## [8] methods base ## ## other attached packages: ## [1] magrittr_2.0.1 batchtools_0.9.15 ## [3] ape_5.4-1 ggplot2_3.3.3 ## [5] systemPipeR_1.25.6 ShortRead_1.48.0 ## [7] GenomicAlignments_1.26.0 SummarizedExperiment_1.20.0 ## [9] Biobase_2.50.0 MatrixGenerics_1.2.1 ## [11] matrixStats_0.58.0 BiocParallel_1.24.1 ## [13] Rsamtools_2.6.0 Biostrings_2.58.0 ## [15] XVector_0.30.0 GenomicRanges_1.42.0 ## [17] GenomeInfoDb_1.26.2 IRanges_2.24.1 ## [19] S4Vectors_0.28.1 BiocGenerics_0.36.0 ## [21] BiocStyle_2.18.1 ## ## loaded via a namespace (and not attached): ## [1] colorspace_2.0-0 rjson_0.2.20 hwriter_1.3.2 ## [4] ellipsis_0.3.1 bit64_4.0.5 AnnotationDbi_1.52.0 ## [7] xml2_1.3.2 codetools_0.2-18 splines_4.0.3 ## [10] cachem_1.0.3 knitr_1.31 jsonlite_1.7.2 ## [13] annotate_1.68.0 GO.db_3.12.1 dbplyr_2.1.0 ## [16] png_0.1-7 pheatmap_1.0.12 graph_1.68.0 ## [19] BiocManager_1.30.10 compiler_4.0.3 httr_1.4.2 ## [22] GOstats_2.56.0 backports_1.2.1 assertthat_0.2.1 ## [25] Matrix_1.3-2 fastmap_1.1.0 limma_3.46.0 ## [28] formatR_1.7 htmltools_0.5.1.1 prettyunits_1.1.1 ## [31] tools_4.0.3 gtable_0.3.0 glue_1.4.2 ## [34] GenomeInfoDbData_1.2.4 Category_2.56.0 dplyr_1.0.4 ## [37] rsvg_2.1 rappdirs_0.3.3 V8_3.4.0 ## [40] Rcpp_1.0.6 vctrs_0.3.6 nlme_3.1-152 ## [43] blogdown_1.1.7 rtracklayer_1.50.0 xfun_0.21 ## [46] stringr_1.4.0 lifecycle_1.0.0.9000 XML_3.99-0.5 ## [49] edgeR_3.32.1 zlibbioc_1.36.0 scales_1.1.1 ## [52] BSgenome_1.58.0 VariantAnnotation_1.36.0 hms_1.0.0 ## [55] RBGL_1.66.0 RColorBrewer_1.1-2 yaml_2.2.1 ## [58] curl_4.3 memoise_2.0.0 biomaRt_2.46.3 ## [61] latticeExtra_0.6-29 stringi_1.5.3 RSQLite_2.2.3 ## [64] genefilter_1.72.1 checkmate_2.0.0 GenomicFeatures_1.42.1 ## [67] DOT_0.1 rlang_0.4.10 pkgconfig_2.0.3 ## [70] bitops_1.0-6 evaluate_0.14 lattice_0.20-41 ## [73] purrr_0.3.4 bit_4.0.4 tidyselect_1.1.0 ## [76] GSEABase_1.52.1 AnnotationForge_1.32.0 bookdown_0.21 ## [79] R6_2.5.0 generics_0.1.0 base64url_1.4 ## [82] DelayedArray_0.16.1 DBI_1.1.1 withr_2.4.1 ## [85] pillar_1.4.7 survival_3.2-7 RCurl_1.98-1.2 ## [88] tibble_3.0.6 crayon_1.4.1 BiocFileCache_1.14.0 ## [91] rmarkdown_2.6 jpeg_0.1-8.1 progress_1.2.2 ## [94] locfit_1.5-9.4 grid_4.0.3 data.table_1.13.6 ## [97] blob_1.2.1 Rgraphviz_2.34.0 digest_0.6.27 ## [100] xtable_1.8-4 brew_1.0-6 openssl_1.4.3 ## [103] munsell_0.5.0 askpass_1.1  References ","categories":"","description":"","excerpt":"document.addEventListener(\"DOMContentLoaded\", function() { …","ref":"/sp/spr/templates/","tags":"","title":"Workflow Templates"},{"body":"  .td-content li a { font-size: 1.5rem; }  systempipeR Functions Reference Manuals The following reference manual was created by systemPipeR Development version.\n systemPipeR Reference Manual   ","categories":"","description":"","excerpt":"  .td-content li a { font-size: 1.5rem; }  systempipeR Functions …","ref":"/sp/spr/spr_funcs/","tags":"","title":"Help Manual"},{"body":"  .td-content li a { font-size: 1.5rem; }  SPS Function Reference Manuals There are many more useful functions one could use from SPS core and sub packages. We could not cover all the details in this manual. To take a look at these functions, click links below to go to the function reference manual of each package. Detailed function usage, example code, and example running results are shown inside.\n   Package Description     systemPipeShiny main package   spsComps SPS components   drawer image editing tool   spsUtil utility functions     table {font-size: 1.5rem}   ","categories":"","description":"","excerpt":"  .td-content li a { font-size: 1.5rem; }  SPS Function Reference …","ref":"/sps/sps_funcs/","tags":"","title":"SPS Functions"},{"body":" The basic usage of SPS with these default modules/tabs has been discussed in previous sections. SPS provides much more than this. To fully use SPS’s framework functionalities, in this section, we will discuss how you can customize SPS and add new features. \n","categories":"","description":"","excerpt":" The basic usage of SPS with these default modules/tabs has been …","ref":"/sps/adv_features/","tags":"","title":"Advanced features"},{"body":"pre code { white-space: pre !important; overflow-x: scroll !important; word-break: keep-all !important; word-wrap: initial !important; }  document.addEventListener(\"DOMContentLoaded\", function() { document.querySelector(\"h1\").className = \"title\"; });  document.addEventListener(\"DOMContentLoaded\", function() { var links = document.links; for (var i = 0, linksLength = links.length; i  systemPipeR Installation To install the systemPipeR package (H Backman and Girke 2016), please use the BiocManager::install command:\nif (!requireNamespace(\"BiocManager\", quietly = TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"systemPipeR\")  To obtain the most recent updates immediately, one can install it directly from github as follow:\nif (!requireNamespace(\"BiocManager\", quietly = TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"tgirke/systemPipeR\", build_vignettes = TRUE, dependencies = TRUE)  Third-party software tools in SPR This guide provides detailed installation instructions for the software tools used within systemPipeR package. This guide provides the instruction for a Linux system.\nBWA Burrow-Wheeler Aligner (BWA) for short-read alignment (Li 2013).\ngit clone https://github.com/lh3/bwa.git cd bwa make vim ~/.bashrc ## Open the Bash shell script export PATH=$PATH:/path/to/bwa ## \"/path/to/bwa\" is an example! Replace with real PATH source ~/.bashrc bwa ## Test if the installation was successful  Note: In some systems, ~/.bash_profile is used in place of ~/.bashrc.\nMore information about how to set an enviroment variable here.\nBLAST Please check the BLAST Command Line Applications User Manual to find the installation introduction for your system environment.\nDownload the latest version from here, as the following example:\nwget https://ftp.ncbi.nlm.nih.gov/blast/executables/LATEST/ncbi-blast-2.11.0+-x64-linux.tar.gz tar zxvpf ncbi-blast-2.11.0+-x64-linux.tar.gz ncbi-blast-2.11.0+/ cd ncbi-blast-2.11.0+/ export PATH=$PATH:/path/to//ncbi-blast-2.11.0+/bin ## \"/path/to/ncbi-blast-2.11.0+\" is an example! Replace with real PATH  Cutadapt Please check the Cutadapt Manual to find the installation introduction for your system environment.\nsudo apt install cutadapt  References H Backman, Tyler W, and Thomas Girke. 2016. “systemPipeR: NGS workflow and report generation environment.” BMC Bioinformatics 17 (1): 388. https://doi.org/10.1186/s12859-016-1241-0.\n Li, Heng. 2013. “Aligning Sequence Reads, Clone Sequences and Assembly Contigs with BWA-MEM,” March. http://arxiv.org/abs/1303.3997.\n  ","categories":"","description":"","excerpt":"pre code { white-space: pre !important; overflow-x: scroll !important; …","ref":"/sp/spr/sprinstall/","tags":"","title":"SPR detailed installation instructions"},{"body":" This secetion discusses how different security concerns are handled in SPS.\nsuppressPackageStartupMessages(library(systemPipeShiny))  SPS mode SPS has an option of mode. This option controls how the file upload is handled by SPS. It can be either “server” or “local”, which is asking whether you are running the app on the “server” or running on your “local” machine.\n “server”: for security, users do not have access of file system on the server, choose files from users' local computer. “local”: Assumes the Shiny server and users' local computer is the same machine, so users can access file Shiny server’s file system.  It is first defined in the global.R file in a SPS project.\noptions(sps = list( mode = \"local\", ... ))  You can check current setting after the app started for the first time\nspsOption(\"mode\")  ## [1] \"local\"  The most affected function is dynamicFile and its server side function dynamicFileServer from the {spsComps} supporting package.\nServer:\nAfter clicking the file selection button in SPS, server mode uses the default shiny file choose, which users can choose from their local computer. You can see from the picture above, this is a user operating system built-in file chooser.\nLocal:\nYou can see this is different than the “server” mode. “Local” mode is able to use the file system of the shiny deploy environment file system, and it is no longer the users' local system.\nThis may be confusing for the first time. Remember “server” and “local” mean where you deploy the shiny app, on the server or run on your local computer.\n   Mode choose file from file pointing method     Server user local computer copy to temp   Local the computer where you deploy the app direct pointer    Pro and cons of modes It does not matter if which mode you choose if you run the app on your own computer, because the deploy server and the user computer are the same.\n However “local” mode will not copy a file to temp after the file chooser, but directly create a pointer. “Server” mode will first upload/copy the file to temp and create a pointer. This will cause resources waste if you are running the app on your own computer. You already have the file on your computer but now it gets copied to temp before Shiny can use it. This will also waste some time to copy the file, especially for large files. There is a limit for default Shiny upload size which is 24MB in “server” mode. You can choose files as large as you desire on “local” mode.  The security issue of local mode There is a security concern of “local” mode when the app is deployed on a remote server. “local” mode enables users to choose files from the remote server, so there is the risk of file leaking and file damaging.\nWe recommend DO NOT use “local” mode for remote deployment, like https://shinyapps.io. You can turn the option warning_toast = TRUE on global and testing the app before deploy. This option will check for security problems and inform you.\nThere are cases where you really need users to choose files from the remote server, like the Workflow module, where all workflow files are stored on the remote server. Then use a sandbox or container environment to isolate the app, and also turn on the login page login_screen = TRUE to limit access will be helpful.\nWarning toast Set warning_toast = TRUE option will check for potential security risks and show a pop-up message if there is any risk when app starts. This is option is helpful on pre-deployment testing.\n check if you have changed the default admin page url “admin” check if you have changed the default admin user “admin” check if you have changed the default user “admin” check if you are on “local” mode  The workflow module The workflow module enables users to manage, design, and run workflows directly from the app and in the final running workflow session, users are allowed to run arbitrary R code in a Rstudio like console in a child R session. Running any R code means they can modify your remote system, and use system commands.\nFor shinyapps.io, it runs in a container and it reset itself once a while, so security is not a big concern, but apparently, shinyapps.io is not a place you want to deploy heavy data analysis workflows. Most users will deploy the SPS with workflow modules in other cloud computing sites or their own servers. For these cases, we recommend you:\n  Turn on the login to give access to limited people.\n  Isolate the app with sandboxes or containers.\n  ","categories":"","description":"","excerpt":" This secetion discusses how different security concerns are handled …","ref":"/sps/adv_features/app_security/","tags":"","title":"App security"},{"body":" Guidelines from bioconductor_docker.\n Running the systemPipeR with Docker Get a copy of the public docker image docker pull systempipe/systempipe_docker:latest  To run RStudio Server: docker run -e PASSWORD=systemPipe -p 8787:8787 \\ systempipe/systempipe_docker:latest  You can then open a web browser pointing to your docker host on port 8787. If you’re on Linux and using default settings, the docker host is 127.0.0.1 (or localhost, so the full URL to RStudio would be http://localhost:8787). If you are on Mac or Windows and running Docker Toolbox, you can determine the docker host with the docker-machine ip default command.\nIn the above command, -e PASSWORD= is setting the RStudio password and is required by the RStudio Docker image. It can be whatever you like except it cannot be rstudio. Log in to RStudio with the username rstudio and whatever password was specified, in this example systemPipe.\nTo run R from the command line: docker run -it --user rstudio systempipe/systempipe_docker:latest R  To open a Bash shell on the container: docker run -it --user rstudio systempipe/systempipe_docker:latest bash   Install Prerequisites: Linux Mac Windows\nInstructions here on how to install Docker Engine on Ubuntu.\nsudo apt-get update sudo apt-get install \\ apt-transport-https \\ ca-certificates \\ curl \\ gnupg curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg echo \\ \"deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \\ $(lsb_release -cs) stable\" | sudo tee /etc/apt/sources.list.d/docker.list \u003e /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io ## Verify that Docker Engine is installed correctly by running the hello-world image. sudo docker run hello-world  Uninstall sudo apt-get remove docker docker-engine docker.io containerd runc   Docker Hub Account To be able to share a custom image, please go to https://hub.docker.com and create a free account.\n Log in to the Docker Hub locally Login with your Docker ID to push and pull images from Docker Hub. If you don’t have a Docker ID, head over to https://hub.docker.com to create one.\ndocker login # Username: XXXX # Password: xxx # Login Succeeded   Run Docker docker run-dP systempipe/systempipe_docker:latest  Make sure the container is running:\ndocker ps # CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS # NAMES #5d007f66a7b3 systempipe/systempipe_docker:latest \"/init\" 5 minutes ago Up 5 minutes 0.0.0.0:49153-\u003e8787/tcp determined_easle  Login to the container Please check the NAMES in this example, determined_easle, to login into the container.\ndocker exec -it determined_easle /bin/bash  Other alternatives to run the container To run RStudio Server: docker run -e PASSWORD=systemPipe -p 8787:8787 \\ systempipe/systempipe_docker:latest  To run R from the command line: docker run -it --user rstudio systempipe/systempipe_docker:latest R  To open a Bash shell on the container: docker run -it --user rstudio systempipe/systempipe_docker:latest bash  Check R Version into the container R --version  Stop Docker docker stop determined_easle   Create your first repository Link Create a repository:  Sign in to Docker Hub. Click Create a Repository on the Docker Hub welcome page: Name it /my-repo. Click Create.  Build and push a container image to Docker Hub from your computer Start by creating a Dockerfile to specify your application mkdir docker_test cd docker_test touch Dockerfile  # Docker inheritance FROM systempipe/systempipe_docker:latest ## Install BiocStyle RUN R -e 'BiocManager::install(\"BiocStyle\")' # Install required Bioconductor package from devel version RUN R -e 'BiocManager::install(\"tgirke/systemPipeR\")' RUN R -e 'BiocManager::install(\"tgirke/systemPipeRdata\")' WORKDIR /home/rstudio/SPRojects COPY --chown=rstudio:rstudio . /home/rstudio/SPRojects # Metadata LABEL name=\"systempipe/systempipe_docker\" \\ version=$BIOCONDUCTOR_DOCKER_systempipe \\ url=\"https://github.com/systemPipeR/systempipe/systempipe_docker\" \\ vendor=\"systemPipeR Project\" \\ maintainer=\"email@gmail.com\" \\ description=\"Bioconductor docker image containing the systemPipeR Project\" \\ license=\"Artistic-2.0\"  Run docker build to build your Docker image docker build -t systempipe/systempipe_docker .  Run docker run to test your Docker image locally docker run -e PASSWORD=systemPipe -p 8787:8787 systempipe/systempipe_docker:latest  Run docker push to push your Docker image to Docker Hub docker push systempipe/systempipe_docker   Your repository in Docker Hub should now display a new latest tag under Tags   Make changes to the container and Create the new image Create a folder, for example:\ndocker run -dP systempipe/systempipe_docker docker ps ## To check the NAME \u003clucid_grothendieck\u003e docker exec -it lucid_grothendieck /bin/bash root@33c758eb1626:/# R  setwd(\"home/rstudio/\") systemPipeRdata::genWorkenvir(\"rnaseq\")  exit docker commit -m \"Added rnaseq template\" -a \"Dani Cassol\" lucid_grothendieck dcassol/systempipeworkshop2021:rnaseq docker push systempipe/systempipe_docker:rnaseq  Run the new image:\ndocker run -e PASSWORD=systemPipe -p 8787:8787 systempipe/systempipe_docker:rnaseq   Commands List which docker machines are available locally {bash, eval=FALSE}docker images\nList running containers {bash, eval=FALSE}docker ps\nList all containers {bash, eval=FALSE}docker ps -a\nResume a stopped container {bash, eval=FALSE}docker start \u003cCONTAINER ID\u003e\nShell into a running container {bash, eval=FALSE}docker exec -it \u003cCONTAINER ID\u003e /bin/bash\nStop OR remove a cointainer {bash, eval=FALSE}docker stop \u003cCONTAINER ID\u003e {bash, eval=FALSE}docker rm \u003cCONTAINER ID\u003e\nRemove a image {bash, eval=FALSE}docker rmi dcassol/systempipeworkshop2021:rnaseq\n Docker and GitHub Actions  To create a new token, go to Docker Hub Settings  1.1. Account Settings » Security » New Access Token 1.2. Add Access Token Description » Create 1.3. Copy the Access Token » Copy and Close\nGo to the Repository at GitHub  2.1. Settings \u003e Secrets \u003e New repository secret 2.2. Create a new secret with the name DOCKER_HUB_USERNAME and your Docker ID as value 2.3. Click at Add secret 2.4. Create a new secret with the name DOCKER_HUB_ACCESS_TOKEN and your Personal Access Token (PAT) as value (generated in the previous step)\nSet up the GitHub Actions workflow  steps: - name: Checkout Repo uses: actions/checkout@v2 - name: Login to Docker Hub uses: docker/login-action@v1 with: username: ${{ secrets.DOCKER_HUB_USERNAME }} password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}   Common Problems ## Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.24/auth: dial unix /var/run/docker.sock: connect: permission denied  Solution:\nsudo chmod 666 /var/run/docker.sock   Singularity Container Please download the Docker image of systemPipe, as follow:\nsingularity pull docker://systempipe/systempipe_docker:latest  You can also use the build command to download pre-built images from Docker. Unlike pull, build will convert the image to the latest Singularity image format after downloading it.\nsingularity build systempipe_docker_latest.sif docker://systempipe/systempipe_docker:latest  To run the container:\nsingularity shell systempipe_docker_latest.sif   Resources  Docker Run: How to create images from an application Docker Hub Quickstart Configure GitHub Actions Singularity  ","categories":"","description":"","excerpt":" Guidelines from bioconductor_docker.\n Running the systemPipeR with …","ref":"/sp/spr/sp_docker/","tags":"","title":"SPR Docker container"},{"body":" How to deploy the application The easiest way to deploy SPS is shinyapps.io{blk}. To get started, it is required to create a shinyapps.io account. Please follow this tutorial{blk} with all the instructions to create an account. There are other ways to deploy, like AWS or Google Cloud with a custom Shiny server. There are a lot of tutorials to deploy the app other than shinyapps.io. We only use shinyapps.io as the example here to discuss how to deploy.\nGet started After you have created a SPS project, it is in a deploy-ready state, which means you can deploy it as soon as you create the project.\nWe have created some important helper code in deploy.R under the project root for you.\n Read the tutorial{blk} After you have the accounts and token set up according to the tutorial, open deploy.R.   If you have used Bioconductor packages (mostly likely yes, unless you do not use the default modules), run the options line in R console before deploy, which is this\noptions(repos = BiocManager::repositories())    If you want to use develop version of Bioconductor packages (not recommended), uncomment following lines and follow the instructions in the file.\n# repos \u003c- BiocManager::repositories() # devel \u003c- \"3.13\" # repos[length(repos) + 1] \u003c- paste0(\"https://bioconductor.org/packages/\", devel, \"/bioc\") # names(repos)[length(repos)] \u003c- \"BioC\" # options(repos = repos) # getOption(\"repos\")    If you wish shinyapps.io to install SPS modules, copy and uncomment following lines for the modules to global.R file below the line # load additional libraries that you want to use below.\n## Workflow module # requireNamespace(\"DOT\"); requireNamespace(\"networkD3\"); requireNamespace(\"pushbar\") # requireNamespace(\"readr\"); requireNamespace(\"rhandsontable\"); requireNamespace(\"shinyTree\") # requireNamespace(\"systemPipeR\"); requireNamespace(\"systemPipeRdata\"); requireNamespace(\"zip\") ## RNA-Seq module # requireNamespace(\"DESeq2\"); requireNamespace(\"Rtsne\"); requireNamespace(\"SummarizedExperiment\") # requireNamespace(\"UpSetR\"); requireNamespace(\"ape\"); requireNamespace(\"ggtree\") # requireNamespace(\"glmpca\"); requireNamespace(\"pheatmap\"); requireNamespace(\"systemPipeR\") ## Quick ggplot module # requireNamespace(\"esquisse\")    The last section in deploy.R is some helper code to set up the account and deploy the app programmatically. If you use the tutorial{blk}, there is no need to do this section.\n  Save your changes in the global.R.\n    Deploy in Rstudio We recommend you to deploy the app with Rstudio. If you have set up your shinyapps.io account, we are ready to go.\n  A  Open global.R in Rstudio, use the “Run App” button to make sure your app can be run without problems locally. Run options(repos = BiocManager::repositories()) and other Bioconductor set up code as listed above. Click the button next to “Run App” and choose “Publish Application…”  B If you have set up your account correctly, you should see your account to the right panel. Select all the files of your SPS project. Change the app “Title” to what you want. hit “Publish”.  RAM requirement If you wish to load the workflow and/or RNAseq module(s) on shinyapps.io, you need at least 2GB of memory. Unfortunately, these 2 modules depend heavily on Bioconductor packages and these packages are not light-weighted. The max RAM for free account on shinyapps.io is 1GB which means you either need to upgrade your account or use other server to deploy your app.\nIf you decide to not load these modules which will allow you to deploy the app with a free account, change the settings to unload these modules in app options in global.R.\n","categories":"","description":"","excerpt":" How to deploy the application The easiest way to deploy SPS is …","ref":"/sps/deploy/","tags":"","title":"Deploy"},{"body":"SPS is not only a framework to run interactive workflow and visualize data, but also a framework with abundant developer tools for building Shiny apps, writing R markdowns, integrating R-HTML and general R utilities.\nThese developer toolkits are distributed in supporting packages. If you like some of the functionality from SPS and think installing the whole framework is too expensive (time consuming), cherry-pick the functionality you want in following categories:\n spsCpmps: Shiny/Rmarkdown UI components, and Shiny server functions. drawer: Shiny/Rmarkdown/HTML interactive image editing tool. spsUtil: General R utilities, like pretty logging functions.  Install All these supporting packages are available on CRAN. We recommend to use the modern package manager {pak} to install packages.\nif(!requireNamespace(\"pak\", quietly = TRUE)) install.packages(\"pak\")  Pick and install packages as your need\npak::pkg_install(\"spsCpmps\") pak::pkg_install(\"drawer\") pak::pkg_install(\"spsUtil\")  Linux If you are on Linux, you need to install some system dependencies before R package installation. To figure out what system dependencies and command, run:\npaste(\"sudo\", pak::pkg_system_requirements(\"spsComps\"))  ## [1] \"sudo apt-get install -y libcurl4-openssl-dev\" ## [2] \"sudo apt-get install -y libssl-dev\" ## [3] \"sudo apt-get install -y make\" ## [4] \"sudo apt-get install -y libicu-dev\" ## [5] \"sudo apt-get install -y pandoc\"  paste(\"sudo\", pak::pkg_system_requirements(\"drawer\"))  ## [1] \"sudo apt-get install -y make\" \"sudo apt-get install -y libicu-dev\" ## [3] \"sudo apt-get install -y pandoc\"  paste(\"sudo\", pak::pkg_system_requirements(\"spsUtil\"))  ## [1] \"sudo apt-get install -y libcurl4-openssl-dev\" ## [2] \"sudo apt-get install -y libssl-dev\" ## [3] \"sudo apt-get install -y libicu-dev\"  We are rendering this doc in Ubuntu. If your Linux distribution is different, commands above will be different.\nFunctions reference manual In documents, we only highlight some important functions. Please read the reference manuals for details of every function.\n","categories":"","description":"","excerpt":"SPS is not only a framework to run interactive workflow and visualize …","ref":"/sps/dev/","tags":"","title":"Developer tools"},{"body":" SPS database is a small SQLite database which stores some basic app information, the project encryption key pair, account information. The database is controlled by 3 SPS R6 classes:\n spsDb: basic database management, queries. spsEncryption: Interact with the SHA256 key pair in the database to encrypt strings, and files. Also methods to view or change the key pair, inherits spsDb spsAcount: use the encyption key pair to manage users and admin accounts in SPS, inherits spsDb and spsEncryption.  First to create a SPS project\nsuppressPackageStartupMessages(library(systemPipeShiny))  app_path \u003c- \".\" spsInit(app_path = app_path, overwrite = TRUE, open_files = FALSE)  ## [SPS-DANGER] 2021-04-22 15:45:04 Done, Db created at '/tmp/Rtmp3KQ3pG/SPS_20210422/config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-22 15:45:04 Key md5 1d1f76c8eecddb382ee2db097d6edbe9 ## [SPS-INFO] 2021-04-22 15:45:04 SPS project setup done!  You can see a database created on a SPS project initiation, you should see a message like this:\n[SPS-DANGER] 2021-04-19 11:06:53 Done, Db created at  Then we can use different class methods to interact with the database.\nspsDb class Reference manual under SPS Functions.\nCreate a class object:\nmydb \u003c- spsDb$new()  ## [SPS-INFO] 2021-04-22 15:45:04 Created SPS database method container  create new db If there is no database, we can create one:\n# delete current one first try(file.remove(\"config/sps.db\")) ## [1] TRUE # create a new one mydb$createDb() ## [SPS-INFO] 2021-04-22 15:45:04 Creating SPS db... ## [SPS-DANGER] 2021-04-22 15:45:04 Done, Db created at 'config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-22 15:45:04 Key md5 6306a50abcf3c7618566911a793b0405  If you create a new database, all information in the old database will be overwritten. All old information will be lost\nGet tables # meta info table mydb$queryValue(\"sps_meta\")  ## info value ## 1 creation_date 20210422154504  # raw blob table to store keys mydb$queryValue(\"sps_raw\")  ## info value ## 1 key blob[1.36 kB]  # account table mydb$queryValue(\"sps_account\")  ## account pass ## 1 admin bef2094c429fa5b0ff7a37abc6bf8dd31c29fa2acd783a2303935d2b8664ea21 ## 2 user 12842a62c410af0f4b4dc866283a90f7b7c42c9d5ca0d0e1c812dca11021b052 ## role ## 1 admin ## 2 user  Insert new records(rows) To add a new row, values of all columns needs to be passed in a SQL string\nmydb$queryInsert(\"sps_meta\", value = \"'new1', '1'\")  ## [SPS-INFO] 2021-04-22 15:45:04 Inerted 1 rows  Or pass in a vector:\nmydb$queryInsert(\"sps_meta\", value = c(\"'new2'\", \"'2'\"))  ## [SPS-INFO] 2021-04-22 15:45:04 Inerted 1 rows  # check the new table mydb$queryValue(\"sps_meta\")  ## info value ## 1 creation_date 20210422154504 ## 2 new1 1 ## 3 new2 2  change values mydb$queryUpdate(\"sps_meta\", value = '234', col = \"value\", WHERE = \"info = 'new1'\")  ## [SPS-INFO] 2021-04-22 15:45:04 Updated 1 rows  # check the update mydb$queryValue(\"sps_meta\")  ## info value ## 1 creation_date 20210422154504 ## 2 new1 234 ## 3 new2 2  remove values mydb$queryDel(\"sps_meta\", WHERE = \"value = '234'\")  ## [SPS-INFO] 2021-04-22 15:45:05 Deleted 1 rows  # check again mydb$queryValue(\"sps_meta\")  ## info value ## 1 creation_date 20210422154504 ## 2 new2 2  spsEncryption class Reference manual under SPS Functions.\nStart by creating a class object\nmy_ecpt \u003c- spsEncryption$new() ## [SPS-INFO] 2021-04-22 15:45:05 Created SPS encryption method container ## [SPS-INFO] 2021-04-22 15:45:05 Default SPS-db found and is working  Get current key To see the public and private keys (in {openssil{blk}} format):\n# private my_ecpt$keyGet()  ## [2048-bit rsa private key] ## md5: 6306a50abcf3c7618566911a793b0405  # public my_ecpt$keyGet()$pubkey  ## [2048-bit rsa public key] ## md5: 6306a50abcf3c7618566911a793b0405  Change the encyption key Be super careful to change the encryption key. This will result any file encrypted by the old key pair unlockable and the password of all current accounts invalid.\nBy default it will prevent you to change the key in case you accidentally run this method\nmy_ecpt$keyChange()  ## [SPS-DANGER] 2021-04-22 15:45:05 ## change this key will result all accounts' password failed to ## authenticate. You have to regenerate all password for all ## accounts. All encrypted file using the old key will fail to ## decrypt. There is NO way to RECOVER the old key, password ## and files. If you wish to continue, recall this function ## with `confirm = TRUE`.  Unless you are super sure with a confirmation\nmy_ecpt$keyChange(confirm = TRUE)  ## [SPS-INFO] 2021-04-22 15:45:05 md5 42e4b167ba388a9e797608eb4e466a18  Encrypt files # imagine a file has one line \"test\" writeLines(text = \"test\", con = \"test.txt\") my_ecpt$encrypt(\"test.txt\", \"test.bin\", overwrite = TRUE)  Decrypt files my_ecpt$decrypt(\"test.bin\", \"test_decpt.txt\", overwrite = TRUE)  Check the decrypted file content\nreadLines('test_decpt.txt')  ## [1] \"test\"  spsAcount class This class is discussed in details in the Accounts, login and admin.\n","categories":"","description":"","excerpt":" SPS database is a small SQLite database which stores some basic app …","ref":"/sps/adv_features/database/","tags":"","title":"SPS database"},{"body":" SPS accounts After a SPS project is initialized, some default accounts has been set up. They will not become useful unless you enable the login and admin features.\nThere are 2 default accounts created for you to test before deploy:\n   name role password     admin admin admin   user user user    The admin account can be used in both the main app login and admin page login, and the user account can only be used for main app login.\nPlease change these accounts before deployment.\nAccount management You can manage accounts in two ways: commandline and admin page. Here is how you manage from commandline, read how to do it from admin page below.\nThere are some helper code inside global.R file under your SPS project root.\n## account information ## PLEASE use following to add your own accounts and remove the default accounts for deployment # mydb \u003c- spsAccount$new() # mydb$accList() # mydb$accAdd(acc_name = \"XXX\", acc_pass = \"$xxxx\", role = \"admin\") # mydb$accRemove(\"admin\") # mydb$accRemove(\"user\")  You can uncomment these lines to practice how to manage accounts, or read the reference manual about spsAccount class.\nFirst to create a SPS project\nsuppressPackageStartupMessages(library(systemPipeShiny))  app_path \u003c- \".\" spsInit(app_path = app_path, overwrite = TRUE, open_files = FALSE)  ## [SPS-INFO] 2021-04-16 16:25:15 Start to create a new SPS project ## [SPS-INFO] 2021-04-16 16:25:15 Create project under /tmp/RtmpMoNp2X/SPS_20210416 ## [SPS-INFO] 2021-04-16 16:25:15 Now copy files ## [SPS-INFO] 2021-04-16 16:25:15 Create SPS database ## [SPS-INFO] 2021-04-16 16:25:15 Created SPS database method container ## [SPS-INFO] 2021-04-16 16:25:15 Creating SPS db... ## [SPS-DANGER] 2021-04-16 16:25:15 Done, Db created at '/tmp/RtmpMoNp2X/SPS_20210416/config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-16 16:25:15 Key md5 db686a4f203edbdd0fe4c2a1642492d2 ## [SPS-INFO] 2021-04-16 16:25:15 SPS project setup done!  Create a spsAccount object. SPS accounts are stored in the config/sps.db database inside your SPS projects.\nacc \u003c- spsAccount$new() ## [SPS-INFO] 2021-04-16 16:25:15 Created SPS account method container ## [SPS-INFO] 2021-04-16 16:25:15 Default SPS-db found and is working  ## [SPS-INFO] 2021-04-16 16:25:15 Creating SPS db...  ## [SPS-DANGER] 2021-04-16 16:25:15 Done, Db created at 'config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-16 16:25:15 Key md5 127f00770c4ed318c48fe92cd67bf783  See what accounts you have\nacc$accList()  ## account role ## 1 admin admin ## 2 user user  You can also listed passwords. They are SHA256 encrypted.\ntibble::as_tibble(acc$accList(include_pass = TRUE)) ## # A tibble: 2 x 3 ## account pass role ## \u003cchr\u003e \u003cchr\u003e \u003cchr\u003e ## 1 admin 0b1460498bf1b50e05cc07e53c7856fc23ecd3fc2bc404d52dbf3aae743f8a9a admin ## 2 user 796be5ca7dafddf5e1e7c032fc71386fe16274f63b6068bb2b064d4c9876eaae user  See SPS database for encryption key details.\nAdd a new user\nacc$accAdd(acc_name = 'user2', acc_pass = '!newuser12345', role = \"user\") ## [SPS-INFO] 2021-04-16 16:25:15 Account user2 created.  Change the role of user2 from “user” to “admin”\nacc$accRoleChange(acc_name = \"user2\", role = \"admin\") ## [SPS-INFO] 2021-04-16 16:25:15 Updated 1 rows ## [SPS-INFO] 2021-04-16 16:25:15 Account user2 role changed.  Remove a user\nacc$accRemove(\"user2\") ## [SPS-INFO] 2021-04-16 16:25:15 Deleted 1 rows ## [SPS-INFO] 2021-04-16 16:25:15 Account user2 removed acc$accList() ## account role ## 1 admin admin ## 2 user user  Change password\nacc$accPassChange(acc_name = \"user\", , acc_pass = '!newuser54321') ## [SPS-INFO] 2021-04-16 16:25:15 Updated 1 rows ## [SPS-INFO] 2021-04-16 16:25:15 Account user password created.  Validate password\nacc$accMatch(acc_name = \"user\", acc_pass = '!newuser54321')  ## [1] TRUE  acc$accMatch(acc_name = \"user\", acc_pass = \"user\")  ## [1] FALSE  acc$accMatch(acc_name = \"abc\", acc_pass = \"123\")  ## [1] FALSE  Validate password + role\nacc$accMatch(acc_name = \"user\", acc_pass = '!newuser54321', match_role = TRUE, role = \"user\")  ## [1] TRUE  acc$accMatch(acc_name = \"user\", acc_pass = '!newuser54321', match_role = TRUE, role = \"admin\")  ## [1] FALSE  Main app login After the account has been set up properly, one can try to turn on the login page for the main app. Use spsOption(\"login_screen\", TRUE) or set login_screen = TRUE in global.R file.\nWhether enabling the login screen is fundamentally different how the Shiny app loads the UI and server code.\n Disabled: loads UI and server on app start Enabled: loads login UI and server on start, loads main app UI and server code only when login is successful.  One advantage of using the login is the app starting time is fast. On app start, it only loads the login logic so it saves some time. The heavy part is the main app logic which will be loaded after a successful login. So the overall loading time is about the same. The difference is at what time point to load main app. Of course, if the user fails to login, main app will never be loaded. This can save some resources to handle unauthorized requests.\nThis difference may cause some javascript not working if you are loading your custom tabs with custom javascript. Set a wait signal in your javascript or report an issue to us if you have troubles.\n The login screen is also controlled by another setting login_theme. By default, if you turn on the “login_screen”, app will show you a random loading theme (login_theme = \"random\") before you see the login panel. You can interact with these themes or change to play with a different theme. Or you can specify your favorate from “vhelix” (DNA double helix), “hhelix” (DNA flow), or “biomatrix” (DNA Matrix).\n If you are not a fan of these themes, you can use login_theme = \"empty\" to directly go to login panel.\n  Admin page SPS Admin page is panel of tabs to help app managers administrate the app. Under current version, SPS provides 2 main features: app information/statistics and account control.\nAdmin login To reach the Admin page, users first need to enable this feature (default is TRUE) in SPS options admin_page use either spsOption(\"admin_page\", TRUE) or set it in global.R. Afterwards, users need type in the correct url to find the page. This can be set with the SPS option admin_url. Default is “admin”, admin_url = \"admin\", but for security we recommend you to change it in deployment. \nTo access it, add “?” + “YOUR_ADMIN_URL” to your app url to visit it. For example:\nwe have a demo https://tgirke.shinyapps.io/systemPipeShiny_loading/\nto visit the admin page, then we visit https://tgirke.shinyapps.io/systemPipeShiny_loading/?admin\n Admin login page\n You can use the testing account “admin” and password “admin” to login, but for security we strongly recommend you to change it in deployment. \nApp information The first tab of the Admin page is current app and server information, like CPU, RAM, size, etc. On this tab, under details, some real-time statistic plots are rendered. You can interact with these plots to dig for more information.\n Admin page app info\n Account control Instead of changing account information from commandline, you can use this tab to add / remove / change password/ change roles of current app accounts.\n Admin page user control\n  Create a new user\n ","categories":"","description":"","excerpt":" SPS accounts After a SPS project is initialized, some default …","ref":"/sps/adv_features/login/","tags":"","title":"Accounts, Login and Admin"},{"body":" There are some options in SPS that will give you more information and help you on debugging. They are: verbose and traceback. You can config (enable/disable) themin a SPS project’s global.R file, or use spsOption(\"verbose\", TRUE) and spsOption(\"traceback\", TRUE) to turn on them.\nSome setup:\nsuppressPackageStartupMessages(library(systemPipeShiny)) app_dir \u003c- tempdir() spsInit(app_path = app_dir, overwrite = TRUE, change_wd = FALSE, open_files = FALSE) ## [SPS-INFO] 2021-04-16 17:46:53 Start to create a new SPS project ## [SPS-INFO] 2021-04-16 17:46:53 Create project under /tmp/RtmpUfKCYR/SPS_20210416 ## [SPS-INFO] 2021-04-16 17:46:53 Now copy files ## [SPS-INFO] 2021-04-16 17:46:53 Create SPS database ## [SPS-INFO] 2021-04-16 17:46:53 Created SPS database method container ## [SPS-INFO] 2021-04-16 17:46:53 Creating SPS db... ## [SPS-DANGER] 2021-04-16 17:46:53 Done, Db created at '/tmp/RtmpUfKCYR/SPS_20210416/config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-16 17:46:53 Key md5 fc8c85a0e87073328864bd542d740801 ## [SPS-INFO] 2021-04-16 17:46:53 SPS project setup done! app_path \u003c- file.path(app_dir, glue::glue(\"SPS_{format(Sys.time(), '%Y%m%d')}\"))  verbose In many SPS functions, there is this argument verbose and usually default is FALSE. It means do not print extra message, keep it clean. You can set in spsOption(\"verbose\", TRUE) or inside global.R file to turn on. These are called global settings, and you can use a local setting to overwrite it (func(..., verbose = TRUE)).\nLet’s use SPS main function sps for example, without the verbose\nspsOption(\"verbose\", FALSE) app \u003c- sps(app_path = app_path) ## Warning: [SPS-WARNING] 2021-04-16 17:46:53 These plot tabs has no image path: ## 'vs_example' ## It is recommended to add an image. It will be used to generate gallery. Now an empty image is used for these tabs' gallery. ## [SPS-INFO] 2021-04-16 17:46:55 App starts ...  Turn on the verbose:\nspsOption(\"verbose\", TRUE) app \u003c- sps(app_path = app_path) ## [SPS-INFO] 2021-04-16 17:46:55 App has 19 default configs, resolving 19 custom configs ## [SPS-INFO] 2021-04-16 17:46:55 Now check the tab info in tabs.csv ## Warning: [SPS-WARNING] 2021-04-16 17:46:55 These plot tabs has no image path: ## 'vs_example' ## It is recommended to add an image. It will be used to generate gallery. Now an empty image is used for these tabs' gallery. ## [SPS-INFO] 2021-04-16 17:46:55 tab.csv info check pass ## [SPS-INFO] 2021-04-16 17:46:55 Using default tabs ## [SPS-INFO] 2021-04-16 17:46:55 check guide ## [SPS-INFO] 2021-04-16 17:46:55 Start to generate UI ## [SPS-INFO] 2021-04-16 17:46:55 parse title and logo ## [SPS-INFO] 2021-04-16 17:46:55 resolve default tabs UI ## [SPS-INFO] 2021-04-16 17:46:55 Loading custom tab UI ... ## [SPS-INFO] 2021-04-16 17:46:55 Loading notifications from developer... ## [SPS-INFO] 2021-04-16 17:46:55 Loading guide UI ## [SPS-INFO] 2021-04-16 17:46:55 Create UI header ... ## [SPS-INFO] 2021-04-16 17:46:55 Create UI sidebar menu ... ## [SPS-INFO] 2021-04-16 17:46:55 Create UI tab content ... ## [SPS-INFO] 2021-04-16 17:46:55 Add tab content to body ... ## [SPS-INFO] 2021-04-16 17:46:55 Merge header, menu, body to dashboard ... ## [SPS-INFO] 2021-04-16 17:46:55 Add overlay loading screen, admin panel. ## Merge everything to app container ... ## [SPS-INFO] 2021-04-16 17:46:55 UI created ## [SPS-INFO] 2021-04-16 17:46:55 Start to create server function ## [SPS-INFO] 2021-04-16 17:46:55 Resolve default tabs server ## [SPS-INFO] 2021-04-16 17:46:55 Load custom tabs servers ## [SPS-INFO] 2021-04-16 17:46:55 Server functions created ## [SPS-INFO] 2021-04-16 17:46:55 App starts ...  Exception There is one exception which is the spsInit. It is used to create a SPS project for you, so it assumes you do not have a SPS project yet and therefore do not have the chance to reach SPS options. So the verbose global setting will not work here. You need to turn it on locally with verbose = TRUE.\nCompare messages of this with the initial spsInit creation on top.\nspsInit(verbose = TRUE, app_path = app_path, overwrite = TRUE, change_wd = FALSE, open_files = FALSE) ## [SPS-INFO] 2021-04-16 17:46:55 Start to create a new SPS project ## [SPS-INFO] 2021-04-16 17:46:55 Create project under /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416 ## [SPS-INFO] 2021-04-16 17:46:55 Now copy files ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/www ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/config ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/R ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/data ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/results ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/README.md ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/deploy.R ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/server.R ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/global.R ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/ui.R ## [SPS-INFO] 2021-04-16 17:46:55 File(s) copied for /tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/server.R ## [SPS-INFO] 2021-04-16 17:46:55 Create SPS database ## [SPS-INFO] 2021-04-16 17:46:55 Created SPS database method container ## [SPS-INFO] 2021-04-16 17:46:55 Db connected ## [SPS-INFO] 2021-04-16 17:46:55 Default SPS-db found and is working ## [SPS-INFO] 2021-04-16 17:46:55 Db connected ## [SPS-INFO] 2021-04-16 17:46:55 Creating SPS db... ## [SPS-INFO] 2021-04-16 17:46:55 Db write the meta table ## [SPS-INFO] 2021-04-16 17:46:56 Db write the raw table ## [SPS-INFO] 2021-04-16 17:46:56 Key generated and stored in db ## [SPS-INFO] 2021-04-16 17:46:56 Db create admin account ## [SPS-DANGER] 2021-04-16 17:46:56 Done, Db created at '/tmp/RtmpUfKCYR/SPS_20210416/SPS_20210416/config/sps.db'. DO NOT share this file with others or upload to open access domains. ## [SPS-INFO] 2021-04-16 17:46:56 Key md5 7dca8a45de4d7260f91ddb1a02d0a6bd ## [SPS-INFO] 2021-04-16 17:46:56 SPS project setup done!  traceback When error happens, it will be helpful if we can know where it happened. This option will give you additional information of which function it happened, the system call list and error file and line of code if possible.\nThis feature is enabled in two functions sps and shinyCatch.\n sps: Adding tracebacks if there are some errors sourcing helper functions located in your SPS project under the R folder. shinyCatch: Traceback errors of expressions inside shinyCatch  Let’s use shinyCatch to demo.\nBefore adding traceback:\nspsOption(\"traceback\", FALSE) shinyCatch({ stop(\"some error message\") })  ## [SPS-ERROR] 2021-04-16 17:46:56 some error message  ## NULL  After\nspsOption(\"traceback\", TRUE) shinyCatch({ stop(\"some error message\") })  ## 1. local({ ## if (length(a \u003c- commandArgs(TRUE)) != 2) ## stop(\"The number of arguments passed to Rscript should be 2.\") ## x = readRDS(a[1]) ## f = x[[1]] ## if (is.character(f)) ## f = eval(parse(text = f), envir = globalenv()) ## r = do.call(f, x[[2]], envir = globalenv()) ## saveRDS(r, a[2]) ## }) ## 2. eval.parent(substitute(eval(quote(expr), envir))) ## 3. eval(expr, p) ## 4. eval(expr, p) ## 5. eval(quote({ ## if (length(a \u003c- commandArgs(TRUE)) != 2) stop(\"The number of arguments passed to Rscript should be 2.\") ## x = readRDS(a[1]) ## f = x[[1]] ## if (is.character(f)) f = eval(parse(text = f), envir = globalenv()) ## r = do.call(f, x[[2]], envir = globalenv()) ## saveRDS(r, a[2]) ## }), new.env()) ## 6. eval(quote({ ## if (length(a \u003c- commandArgs(TRUE)) != 2) stop(\"The number of arguments passed to Rscript should be 2.\") ## x = readRDS(a[1]) ## f = x[[1]] ## if (is.character(f)) f = eval(parse(text = f), envir = globalenv()) ## r = do.call(f, x[[2]], envir = globalenv()) ## saveRDS(r, a[2]) ## }), new.env()) ## 7. do.call(f, x[[2]], envir = globalenv()) ## 8. (function (input, output, to_md = file_ext(output) != \"html\", quiet = TRUE) ## { ## options(htmltools.dir.version = FALSE) ## setwd(dirname(input)) ## input = basename(input) ## if (to_md) ## options(bookdown.output.markdown = TRUE) ## res = rmarkdown::render(input, \"blogdown::html_page\", output_file = output, envir = globalenv(), quiet = quiet, run_pandoc = !to_md, clean = !to_md) ## x = read_utf8(res) ## if (to_md) ## x = process_markdown(res, x) ## unlink(res) ## x ## })(\"content/en/sps/adv_features/debug.Rmd\", \"debug.md~\", TRUE, TRUE) ## 9. rmarkdown::render(input, \"blogdown::html_page\", output_file = output, envir = globalenv(), quiet = quiet, run_pandoc = !to_md, clean = !to_md) ## 10. knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet) ## 11. process_file(text, output) ## 12. withCallingHandlers(if (tangle) process_tangle(group) else process_group(group), error = function(e) { ## setwd(wd) ## cat(res, sep = \"\\n\", file = output %n% \"\") ## message(\"Quitting from lines \", paste(current_lines(i), collapse = \"-\"), \" (\", knit_concord$get(\"infile\"), \") \") ## }) ## 13. process_group(group) ## 14. process_group.block(group) ## 15. call_block(x) ## 16. block_exec(params) ## 17. in_dir(input_dir(), evaluate(code, envir = env, new_device = FALSE, keep_warning = !isFALSE(options$warning), keep_message = !isFALSE(options$message), stop_on_error = if (options$error \u0026\u0026 options$include) 0 else 2, output_handler = knit_handlers(options$render, options))) ## 18. evaluate(code, envir = env, new_device = FALSE, keep_warning = !isFALSE(options$warning), keep_message = !isFALSE(options$message), stop_on_error = if (options$error \u0026\u0026 options$include) 0 else 2, output_handler = knit_handlers(options$render, options)) ## 19. evaluate::evaluate(...) ## 20. evaluate_call(expr, parsed$src[[i]], envir = envir, enclos = enclos, debug = debug, last = i == length(out), use_try = stop_on_error != 2, keep_warning = keep_warning, keep_message = keep_message, output_handler = output_handler, include_timing = include_timing) ## 21. timing_fn(handle(ev \u003c- withCallingHandlers(withVisible(eval(expr, envir, enclos)), warning = wHandler, error = eHandler, message = mHandler))) ## 22. handle(ev \u003c- withCallingHandlers(withVisible(eval(expr, envir, enclos)), warning = wHandler, error = eHandler, message = mHandler)) ## 23. withCallingHandlers(withVisible(eval(expr, envir, enclos)), warning = wHandler, error = eHandler, message = mHandler) ## 24. withVisible(eval(expr, envir, enclos)) ## 25. eval(expr, envir, enclos) ## 26. eval(expr, envir, enclos) ## 27. shinyCatch({ ## stop(\"some error message\") ## }) ## 28. tryCatch(suppressMessages(suppressWarnings(withCallingHandlers(expr, message = function(m) toastr_actions$message(m), warning = function(m) toastr_actions$warning(m), error = function(m) if (trace_back) printTraceback(sys.calls())))), error = function(m) { ## toastr_actions$error(m) ## return(NULL) ## }) ## 29. tryCatchList(expr, classes, parentenv, handlers) ## 30. tryCatchOne(expr, names, parentenv, handlers[[1]]) ## 31. doTryCatch(return(expr), name, parentenv, handler) ## 32. suppressMessages(suppressWarnings(withCallingHandlers(expr, message = function(m) toastr_actions$message(m), warning = function(m) toastr_actions$warning(m), error = function(m) if (trace_back) printTraceback(sys.calls())))) ## 33. withCallingHandlers(expr, message = function(c) if (inherits(c, classes)) tryInvokeRestart(\"muffleMessage\")) ## 34. suppressWarnings(withCallingHandlers(expr, message = function(m) toastr_actions$message(m), warning = function(m) toastr_actions$warning(m), error = function(m) if (trace_back) printTraceback(sys.calls()))) ## 35. withCallingHandlers(expr, warning = function(w) if (inherits(w, classes)) tryInvokeRestart(\"muffleWarning\")) ## 36. withCallingHandlers(expr, message = function(m) toastr_actions$message(m), warning = function(m) toastr_actions$warning(m), error = function(m) if (trace_back) printTraceback(sys.calls())) ## [SPS-ERROR] 2021-04-16 17:46:56 some error message  ## NULL  Or use local setting to overwrite the global, even we have spsOption(\"traceback\", TRUE), but traceback is still muted by  trace_back = FALSE.\nspsOption(\"traceback\", TRUE) shinyCatch({ stop(\"some error message\") }, trace_back = FALSE)  ## [SPS-ERROR] 2021-04-16 17:46:56 some error message  ## NULL  Traceback with file and line number Let’s write an R file with functions, source it and then call the function from this file. Try it on your own computer:\ntemp_file \u003c- tempfile(fileext = \".R\") writeLines( \"myFunc \u003c- function(){ myFunc2() } myFunc2 \u003c- function(){ stop('some error message') } \", temp_file ) source(temp_file) shinyCatch({ myFunc() })  You can see the error happened in myFunc line No. 2 and then inside this function it calls another function myFunc2 which caused the final error. In myFunc2 it is also the line No. 2 caused the issue and error is coming from /tmp/... file.\nother Shiny built-in options There are some Shiny options can also be helpful on debugging:\n# developer mode, use ?devmode to see details devmode(TRUE) # inspect reactivity in shiny options(shiny.reactlog = TRUE) # similar to SPS's traceback but on the whole app level options(shiny.fullstacktrace = TRUE) # open the `browser` debug mode on error options(shiny.error = browser) # when a shiny app file saves, reload the app, not working with modular apps like SPS at this moment options(shiny.autoreload = TRUE)  See Shiny option website{blk} for more details\n","categories":"","description":"","excerpt":" There are some options in SPS that will give you more information and …","ref":"/sps/adv_features/debug/","tags":"","title":"Debugging"},{"body":" Title and logo The default for title is “systemPipeShiny” and default logo is the “img/sps_small.png” which is a relative path of the www folder in your SPS project. You can change them in SPS options. The affected places are:\n","categories":"","description":"","excerpt":" Title and logo The default for title is “systemPipeShiny” and default …","ref":"/sps/adv_features/other_customizations/","tags":"","title":"Other customizations"},{"body":" Some screenshots of SPS Full app Loading screens Workflow module Workflow Execution RNASeq module Canvas Canvas Admin Debugging ","categories":"","description":"","excerpt":" Some screenshots of SPS Full app Loading screens Workflow module …","ref":"/sps/screenshots/","tags":"","title":"Screenshots"},{"body":"layout: true background-image: url(https://raw.githubusercontent.com/systemPipeR/systemPipeShiny/master/inst/app/www/img/sps_small.png) background-position: 100% 0% background-size: 10%\n","categories":"","description":"","excerpt":"layout: true background-image: …","ref":"/presentations/sps/sps_intro/","tags":"","title":"![](https://raw.githubusercontent.com/systemPipeR/systemPipeShiny/master/inst/app/www/img/sps_small.png){width=1in}\n\u003cbr/\u003e_systemPipeShiny_\n"},{"body":" layout: true background-image: url(https://raw.githubusercontent.com/tgirke/systemPipeR/gh-pages/images/systemPipeR.png) background-position: 99% 1% background-size: 10%\n class: middle\nOutline Introduction Design How to run a Workflow Workflows Tutorial Live Demo  class: inverse, center, middle\n Introduction  Introduction  systemPipeR provides a suite of R/Bioconductor packages for designing, building and running end-to-end analysis workflows on local machines, HPC clusters and cloud systems, while generating at the same time publication quality analysis reports\n systemPipeR offers many utilities to build, control, and execute workflows entirely from R\n The environment takes advantage of central community S4 classes of the Bioconductor ecosystem\n Workflows are managed by generic workflow management containers supporting both analysis routines implemented in R code and/or command-line software\n Simple annotation system targets\n systemPipeR’s Core Functionalities .center[ ]\n  Structural Features – .left-column[\nWF infrastructure ]\n.right-column[ systemPipeR offers many utilities to build, control, and execute workflows entirely from R. The environment takes advantage of central community S4 classes of the Bioconductor ecosystem. Workflows are managed by generic workflow management containers supporting both analysis routines implemented in R code and/or command-line software. A layered monitoring infrastructure is provided to design, control and debug each step in a workflow. The run environment allows to execute workflows entirely or with a intuitive to use step-wise execution syntax using R’s standard subsetting syntax (runWF(sys[1:3])) or pipes (%\u003e%). ]\n  Structural Features .left-column[\nWF infrastructure Command-line support ]\n.right-column[ An important feature of systemPipeR is support for running command-line software by adopting the Common Workflow Language (CWL). The latter is a widely adopted community standard for describing analysis workflows. This design offers several advantages such as:\n seamless integration of most command-line software\n support to run systemPipeR workflows from R or many other popular computer languages\n efficient sharing of workflows across different workflow environments. ]\n  Structural Features .left-column[\nWF infrastructure Command-line support Parallel evaluation ]\n.right-column[ The processing time of workflows can be greatly reduced by making use of parallel evaluations across several CPU cores on single machines, or multiple nodes of computer clusters and cloud-based systems. systemPipeR simplifies these parallelization tasks without creating any limitations for users who do not have access to high-performance computer resources ]\n  Structural Features .left-column[\nWF infrastructure Command-line support Parallel evaluation Reports infrastructure ]\n.right-column[ systemPipeR’s reporting infrastructure includes three types of interconnected reports each serving a different purpose:\n a scientific report, based on R Markdown, contains all scientifically relevant results\n a technical report captures all technical information important for each workflow step, including parameter settings, software versions, and warning/error messages, etc.\n a visual report depicts the entire workflow topology including its run status in form of a workflow graph\n]\n  Structural Features .left-column[\nWF infrastructure Command-line support Parallel evaluation Reports infrastructure Shiny Web Interface ]\n.right-column[ Recently, the systemPipeShiny package has been added that allows users to design workflows in an interactive graphical user interface (GUI). In addition to designing workflows, this new interface allows users to run and to monitor workflows in an intuitive manner without the need of knowing R. ]\n  Structural Features .left-column[\nWF infrastructure Command-line support Parallel evaluation Reports infrastructure Shiny Web Interface Workflow Templates ]\n.right-column[ A rich set of end-to-end workflow templates is provided by this project for a wide range omics applications. In addition, users can contribute and share their workflows with the community by submitting them to a central GitHub repository ]\n Important Functions .small[\n   Function Name Description Category     genWorkenvir Generates workflow templates provided by systemPipeRdata helper package / or from the individuals’ pipelines packages Accessory   loadWorkflow Constructs SYSargs2 object from CWL param and targets files SYSargs2   renderWF Populate all the command-line in an SYSargs2 object SYSargs2   subsetWF Subsetting SYSargs2 class slots SYSargs2   runCommandline Executes command-line software on samples and parameters specified in SYSargs2 object SYSargs2   clusterRun Runs command-line software in parallel mode on a computer cluster SYSargs2   writeTargetsout Write updated targets out to file/Generate targets file with reference SYSargs2   output_update Updates the output files paths in the SYSargs2 object SYSargs2   singleYML Create automatically the param.yml SYSargs2   createWF Create automatically param.cwl and the param.yml based on the command line SYSargs2   config.param Custom configuration of the CWL param files from R SYSargs2   ]       Important Functions .small[\n   Function Name Description Category     initWF Constructs SYSargsList workflow control module (S4 object) from script file SYSargsList   configWF Control of which step of the workflow will be run and the generation of the new RMarkdown SYSargsList   runWF Runs all the R chunk define in the RMarkdown file or a subset, e.g. runWF[1:3] SYSargsList   renderReport Render Scientific Report based on RMarkdown SYSargsList   subsetRmd Write updated subset Rmarkdown of R chunk with text associate in the step selected SYSargsList   plotWF Plot visual workflow designs and topologies with different graphical layouts SYSargsList   statusWF Return the overview of the workflow steps computational status SYSargsList   evalCode Turn eval option TRUE or FALSE on RMarkdown file Accessory   tryCL Checks if third-party software or utility is installed and set in the PATH Accessory   ]       class: inverse, center, middle\n Design  Workflow Management Solutions  systemPipeR central concept for designing workflows is workflow management containers (S4 class)\n SYSargs2 controls workflow steps with input/output file operations\n SYSargs2 requires a targets and a set of workflow definition files (here param.cwl and param.yml)\n --  SYSargsList objects organize one or many SYSargs2 containers in a single compound object capturing all information required to run, control and monitor complex workflows from start to finish\n.center[ ]\n  Directory Structure The workflow templates generated by genWorkenvir contain the following preconfigured directory structure:\n  Workflows Collection Browse pipelines that are currently available as part of the systemPipeR toolkit\n.small[\n   WorkFlow Description Version GitHub CI Testing     systemPipeChIPseq ChIP-Seq Workflow Template v1.0      systemPipeRIBOseq RIBO-Seq Workflow Template v1.0      systemPipeRNAseq RNA-Seq Workflow Template v1.0      systemPipeVARseq VAR-Seq Workflow Template v1.0      systemPipeMethylseq Methyl-Seq Workflow Template devel      systemPipeDeNovo De novo transcriptome assembly Workflow Template devel      systemPipeCLIPseq CLIP-Seq Workflow Template devel      systemPipeMetaTrans Metatranscriptomic Sequencing Workflow Template devel      ]         class: inverse, center, middle\nCWL   CWL  TODO: Add section with CWL details\n.center[ ]\n  CWL and SPR  TODO: How to use CWL definition with systemPipeR\n SYSargs2 instances are constructed from a targets file and two param file  hisat2-mapping-se.cwl file contains the settings for running command-line software hisat2-mapping-se.yml file define all the variables to be input in the specific command-line step     targets \u003c- system.file(\"extdata\", \"targets.txt\", package=\"systemPipeR\") dir_path \u003c- system.file(\"extdata/cwl/hisat2/hisat2-se\", package=\"systemPipeR\") align \u003c- loadWF(targets=targets, wf_file=\"hisat2-mapping-se.cwl\", input_file=\"hisat2-mapping-se.yml\", dir_path=dir_path) align \u003c- renderWF(align, inputvars=c(FileName=\"_FASTQ_PATH_\", SampleName=\"_SampleName_\")) ## Instance of 'SYSargs2': ## Slot names/accessors: ## targets: 18 (M1A...V12B), targetsheader: 4 (lines) ## modules: 2 ## wf: 0, clt: 1, yamlinput: 7 (components) ## input: 18, output: 18 ## cmdlist: 18 ## WF Steps: ## 1. hisat2-mapping-se.cwl (rendered: TRUE)    CWL and SPR SYSargs2 instance  Slots and accessor functions have the same names  names(align) # [1] \"targets\" \"targetsheader\" \"modules\" \"wf\" \"clt\" # [6] \"yamlinput\" \"cmdlist\" \"input\" \"output\" \"cwlfiles\" # [11] \"inputvars\"   cmdlist return command-line arguments for the specific software, here HISAT2 for the first sample  cmdlist(align)[1] # $M1A # $M1A$`hisat2-mapping-se.cwl` # [1] \"hisat2 -S results/M1A.sam -x ./data/tair10.fasta -k 1 --min-intronlen 30 --max-intronlen 3000 -U ./data/SRR446027_1.fastq.gz --threads 4\"   The output components of SYSargs2 define all the expected output files for each step in the workflow; some of which are the input for the next workflow step  output(align)[1] # $M1A # $M1A$`hisat2-mapping-se.cwl` # [1] \"results/M1A.sam\"   class: inverse, center, middle\n Metadata   Targets file organizes samples  Structure of targets file for single-end (SE) library  targetspath \u003c- system.file(\"extdata\", \"targets.txt\", package=\"systemPipeR\") read.delim(targetspath, comment.char = \"#\")[1:3,1:4]  ## FileName SampleName Factor SampleLong ## 1 ./data/SRR446027_1.fastq.gz M1A M1 Mock.1h.A ## 2 ./data/SRR446028_1.fastq.gz M1B M1 Mock.1h.B ## 3 ./data/SRR446029_1.fastq.gz A1A A1 Avr.1h.A   Structure of targets file for paired-end (PE) library  targetspath \u003c- system.file(\"extdata\", \"targetsPE.txt\", package=\"systemPipeR\") read.delim(targetspath, comment.char = \"#\")[1:3,1:5]  ## FileName1 FileName2 SampleName Factor ## 1 ./data/SRR446027_1.fastq.gz ./data/SRR446027_2.fastq.gz M1A M1 ## 2 ./data/SRR446028_1.fastq.gz ./data/SRR446028_2.fastq.gz M1B M1 ## 3 ./data/SRR446029_1.fastq.gz ./data/SRR446029_2.fastq.gz A1A A1 ## SampleLong ## 1 Mock.1h.A ## 2 Mock.1h.B ## 3 Avr.1h.A    Integration with SummarizedExperiment  Integrates targets files and count table from systemPipeR to a SummarizedExperiment object  ## Create an object with targets file and comparison and count table sprSE \u003c- SPRdata(targetspath = targetspath, cmp=TRUE) metadata(sprSE) # $version # [1] ‘1.23.9’ # # $comparison # $comparison$CMPset1 # [,1] [,2] # [1,] \"M1\" \"A1\" # [2,] \"M1\" \"V1\" # [3,] \"A1\" \"V1\" # [4,] \"M6\" \"A6\" colData(sprSE) # DataFrame with 18 rows and 6 columns # FileName SampleName Factor SampleLong # \u003ccharacter\u003e \u003ccharacter\u003e \u003ccharacter\u003e \u003ccharacter\u003e # M1A ./data/SRR446027_1.f.. M1A M1 Mock.1h.A # M1B ./data/SRR446028_1.f.. M1B M1 Mock.1h.B # ... ... ... ... ... # M12B ./data/SRR446040_1.f.. M12B M12 Mock.12h.B   class: inverse, center, middle\n Live Demo   Install Package Install the systemPipeRdata package from Bioconductor:\nif (!requireNamespace(\"BiocManager\", quietly=TRUE)) install.packages(\"BiocManager\") BiocManager::install(\"systemPipeR\")   Load Package and Documentation  Load package:\nlibrary(\"systemPipeR\")   Access help:\nlibrary(help=\"systemPipeR\") vignette(\"systemPipeR\")    Quick Start Load Sample Workflow  systemPipeRdata\n Helper package to generate with a single command workflow templates for systemPipeR Includes sample data for testing User can create new workflows or change and extend existing ones Template Workflows:  Sample workflows can be loaded with the genWorkenvir function from systemPipeRdata     Generate workflow template:\nlibrary(systemPipeRdata) genWorkenvir(workflow=\"rnaseq\") setwd(\"rnaseq\")   More details about systemPipeRdata package here\n  Install Workflow  Check the workflow template availability\navailableWF(github = TRUE) # $systemPipeRdata # [1] \"chipseq\" \"new\" \"riboseq\" \"rnaseq\" \"varseq\" # # $github # workflow branches version html description # 1 systemPipeR/systemPipeChIPseq master release https://github.com/systemPipeR/systemPipeChIPseq Workflow Template # 2 systemPipeR/systemPipeRIBOseq master release https://github.com/systemPipeR/systemPipeRIBOseq Workflow Template # 3 systemPipeR/systemPipeRNAseq cluster, master, singleMachine release https://github.com/systemPipeR/systemPipeRNAseq Workflow Template # 4 systemPipeR/systemPipeVARseq master release https://github.com/systemPipeR/systemPipeVARseq Workflow Template # 5 systemPipeR/systemPipeCLIPseq master devel https://github.com/systemPipeR/systemPipeCLIPseq Workflow Template # 6 systemPipeR/systemPipeDeNovo master devel https://github.com/systemPipeR/systemPipeDeNovo Workflow Template # 7 systemPipeR/systemPipeMetaTrans master devel https://github.com/systemPipeR/systemPipeMetaTrans Workflow Template # 8 systemPipeR/systemPipeMethylseq master devel https://github.com/systemPipeR/systemPipeMethylseq Workflow Template    Dynamic Workflow Template  Create dynamic Workflow Templates with RStudio\n File -\u003e New File -\u003e R Markdown -\u003e From Template .center[ ]\n  Run a Workflow .left-column[\nSetup ]\n.right-column[\nlibrary(systemPipeR) targetspath \u003c- system.file(\"extdata\", \"targets.txt\", package=\"systemPipeR\") read.delim(targetspath, comment.char = \"#\")[1:4,1:4]  ## FileName SampleName Factor SampleLong ## 1 ./data/SRR446027_1.fastq.gz M1A M1 Mock.1h.A ## 2 ./data/SRR446028_1.fastq.gz M1B M1 Mock.1h.B ## 3 ./data/SRR446029_1.fastq.gz A1A A1 Avr.1h.A ## 4 ./data/SRR446030_1.fastq.gz A1B A1 Avr.1h.B  script \u003c- system.file(\"extdata/workflows/rnaseq\", \"systemPipeRNAseq.Rmd\", package=\"systemPipeRdata\")  ]\n  Run a Workflow .left-column[\nSetup initWF ]\n.right-column[\nsysargslist \u003c- initWF(script = script, targets = targetspath, overwrite = TRUE) # Project started with success: ./SYSproject and SYSconfig.yml were created.  ]\n  Run a Workflow .left-column[\nSetup initWF configWF ]\n.right-column[\nsysargslist \u003c- configWF(sysargslist, input_steps = \"1:3\") sysargslist # Instance of 'SYSargsList': # WF Steps: # 1. Rmarkdown/HTML setting # 2. Introduction # 3. Samples and environment settings # 3.1. Environment settings and input data # 3.2. Required packages and resources # 3.3. Experiment definition provided by `targets` file  ]\n  Run a Workflow .left-column[\nSetup initWF configWF runWF ]\n.right-column[\nsysargslist \u003c- runWF(sysargslist, steps = \"1:2\") # Step: 1: Introduction --\u003e DONE # Step: 2: Samples and environment settings --\u003e DONE # Step: 2.1: Environment settings and input data --\u003e DONE # Step: 2.2: Required packages and resources --\u003e DONE # Step: 2.3: Experiment definition provided by `targets` file --\u003e DONE  sysargslist \u003c- runWF(sysargslist, steps = \"ALL\")  ]\n  Run a Workflow .left-column[\nSetup initWF configWF runWF renderReport ]\n.right-column[\nsysargslist \u003c- renderReport(sysargslist = sysargslist)  ]\n How to Use Pipes %\u003e% Consider the following example, in which the steps are the initialization, configuration and running the entire workflow.\nlibrary(systemPipeR) sysargslist \u003c- initWF(script =\"systemPipeRNAseq.Rmd\", overwrite = TRUE) %\u003e% configWF(input_steps = \"1:6\") %\u003e% runWF(steps = \"1:2\")   class: inverse, center, middle\n Project Updates  targets x SummarizedExperiment  Extension “SummarizedExperiment” methods:\nsprSE \u003c- addAssay(sprSE, assay(countMatrix), xName=\"countMatrix\") sprSE \u003c- addMetadata(sprSE, list(targets), xName=\"metadata\")   New Function:\n## Create empty SummarizedExperiment sprSE \u003c- SPRdata() ## Create an object with targets file and comparison and count table sprSE \u003c- SPRdata(counts = countMatrix, cmp=TRUE, targetspath = targetspath) metadata(sprSE) colData(sprSE) assays(sprSE)   SPR Paper  Link to draft\n Added the main points to discuss in the draft\n Writing: Results and introduction\n Improve Graphical Abstract\n Show case?\n SYSargsList  Explain how SYSargsList is implemented - Vignette\n.small[\n   Function Name Description     initWF Constructs SYSargsList workflow control module (S4 object) from script file   configWF Control of which step of the workflow will be run and the generation of the new RMarkdown   runWF Runs all the R chunk define in the RMarkdown file or a subset, e.g. runWF[1:3]   renderReport Render Scientific Report based on RMarkdown   renderLog Render logs Report based on RMarkdown   updateWF Recover the SYSargsList workflow previous ran and restarts the WF   plotWF Plot visual workflow designs and topologies with different graphical layouts   statusWF Return the overview of the workflow steps computational status   evalCode Turn eval option TRUE or FALSE on RMarkdown file   tryCL Checks if third-party software or utility is installed and set in the PATH   ]      Improve statusWF()\n Visualization in systemPipeR  Add to vignette (SPR or SPS)\n exploreDDS, exploreDDSplot, GLMplot, MAplot, MDSplot, PCAplot, hclustplot, heatMaplot, tSNEplot, volcanoplot   Enrichment analysis and visualization tool for SPR\n Integration with FGSEA   WebSite  Updated the vignette\n Added systemPipeRdata vignette and presentation: link\n Redirect http://girke.bioinformatics.ucr.edu/systemPipeR/ to new page\n Add content to FAQ section\n Add tutorials videos\n class: middle\nThanks!  Browse source code at \n Ask a question about systemPipeR at Bioconductor Support Page \n systemPipeRdata at Bioconductor\n https://systempipe.org/\n","categories":"","description":"","excerpt":" layout: true background-image: …","ref":"/presentations/spr/spr_project/","tags":"","title":"_systemPipeR_"},{"body":"systemPipeR 1.24 is available OVERVIEW The following enhancements have been added to systemPipeR.\n  With the upgrades provided in this release, systemPipeR has become a much more generic data analysis workflow environment that is no longer limited to analyzing just NGS data. Now it can be efficiently used for data analysis tasks in many omics areas, including genomics, proteomics, metabolomics and drug discovery.\n  A workflow control class (SYSargsList) has been added allowing users to manage multiple-step workflows from a single container. This way one can select and execute multiple workflow steps with standard R subsetting syntax, e.g. runWF[1:3].\n  Various improvements have been added to systemPipeR’s new command-line interface including the recently introduced SYSargs2 class that supports the Common Workflow Language (CWL). Utilities have been added to visualize workflow designs and topologies with different graphical layouts.\n  Improvements have been added to monitor the run status of workflows, as well as tracking of warning and error messages. This includes the generation of both scientific and technical status reports. \n   ","categories":"","description":"","excerpt":"systemPipeR 1.24 is available OVERVIEW The following enhancements have …","ref":"/news/release_3_12/","tags":"","title":"Bioconductor 3.12"},{"body":" Find here all the documentation!\n Motivation The analysis of Next-generation sequencing (NGS) data remains a major obstacle to the efficient utilization of the technology. While substantial effort has been invested on the development of software dedicated to the individual analysis steps of NGS experiments, insufficient resources are currently available for integrating the individual software components within the widely used R/Bioconductor environment into automated workflows capable of running the analysis of most types of NGS applications from start-to-finish in a time-efficient and reproducible manner.\nResults To address this need, we have developed the R/Bioconductor package systemPipeR. It is an extensible environment for both building and running end-to-end analysis workflows with automated report generation for a wide range of NGS applications. Its unique features include a uniform workflow interface across different NGS applications, automated report generation, and support for running both R and command-line software on local computers and computer clusters. A flexible sample annotation infrastructure efficiently handles complex sample sets and experimental designs.\nTo simplify the analysis of widely used NGS applications, the package provides pre-configured workflows and reporting templates for a wide range of NGS applications that are listed on the Workflow Collection page of this site. Additional workflow templates will be provided in the future. systemPipeR accelerates the extraction of reproducible analysis results from NGS experiments. By combining the capabilities of many R/Bioconductor and command-line tools, it makes efficient use of existing software resources without limiting the user to a set of predefined methods or environments.\n\nAvailability systemPipeR is freely available for all common operating systems from Bioconductor. Its GitHub repository is here.\n  Acknowledgement This project is funded by NSF award ABI-1661152.\n","categories":"","description":"","excerpt":" Find here all the documentation!\n Motivation The analysis of …","ref":"/sp/","tags":"","title":"systemPipeR Documentation"},{"body":"Starting with pre-configured workflow templates To test workflows quickly or design new ones from existing templates, users can generate with a single command workflow instances fully populated with sample data and parameter files required for running a chosen workflow.\nLoad one of the available workflows into your current working directory. The following does this for the varseq workflow template. The name of the resulting workflow directory can be specified under the mydirname argument. The default NULL uses the name of the chosen workflow. An error is issued if a directory of the same name and path exists already.\nlibrary(\"systemPipeRdata\") genWorkenvir(workflow=\"systemPipeR/SPvarseq\", mydirname=NULL) setwd(\"varseq\")  On Linux and OS X systems the same can be achieved from the command-line of a terminal with the following commands.\n$ Rscript -e \"systemPipeRdata::genWorkenvir(workflow='systemPipeR/SPvarseq', mydirname=NULL)\"  Check availability of workflow templates A collection of workflow templates are available, and it is possible to browse the current availability, as follows:\navailableWF(github = TRUE)  ## $systemPipeRdata ## [1] \"chipseq\" \"new\" \"riboseq\" \"rnaseq\" \"varseq\" ## ## $github ## workflow branches version ## 1 systemPipeR/SPchipseq master release ## 2 systemPipeR/SPriboseq master release ## 3 systemPipeR/SPrnaseq cluster, master, singleMachine release ## 4 systemPipeR/SPvarseq master release ## 5 systemPipeR/SPclipseq master devel ## 6 systemPipeR/SPdenovo master devel ## 7 systemPipeR/SPmetatrans master devel ## 8 systemPipeR/SPmethylseq master devel ## 9 systemPipeR/SPmirnaseq master devel ## html description ## 1 https://github.com/systemPipeR/SPchipseq Workflow Template ## 2 https://github.com/systemPipeR/SPriboseq Workflow Template ## 3 https://github.com/systemPipeR/SPrnaseq Workflow Template ## 4 https://github.com/systemPipeR/SPvarseq Workflow Template ## 5 https://github.com/systemPipeR/SPclipseq Workflow Template ## 6 https://github.com/systemPipeR/SPdenovo Workflow Template ## 7 https://github.com/systemPipeR/SPmetatrans Workflow Template ## 8 https://github.com/systemPipeR/SPmethylseq Workflow Template ## 9 https://github.com/systemPipeR/SPmirnaseq Workflow Template  This function returns the list of workflow templates available within the package and systemPipeR Project Organization on GitHub. Each one listed template can be created as described above.\nThe workflow template choose from Github will be installed as an R package, and also it creates an environment with all the settings and files to run the demo analysis.\ngenWorkenvir(workflow=\"systemPipeR/SPrnaseq\", mydirname=\"NULL\") setwd(\"systemPipeVARseq\")  Besides, it is possible to choose different versions of the workflow template, defined through other branches on the GitHub Repository. By default, the master branch is selected, however, it is possible to define a different branch with the ref argument.\ngenWorkenvir(workflow=\"systemPipeR/SPrnaseq\", ref = \"singleMachine\") setwd(\"systemPipeRNAseq\")  ","categories":"","description":"","excerpt":"Starting with pre-configured workflow templates To test workflows …","ref":"/spr_wf/installwf/","tags":"","title":"How to install systemPipe Workflows"},{"body":" Stay tuned for all sytemPipe project news!\n ","categories":"","description":"","excerpt":" Stay tuned for all sytemPipe project news!\n ","ref":"/news/","tags":"","title":"News"},{"body":"","categories":"","description":"","excerpt":"","ref":"/presentations/","tags":"","title":"Presentations"},{"body":"How to create a new Workflow Template SPRthis package expand usethis package, providing automation to create systemPipeR workflows templates.\nInstallation To install SPRthis using from BiocManager the following code:\nif (!requireNamespace(\"BiocManager\", quietly = TRUE)) { install.packages(\"BiocManager\") BiocManager::install(\"dcassol/SPRthis\")  Quick start to using to SPRthis ## Load the package library(SPRthis) ## create Package sprthis(wfName=\"SPRtest\", analysis=\"SPRtest\", path=tempdir())  ## ✓ Setting active project to '/tmp/RtmpWTh4tB'  ## ✓ Creating 'R/'  ## ✓ Writing 'DESCRIPTION'  ## Package: SPRtest ## Title: SPRtest ## Version: 0.9.0 ## Authors@R (parsed): ## * First Last \u003cfirst.last@example.com\u003e [aut, cre] (YOUR-ORCID-ID) ## Description: This package provides a pre-configured workflow and reporting ## template for SPRtest. ## License: Artistic-2.0 ## URL: https://github.com/systemPipeR/SPRtest ## Imports: ## systemPipeR (\u003e= 1.25.0) ## Suggests: ## BiocStyle, ## knitr, ## rmarkdown ## VignetteBuilder: ## knitr ## biocViews: Infrastructure, ... ## Encoding: UTF-8 ## LazyData: true ## Roxygen: list(markdown = TRUE) ## RoxygenNote: 7.1.1 ## SystemRequirements: SPRtest can be used to run external command-line ## software, but the corresponding tool needs to be installed on a ## system.  ## ✓ Writing 'NAMESPACE'  ## ✓ Setting active project to '\u003cno active project\u003e'  ## [1] \"/tmp/RtmpWTh4tB\"   SPRtest/ ├── DESCRIPTION ├── NAMESPACE ├── README.md ├── SPRtest.Rproj ├── .gitignore ├── .Rbuildignore ├── .Rproj.user/ ├── R/ │ ├── functions.R ├── vignettes │ ├── bibtex.bib │ ├── SPRtest.Rmd └── inst ├── rmarkdown │ └── templates │ └── SPRtest │ ├── template.yml │ └── skeleton │ ├── batchtools.slurm.tmpl │ ├── .batchtools.conf.R │ ├── bibtex.bib │ ├── NEWS │ ├── SPRconfig.yml │ ├── skeleton.Rmd │ ├── targetsPE.txt │ ├── data/ │ ├── param/ │ └── results/  Help functions to create the package Create the webiste for the package with pkgdown Edit the _pkgdown.yml file and run:\npkgdown::build_site()  Documentation with roxygen2 roxygen2::roxygenise()  Testing the code with testthat To test the code, you can run\ndevtools::test()  Package available to genWorkenvir Function After creating the new repository on GitHub systemPipeR Organization, please follow:\n Rules:  The Workflow Template need to be available under systemPipeR Organization; The repository needs to be public; About setting:  Description: keywords in the description are required: “Workflow Template”; Topics: we expected “systempiper” and “release” or “development” words on Topics section;   Branch name: To make simple, please name the branch as “master”.    Check availability of workflow templates A collection of workflow templates are available, and it is possible to browse the current availability, as follows:\nsystemPipeRdata::availableWF(github = TRUE)  ## $systemPipeRdata ## [1] \"chipseq\" \"new\" \"riboseq\" \"rnaseq\" \"varseq\" ## ## $github ## workflow branches version ## 1 systemPipeR/SPchipseq master release ## 2 systemPipeR/SPriboseq master release ## 3 systemPipeR/SPrnaseq cluster, master, singleMachine release ## 4 systemPipeR/SPvarseq master release ## 5 systemPipeR/SPclipseq master devel ## 6 systemPipeR/SPdenovo master devel ## 7 systemPipeR/SPmetatrans master devel ## 8 systemPipeR/SPmethylseq master devel ## 9 systemPipeR/SPmirnaseq master devel ## html description ## 1 https://github.com/systemPipeR/SPchipseq Workflow Template ## 2 https://github.com/systemPipeR/SPriboseq Workflow Template ## 3 https://github.com/systemPipeR/SPrnaseq Workflow Template ## 4 https://github.com/systemPipeR/SPvarseq Workflow Template ## 5 https://github.com/systemPipeR/SPclipseq Workflow Template ## 6 https://github.com/systemPipeR/SPdenovo Workflow Template ## 7 https://github.com/systemPipeR/SPmetatrans Workflow Template ## 8 https://github.com/systemPipeR/SPmethylseq Workflow Template ## 9 https://github.com/systemPipeR/SPmirnaseq Workflow Template  This function returns the list of workflow templates available within the package and systemPipeR Project Organization on GitHub. Each one listed template can be created as described above.\n","categories":"","description":"","excerpt":"How to create a new Workflow Template SPRthis package expand usethis …","ref":"/spr_wf/newwf/","tags":"","title":"Rules to create a new Workflow Template"},{"body":"","categories":"","description":"","excerpt":"","ref":"/search/","tags":"","title":"Search Results"},{"body":"Workflow Templates systemPipeR project provides pre-configured workflows and reporting templates for a wide range of NGS applications that are listed bellow. The systemPipeR project provides a suite of R/Bioconductor packages for designing, building and running end-to-end analysis workflows on local machines, HPC clusters and cloud systems, while generating at the same time publication quality analysis reports.\nsystemPipeRdata is a helper package to generate with a single command workflow templates that are intended to be used by its parent package systemPipeR (H Backman and Girke 2016).\n   WorkFlow Description Version GitHub R-CMD-check     SPchipseq ChIP-Seq Workflow Template      SPriboseq RIBO-Seq Workflow Template      SPrnaseq RNA-Seq Workflow Template      SPvarseq VAR-Seq Workflow Template      SPmethylseq Methyl-Seq Workflow Template      SPdenovo De novo transcriptome assembly Workflow Template      SPclipseq CLIP-Seq Workflow Template      SPmetatrans Metatranscriptomic Sequencing Workflow Template      SPatacseq ATAC-Seq Workflow Template      SPpolyriboseq Polyribosomal RNA-Seq Workflow Template      SPhic Hi-C Workflow Template      SPmirnaseq MicroRNA-Seq Workflow Template      SPblast BLAST Workflow Template      SPscrnaseq Single-Cell RNA-Seq Workflow Template       Reference H Backman, Tyler W, and Thomas Girke. 2016. “systemPipeR: NGS workflow and report generation environment.” BMC Bioinformatics 17 (1): 388. https://doi.org/10.1186/s12859-016-1241-0.\n  ","categories":"","description":"","excerpt":"Workflow Templates systemPipeR project provides pre-configured …","ref":"/spr_wf/","tags":"","title":"SPR WorkFlow Collection"},{"body":" #td-cover-block-0 { background-image: url(\"background.jpg\") }  systemPipe Workflow Environment    loadLogo(\"images/sp_org_links.svg\")   Learn More   Bioconductor 3.12   New Features   A workflow design, report generation, and visualization framework for data analysis\n         systemPipe Project\nThe systemPipe project provides a suite of R/Bioconductor packages for designing, building and running end-to-end analysis workflows on local machines, HPC clusters and cloud systems, while generating at the same time publication quality analysis reports.\n      Bioconductor Download a stable release from Bioconductor of systemPipeR, systemPipeShiny, systemPipeRdata.\n   Contributions welcome! Get the lastest version, submit issues and pull requests.\nRead more …\n   Demo Try an interactive demo and tutorial.\n    ","categories":"","description":"","excerpt":" #td-cover-block-0 { background-image: url(\"background.jpg\") } …","ref":"/","tags":"","title":"systemPipe"},{"body":"   loadLogo(\"/images/sps_logos.svg\")   #svg-logo { background: radial-gradient(circle, transparent 0%, rgba(255,255,255,1) 30%), linear-gradient(to right, var(--color), var(--color)), var(--image2); background-repeat: no-repeat; background-size: auto 1200px; background-position: center center; background-blend-mode: var(--blend-top, normal), var(--blend-bottom, saturation), normal; --image2: url(\"/background.jpg\"); --color-v: rgba(76,169,237,1); --color: rgba(76,169,237,1); } #svg-logo svg { margin: 0 auto; display: block; padding: 50px; }  systemPipeShiny (SPS) extends the widely used systemPipeR (SPR) workflow environment with a versatile graphical user interface provided by a Shiny App. This allows non-R users, such as experimentalists, to run many systemPipeR’s workflow designs, control, and visualization functionalities interactively without requiring knowledge of R. Most importantly, SPS has been designed as a general purpose framework for interacting with other R packages in an intuitive manner. Like most Shiny Apps, SPS can be used on both local computers as well as centralized server-based deployments that can be accessed remotely as a public web service for using SPR’s functionalities with community and/or private data. The framework can integrate many core packages from the R/Bioconductor ecosystem. Examples of SPS' current functionalities include:\n A default interactive workflow module to create experimental designs, visualize and customize workflow topologies with previews, and programming free workflow execution within the application. An interactive module with extensive plot options to visualize downstream analysis of a RNA-Seq workflow. A quick ggplot module to make all variety of scientific plots from any user defined tabular data. An extendable set of visualization functionalities makes it easy to design custom Shiny Apps under SPS framework without any knowledge of Shiny. A ‘Canvas Workbench’ to manage complex visual results. It allows users to organize and to compare plots in an efficient manner combined with a session screenshot feature to edit scientific and publishable figures. Three other supporting packages to help all users from beginners and advanced developers to extend under current SPS framework or on their own visualization apps.  Demo View our online demo app:\n   Type and link option changed notes     Default full installation{blk} See installation full app   Minimum installation{blk} See installation no modules installed   Login enabled{blk} login_screen = TRUE; login_theme = \"empty\" no modules installed   Login and login themes{blk} login_screen = TRUE; login_theme = \"random\" no modules installed   App admin page{blk} admin_page = TRUE or simply add “?admin” to the end of URL of demos    For the login required demos, the app account name is “user” password “user”.\nFor the admin panel login, account name “admin”, password “admin”.\nPlease DO NOT delete or change password when you are using the admin features. shinyapp.io will reset the app once a while, but this will affect other people who are trying the demo simultaneously.\nOther packages in systemPipeShiny    Package Description Documents Function reference Demo     systemPipeShiny{blk} SPS main package website link demo{blk}   spsComps{blk} SPS UI and server components website link demo{blk}   drawer{blk} SPS interactive image editing tool website link demo{blk}   spsUtil{blk} SPS utility functions website link NA    ","categories":"","description":"","excerpt":"   loadLogo(\"/images/sps_logos.svg\")   #svg-logo { background: …","ref":"/sps/","tags":"","title":"systemPipeShiny Documentation"}]
\ No newline at end of file
diff --git a/public/presentations/index.html b/public/presentations/index.html
index 41ab69f43..4f128bee2 100644
--- a/public/presentations/index.html
+++ b/public/presentations/index.html
@@ -493,7 +493,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/presentations/spr/spr_project/index.html b/public/presentations/spr/spr_project/index.html
index ea28b4eca..6a27de879 100644
--- a/public/presentations/spr/spr_project/index.html
+++ b/public/presentations/spr/spr_project/index.html
@@ -510,7 +510,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/presentations/sps/sps_intro/index.html b/public/presentations/sps/sps_intro/index.html
index a8fa09ea2..7cd515f47 100644
--- a/public/presentations/sps/sps_intro/index.html
+++ b/public/presentations/sps/sps_intro/index.html
@@ -506,7 +506,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/search/index.html b/public/search/index.html
index 3f14363c2..17390c685 100644
--- a/public/search/index.html
+++ b/public/search/index.html
@@ -498,7 +498,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sitemap.xml b/public/sitemap.xml
index 148398ea8..9f775739d 100644
--- a/public/sitemap.xml
+++ b/public/sitemap.xml
@@ -34,7 +34,7 @@
   
   <url>
     <loc>/sps/dev/spscomps/</loc>
-    <lastmod>2021-05-09T15:49:11-07:00</lastmod>
+    <lastmod>2021-05-09T15:52:02-07:00</lastmod>
   </url>
   
   <url>
diff --git a/public/sp/index.html b/public/sp/index.html
index 4227a1ae1..02ebf7230 100644
--- a/public/sp/index.html
+++ b/public/sp/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/sp_docker/index.html b/public/sp/sp_docker/index.html
index 9d04b970e..0b107b94e 100644
--- a/public/sp/sp_docker/index.html
+++ b/public/sp/sp_docker/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/gettingstarted/index.html b/public/sp/spr/gettingstarted/index.html
index 6bb83a678..4eccd2b57 100644
--- a/public/sp/spr/gettingstarted/index.html
+++ b/public/sp/spr/gettingstarted/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/index.html b/public/sp/spr/index.html
index fa0d0e943..56e5c4510 100644
--- a/public/sp/spr/index.html
+++ b/public/sp/spr/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/introduction/index.html b/public/sp/spr/introduction/index.html
index 7907dc3e0..897b0062d 100644
--- a/public/sp/spr/introduction/index.html
+++ b/public/sp/spr/introduction/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/sp_docker/index.html b/public/sp/spr/sp_docker/index.html
index ccfec27fa..d8c0922f4 100644
--- a/public/sp/spr/sp_docker/index.html
+++ b/public/sp/spr/sp_docker/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/spr_funcs/index.html b/public/sp/spr/spr_funcs/index.html
index 6fb5137ac..28cb09cb9 100644
--- a/public/sp/spr/spr_funcs/index.html
+++ b/public/sp/spr/spr_funcs/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/spr_run/index.html b/public/sp/spr/spr_run/index.html
index 92493d4c7..18b7088d5 100644
--- a/public/sp/spr/spr_run/index.html
+++ b/public/sp/spr/spr_run/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/sprinstall/index.html b/public/sp/spr/sprinstall/index.html
index 9ea81f77b..1171bee03 100644
--- a/public/sp/spr/sprinstall/index.html
+++ b/public/sp/spr/sprinstall/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/steps/index.html b/public/sp/spr/steps/index.html
index fe9947712..bbbeec773 100644
--- a/public/sp/spr/steps/index.html
+++ b/public/sp/spr/steps/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spr/templates/index.html b/public/sp/spr/templates/index.html
index decba31a5..2f8b6fafa 100644
--- a/public/sp/spr/templates/index.html
+++ b/public/sp/spr/templates/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/sprdata/index.html b/public/sp/sprdata/index.html
index 46ee9b91e..7ef1c7a78 100644
--- a/public/sp/sprdata/index.html
+++ b/public/sp/sprdata/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/sprdata/newwf/index.html b/public/sp/sprdata/newwf/index.html
index 41d8810a5..7e133cbb0 100644
--- a/public/sp/sprdata/newwf/index.html
+++ b/public/sp/sprdata/newwf/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/sprdata/systempiperdata/index.html b/public/sp/sprdata/systempiperdata/index.html
index 35cb770ac..4d590b9a0 100644
--- a/public/sp/sprdata/systempiperdata/index.html
+++ b/public/sp/sprdata/systempiperdata/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spt/index.html b/public/sp/spt/index.html
index b7a054e35..f7994e5a3 100644
--- a/public/sp/spt/index.html
+++ b/public/sp/spt/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sp/spt/systempipetools/index.html b/public/sp/spt/systempipetools/index.html
index b8b718407..42fe2ad1d 100644
--- a/public/sp/spt/systempipetools/index.html
+++ b/public/sp/spt/systempipetools/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/spr_wf/index.html b/public/spr_wf/index.html
index a71757528..89811bd47 100644
--- a/public/spr_wf/index.html
+++ b/public/spr_wf/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/spr_wf/installwf/index.html b/public/spr_wf/installwf/index.html
index 87f359007..62c0608d0 100644
--- a/public/spr_wf/installwf/index.html
+++ b/public/spr_wf/installwf/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/spr_wf/newwf/index.html b/public/spr_wf/newwf/index.html
index 66b351c6c..cf0e13188 100644
--- a/public/spr_wf/newwf/index.html
+++ b/public/spr_wf/newwf/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/app_security/index.html b/public/sps/adv_features/app_security/index.html
index 5c7eaf03e..83d349460 100644
--- a/public/sps/adv_features/app_security/index.html
+++ b/public/sps/adv_features/app_security/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/config/index.html b/public/sps/adv_features/config/index.html
index 18164374c..a91e729bb 100644
--- a/public/sps/adv_features/config/index.html
+++ b/public/sps/adv_features/config/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/database/index.html b/public/sps/adv_features/database/index.html
index 3a961da54..13a4d7a63 100644
--- a/public/sps/adv_features/database/index.html
+++ b/public/sps/adv_features/database/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/debug/index.html b/public/sps/adv_features/debug/index.html
index f6a872672..7050e5abc 100644
--- a/public/sps/adv_features/debug/index.html
+++ b/public/sps/adv_features/debug/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/displaytabs/index.html b/public/sps/adv_features/displaytabs/index.html
index a28eb357a..fc9bd2534 100644
--- a/public/sps/adv_features/displaytabs/index.html
+++ b/public/sps/adv_features/displaytabs/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/guide/index.html b/public/sps/adv_features/guide/index.html
index 5fa1d055d..ae8d8ce77 100644
--- a/public/sps/adv_features/guide/index.html
+++ b/public/sps/adv_features/guide/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/index.html b/public/sps/adv_features/index.html
index 2d282cf88..758567c0e 100644
--- a/public/sps/adv_features/index.html
+++ b/public/sps/adv_features/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/login/index.html b/public/sps/adv_features/login/index.html
index c0d538715..ab6d5a151 100644
--- a/public/sps/adv_features/login/index.html
+++ b/public/sps/adv_features/login/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/notification/index.html b/public/sps/adv_features/notification/index.html
index 0b23b3cd0..67a0b005f 100644
--- a/public/sps/adv_features/notification/index.html
+++ b/public/sps/adv_features/notification/index.html
@@ -517,7 +517,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -549,7 +549,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/other_customizations/index.html b/public/sps/adv_features/other_customizations/index.html
index 0bb302e19..8312f184e 100644
--- a/public/sps/adv_features/other_customizations/index.html
+++ b/public/sps/adv_features/other_customizations/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/overwritetabs/index.html b/public/sps/adv_features/overwritetabs/index.html
index fcd54e0e1..11179596f 100644
--- a/public/sps/adv_features/overwritetabs/index.html
+++ b/public/sps/adv_features/overwritetabs/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/adv_features/tabs/index.html b/public/sps/adv_features/tabs/index.html
index bf9afa146..deb4a0bd5 100644
--- a/public/sps/adv_features/tabs/index.html
+++ b/public/sps/adv_features/tabs/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/canvas/index.html b/public/sps/canvas/index.html
index 404494bf4..56d3c2afe 100644
--- a/public/sps/canvas/index.html
+++ b/public/sps/canvas/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/deploy/index.html b/public/sps/deploy/index.html
index 3b786627c..b034acf8d 100644
--- a/public/sps/deploy/index.html
+++ b/public/sps/deploy/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/dev/drawer/index.html b/public/sps/dev/drawer/index.html
index ed1262f72..83f368813 100644
--- a/public/sps/dev/drawer/index.html
+++ b/public/sps/dev/drawer/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/dev/drawer/rmd/index.html b/public/sps/dev/drawer/rmd/index.html
index 79f89c4b3..21c8007ab 100644
--- a/public/sps/dev/drawer/rmd/index.html
+++ b/public/sps/dev/drawer/rmd/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/dev/drawer/shiny/index.html b/public/sps/dev/drawer/shiny/index.html
index 476f5adb1..3b2a4dd02 100644
--- a/public/sps/dev/drawer/shiny/index.html
+++ b/public/sps/dev/drawer/shiny/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/dev/index.html b/public/sps/dev/index.html
index 39c443ea6..b5c2312d3 100644
--- a/public/sps/dev/index.html
+++ b/public/sps/dev/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/dev/index.xml b/public/sps/dev/index.xml
index f55f2807e..94e7dcfd9 100644
--- a/public/sps/dev/index.xml
+++ b/public/sps/dev/index.xml
@@ -26,8 +26,32 @@
 &lt;p&gt;systemPipeShiny Components (spsComps) package is a collection of custom UI  and
 server components that are used in SPS main framework. If you see a component
 in SPS but want to outside the SPS framework, like in your own Shiny apps,
-take a look at these components. Here is a &lt;strong&gt;&lt;a href=&#34;https://lezhang.shinyapps.io/spsComps&#34;&gt;online demo&lt;/a&gt;&lt;/strong&gt;
-of these components.&lt;/p&gt;
+take a look at these components.&lt;/p&gt;
+&lt;h2 id=&#34;demos&#34;&gt;Demos&lt;/h2&gt;
+&lt;table&gt;
+&lt;thead&gt;
+&lt;tr&gt;
+&lt;th&gt;Demo&lt;/th&gt;
+&lt;th&gt;type&lt;/th&gt;
+&lt;th&gt;source code&lt;/th&gt;
+&lt;/tr&gt;
+&lt;/thead&gt;
+&lt;tbody&gt;
+&lt;tr&gt;
+&lt;td&gt;&lt;a href=&#34;https://lezhang.shinyapps.io/spsComps&#34;&gt;shiny{blk}&lt;/a&gt;&lt;/td&gt;
+&lt;td&gt;shinyapps.io&lt;/td&gt;
+&lt;td&gt;&lt;a href=&#34;https://github.com/lz100/spsComps/tree/master/examples/demo&#34;&gt;Github{blk}&lt;/a&gt;&lt;/td&gt;
+&lt;/tr&gt;
+&lt;tr&gt;
+&lt;td&gt;&lt;a href=&#34;https://systempipe.org/sps/dev/spscomps/ui/&#34;&gt;Rmd&lt;/a&gt;&lt;/td&gt;
+&lt;td&gt;Rmarkdown rendered&lt;/td&gt;
+&lt;td&gt;&lt;a href=&#34;https://raw.githubusercontent.com/systemPipeR/systemPipeR.github.io/main/content/en/sps/dev/spscomps/ui.Rmd&#34;&gt;Raw{blk}&lt;/a&gt;&lt;/td&gt;
+&lt;/tr&gt;
+&lt;/tbody&gt;
+&lt;/table&gt;
+&lt;style&gt;
+table {font-size: 1.5rem}
+&lt;/style&gt;
 &lt;h2 id=&#34;installation&#34;&gt;Installation&lt;/h2&gt;
 &lt;p&gt;Read the &lt;a href=&#34;..&#34;&gt;developer tools&lt;/a&gt; main page, not repeating here.&lt;/p&gt;
 &lt;h2 id=&#34;category&#34;&gt;Category&lt;/h2&gt;
diff --git a/public/sps/dev/spscomps/index.html b/public/sps/dev/spscomps/index.html
index d51930ccf..7ec530325 100644
--- a/public/sps/dev/spscomps/index.html
+++ b/public/sps/dev/spscomps/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -963,6 +963,7 @@
     <li>
       <ul>
         <li><a href="#sps-components-package">SPS Components package</a></li>
+        <li><a href="#demos">Demos</a></li>
         <li><a href="#installation">Installation</a></li>
         <li><a href="#category">Category</a></li>
         <li><a href="#functions-reference-manual">Functions reference manual</a></li>
@@ -1048,8 +1049,32 @@ <h2 id="sps-components-package">SPS Components package</h2>
 <p>systemPipeShiny Components (spsComps) package is a collection of custom UI  and
 server components that are used in SPS main framework. If you see a component
 in SPS but want to outside the SPS framework, like in your own Shiny apps,
-take a look at these components. Here is a <strong><a href="https://lezhang.shinyapps.io/spsComps">online demo</a></strong>
-of these components.</p>
+take a look at these components.</p>
+<h2 id="demos">Demos</h2>
+<table>
+<thead>
+<tr>
+<th>Demo</th>
+<th>type</th>
+<th>source code</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><a href="https://lezhang.shinyapps.io/spsComps">shiny{blk}</a></td>
+<td>shinyapps.io</td>
+<td><a href="https://github.com/lz100/spsComps/tree/master/examples/demo">Github{blk}</a></td>
+</tr>
+<tr>
+<td><a href="https://systempipe.org/sps/dev/spscomps/ui/">Rmd</a></td>
+<td>Rmarkdown rendered</td>
+<td><a href="https://raw.githubusercontent.com/systemPipeR/systemPipeR.github.io/main/content/en/sps/dev/spscomps/ui.Rmd">Raw{blk}</a></td>
+</tr>
+</tbody>
+</table>
+<style>
+table {font-size: 1.5rem}
+</style>
 <h2 id="installation">Installation</h2>
 <p>Read the <a href="..">developer tools</a> main page, not repeating here.</p>
 <h2 id="category">Category</h2>
@@ -1185,7 +1210,7 @@ <h5>
 
 	
 	
-	<div class="text-muted mt-5 pt-3 border-top">Last modified 2021-05-09: <a  href="https://github.com/systemPipeR/systemPipeR.github.io/commit/bb837e458f439de356a21f7e45e72e4946eac2f7">spscomps (bb837e4)</a>
+	<div class="text-muted mt-5 pt-3 border-top">Last modified 2021-05-09: <a  href="https://github.com/systemPipeR/systemPipeR.github.io/commit/701fe024c0a612837ceb75794c5f688834fa1b88">spscomps no_render (701fe02)</a>
 </div>
 </div>
 
diff --git a/public/sps/dev/spscomps/index.xml b/public/sps/dev/spscomps/index.xml
index 062fc080e..3189391be 100644
--- a/public/sps/dev/spscomps/index.xml
+++ b/public/sps/dev/spscomps/index.xml
@@ -101,11 +101,10 @@
 &lt;link href=&#34;../../../rmarkdown-libs/spsComps-css-loader-css/css/css_loader.css&#34; rel=&#34;stylesheet&#34; /&gt;
 &lt;script src=&#34;../../../rmarkdown-libs/spsComps-css-loader-js/js/sps_cssloader.js&#34;&gt;&lt;/script&gt;
 &lt;link href=&#34;../../../rmarkdown-libs/spsComps-css-loader-css/css/css_loader.css&#34; rel=&#34;stylesheet&#34; /&gt;
-\#\# UI Components
-For most of the UI components, you can view them in the [online Shiny demo{blk}](https://lezhang.shinyapps.io/spsComps).
-Most but **not all** UI components work in a Rmarkdown document. Here we demostrate
+&lt;p&gt;For most of the UI components, you can view them in the &lt;a href=&#34;https://lezhang.shinyapps.io/spsComps&#34;&gt;online Shiny demo{blk}&lt;/a&gt;.
+Most but &lt;strong&gt;not all&lt;/strong&gt; UI components work in a Rmarkdown document. Here we demostrate
 how you could use some of them in a Rmarkdown doc. The source code of this
-document is on [Github{blk}](https://raw.githubusercontent.com/systemPipeR/systemPipeR.github.io/main/content/en/sps/dev/spscomps/ui.Rmd).
+document is on &lt;a href=&#34;https://raw.githubusercontent.com/systemPipeR/systemPipeR.github.io/main/content/en/sps/dev/spscomps/ui.Rmd&#34;&gt;Github{blk}&lt;/a&gt;.&lt;/p&gt;
 &lt;h2 id=&#34;load-package&#34;&gt;load package&lt;/h2&gt;
 &lt;p&gt;To start to use &lt;code&gt;spsComps&lt;/code&gt;, load it in your Shiny app file or Rmarkdown file&lt;/p&gt;
 &lt;pre&gt;&lt;code class=&#34;language-r&#34;&gt;library(spsComps)
@@ -168,46 +167,46 @@ gallery(
   enlarge_method = &amp;quot;modal&amp;quot;
 )
 &lt;/code&gt;&lt;/pre&gt;
-&lt;div id=&#34;gallery1872830&#34; class=&#34;col sps-gallery&#34; style=&#34;&#34;&gt;
+&lt;div id=&#34;gallery2917503&#34; class=&#34;col sps-gallery&#34; style=&#34;&#34;&gt;
 &lt;p class=&#34;text-center h2&#34; style=&#34;color: #0275d8;&#34;&gt;Gallery&lt;/p&gt;
-&lt;div class=&#34;row&#34; style=&#34;  margin: 10px;&#34;&gt;&lt;div  id=gallery1872830-1 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
+&lt;div class=&#34;row&#34; style=&#34;  margin: 10px;&#34;&gt;&lt;div  id=gallery2917503-1 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
   &lt;img
     src=&#34;https://github.com/lz100/spsComps/blob/master/img/1.jpg?raw=true&#34; class=&#34;img-gallery&#34;
     height=300 width=400
     style=&#34;width: 100%;&#34;
-    onclick=galEnlarge(&#34;#gallery1872830-1&#34;)
+    onclick=galEnlarge(&#34;#gallery2917503-1&#34;)
   &gt;
   &lt;a href=&#34;https://github.com/lz100/spsComps/blob/master/img/1.jpg?raw=true&#34; &gt;&lt;p class=&#34;text-center h4 &#34;&gt;p1&lt;/p&gt;&lt;/a&gt;
-&lt;/div&gt; &lt;div  id=gallery1872830-2 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
+&lt;/div&gt; &lt;div  id=gallery2917503-2 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
   &lt;img
     src=&#34;https://github.com/lz100/spsComps/blob/master/img/2.jpg?raw=true&#34; class=&#34;img-gallery&#34;
     height=300 width=400
     style=&#34;width: 100%;&#34;
-    onclick=galEnlarge(&#34;#gallery1872830-2&#34;)
+    onclick=galEnlarge(&#34;#gallery2917503-2&#34;)
   &gt;
   &lt;a href=&#34;https://github.com/lz100/spsComps/blob/master/img/2.jpg?raw=true&#34; &gt;&lt;p class=&#34;text-center h4 &#34;&gt;p2&lt;/p&gt;&lt;/a&gt;
-&lt;/div&gt; &lt;div  id=gallery1872830-3 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
+&lt;/div&gt; &lt;div  id=gallery2917503-3 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
   &lt;img
     src=&#34;https://github.com/lz100/spsComps/blob/master/img/3.jpg?raw=true&#34; class=&#34;img-gallery&#34;
     height=300 width=400
     style=&#34;width: 100%;&#34;
-    onclick=galEnlarge(&#34;#gallery1872830-3&#34;)
+    onclick=galEnlarge(&#34;#gallery2917503-3&#34;)
   &gt;
   &lt;a  &gt;&lt;p class=&#34;text-center h4 gallery-nohover&#34;&gt;&amp;nbsp;&lt;/p&gt;&lt;/a&gt;
-&lt;/div&gt; &lt;div  id=gallery1872830-4 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
+&lt;/div&gt; &lt;div  id=gallery2917503-4 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
   &lt;img
     src=&#34;https://github.com/lz100/spsComps/blob/master/img/4.jpg?raw=true&#34; class=&#34;img-gallery&#34;
     height=300 width=400
     style=&#34;width: 100%;&#34;
-    onclick=galEnlarge(&#34;#gallery1872830-4&#34;)
+    onclick=galEnlarge(&#34;#gallery2917503-4&#34;)
   &gt;
   &lt;a href=&#34;https://github.com/lz100/spsComps/blob/master/img/4.jpg?raw=true&#34; &gt;&lt;p class=&#34;text-center h4 &#34;&gt;p4&lt;/p&gt;&lt;/a&gt;
-&lt;/div&gt; &lt;div  id=gallery1872830-5 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
+&lt;/div&gt; &lt;div  id=gallery2917503-5 class=&#34;col-sm-4 sps-tab-link&#34; style=&#34;right: 1px;&#34;&gt;
   &lt;img
     src=&#34;https://github.com/lz100/spsComps/blob/master/img/5.jpg?raw=true&#34; class=&#34;img-gallery&#34;
     height=300 width=400
     style=&#34;width: 100%;&#34;
-    onclick=galEnlarge(&#34;#gallery1872830-5&#34;)
+    onclick=galEnlarge(&#34;#gallery2917503-5&#34;)
   &gt;
   &lt;a href=&#34;https://github.com/lz100/spsComps/blob/master/img/5.jpg?raw=true&#34; &gt;&lt;p class=&#34;text-center h4 &#34;&gt;p5&lt;/p&gt;&lt;/a&gt;
 &lt;/div&gt;&lt;/div&gt;
@@ -218,7 +217,7 @@ gallery(
 &lt;div class=&#34;gallery-caption&#34;&gt;&lt;/div&gt;
 &lt;/div&gt;
 &lt;!--/SHINY.SINGLETON[7fdec3ac87205a269983f4a367a45b9599c89d0b]--&gt;
-&lt;script&gt;fixGalHeight(&#34;gallery1872830&#34;)&lt;/script&gt;
+&lt;script&gt;fixGalHeight(&#34;gallery2917503&#34;)&lt;/script&gt;
 &lt;/div&gt;
 &lt;p&gt;You can show a gallery of plots you make in the Rmd and when people click it,
 it will be enlarged. You can also specify a link for each image.&lt;/p&gt;
@@ -283,7 +282,7 @@ it will be enlarged. You can also specify a link for each image.&lt;/p&gt;
     text_colors = c(&amp;quot;#caffc1&amp;quot;, &amp;quot;black&amp;quot;, &amp;quot;blue&amp;quot;)
 )
 &lt;/code&gt;&lt;/pre&gt;
-&lt;div id=&#34;list-tab6143613&#34; class=&#34;col&#34;&gt;
+&lt;div id=&#34;list-tab9709371&#34; class=&#34;col&#34;&gt;
 &lt;p class=&#34;h4&#34; style=&#34;color: #0275d8; text-align: left;&#34;&gt;Different background and text colors&lt;/p&gt;
 &lt;div&gt;&lt;a
   href=&#34;#&#34; 
@@ -319,7 +318,7 @@ it will be enlarged. You can also specify a link for each image.&lt;/p&gt;
     style = &amp;quot;display: table;&amp;quot;
 )
 &lt;/code&gt;&lt;/pre&gt;
-&lt;table id=&#34;list-table2346973&#34; class=&#34;table table-hover table-href table-striped&#34; style=&#34;display: table;&#34;&gt;
+&lt;table id=&#34;list-table9981148&#34; class=&#34;table table-hover table-href table-striped&#34; style=&#34;display: table;&#34;&gt;
 &lt;caption class=&#34;text-center h2&#34; style=&#34;color: #0275d8;&#34;&gt;Change button color and text color&lt;/caption&gt;
 &lt;thead&gt;
   &lt;tr class=&#34;info&#34;&gt;
@@ -368,7 +367,7 @@ it will be enlarged. You can also specify a link for each image.&lt;/p&gt;
     style = &amp;quot;display: table;&amp;quot;
 )
 &lt;/code&gt;&lt;/pre&gt;
-&lt;table id=&#34;list-table7826614&#34; class=&#34;table table-hover table-href table-striped&#34; style=&#34;display: table;&#34;&gt;
+&lt;table id=&#34;list-table7341719&#34; class=&#34;table table-hover table-href table-striped&#34; style=&#34;display: table;&#34;&gt;
 &lt;caption class=&#34;text-center h2&#34; style=&#34;color: #0275d8;&#34;&gt;Change row name colors and width&lt;/caption&gt;
 &lt;thead&gt;
   &lt;tr class=&#34;info&#34;&gt;
@@ -605,10 +604,10 @@ Rmarkdown documents does not have a server, but you can still add some loaders.&
 “ring,” “roller,” “default,” “ellipsis,” “grid,” “hourglass,” “ripple,” “spinner.”&lt;/p&gt;
 &lt;pre&gt;&lt;code class=&#34;language-r&#34;&gt;cssLoader(height = &amp;quot;100px&amp;quot;)
 &lt;/code&gt;&lt;/pre&gt;
-&lt;div id=&#34;spsloader-594362718&#34; class=&#34;sps-cssloader&#34; style=&#34;height: 100px; width: 100px; display: block; opacity: 1;&#34;&gt;
+&lt;div id=&#34;spsloader-857314629&#34; class=&#34;sps-cssloader&#34; style=&#34;height: 100px; width: 100px; display: block; opacity: 1;&#34;&gt;
 &lt;script&gt;
   $(function(){
-    $(&#34;#spsloader-594362718&#34;).prepend(chooseLoader(&#34;spsloader-594362718&#34;, &#34;default&#34;, &#34;&#34;, &#34;#337ab7&#34;, &#34;100px&#34;, &#34;100px&#34;));
+    $(&#34;#spsloader-857314629&#34;).prepend(chooseLoader(&#34;spsloader-857314629&#34;, &#34;default&#34;, &#34;&#34;, &#34;#337ab7&#34;, &#34;100px&#34;, &#34;100px&#34;));
   });
 &lt;/script&gt;
 &lt;/div&gt;
@@ -616,10 +615,10 @@ Rmarkdown documents does not have a server, but you can still add some loaders.&
 &lt;p&gt;customize it:&lt;/p&gt;
 &lt;pre&gt;&lt;code class=&#34;language-r&#34;&gt;cssLoader(type = &amp;quot;grid&amp;quot;, height = &amp;quot;150px&amp;quot;, color = &amp;quot;orange&amp;quot;)
 &lt;/code&gt;&lt;/pre&gt;
-&lt;div id=&#34;spsloader-917283456&#34; class=&#34;sps-cssloader&#34; style=&#34;height: 150px; width: 150px; display: block; opacity: 1;&#34;&gt;
+&lt;div id=&#34;spsloader-481753296&#34; class=&#34;sps-cssloader&#34; style=&#34;height: 150px; width: 150px; display: block; opacity: 1;&#34;&gt;
 &lt;script&gt;
   $(function(){
-    $(&#34;#spsloader-917283456&#34;).prepend(chooseLoader(&#34;spsloader-917283456&#34;, &#34;grid&#34;, &#34;&#34;, &#34;orange&#34;, &#34;150px&#34;, &#34;150px&#34;));
+    $(&#34;#spsloader-481753296&#34;).prepend(chooseLoader(&#34;spsloader-481753296&#34;, &#34;grid&#34;, &#34;&#34;, &#34;orange&#34;, &#34;150px&#34;, &#34;150px&#34;));
   });
 &lt;/script&gt;
 &lt;/div&gt;
@@ -633,10 +632,10 @@ Rmarkdown documents does not have a server, but you can still add some loaders.&
 )
 &lt;/code&gt;&lt;/pre&gt;
 &lt;button&gt;
-&lt;i id=&#34;spsloader-561273984&#34; class=&#34;sps-cssloader&#34; style=&#34;height: 1.5rem; width: 1.5rem; display: inline-block; opacity: 1;&#34;&gt;
+&lt;i id=&#34;spsloader-369175824&#34; class=&#34;sps-cssloader&#34; style=&#34;height: 1.5rem; width: 1.5rem; display: inline-block; opacity: 1;&#34;&gt;
 &lt;script&gt;
   $(function(){
-    $(&#34;#spsloader-561273984&#34;).prepend(chooseLoader(&#34;spsloader-561273984&#34;, &#34;default&#34;, &#34;&#34;, &#34;#3a7bd5&#34;, &#34;1.5rem&#34;, &#34;1.5rem&#34;));
+    $(&#34;#spsloader-369175824&#34;).prepend(chooseLoader(&#34;spsloader-369175824&#34;, &#34;default&#34;, &#34;&#34;, &#34;#3a7bd5&#34;, &#34;1.5rem&#34;, &#34;1.5rem&#34;));
   });
 &lt;/script&gt;
 &lt;/i&gt;
diff --git a/public/sps/dev/spscomps/server/index.html b/public/sps/dev/spscomps/server/index.html
index f42998408..25fd7350f 100644
--- a/public/sps/dev/spscomps/server/index.html
+++ b/public/sps/dev/spscomps/server/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/dev/spscomps/ui/index.html b/public/sps/dev/spscomps/ui/index.html
index f24247012..fd9978715 100644
--- a/public/sps/dev/spscomps/ui/index.html
+++ b/public/sps/dev/spscomps/ui/index.html
@@ -18,21 +18,21 @@
 <link rel="icon" type="image/png" href="../../../../favicons/android-192x192.png" sizes="192x192">
 
 <title>UI components | sysPipe</title><meta property="og:title" content="UI components" />
-<meta property="og:description" content="\#\# UI Components For most of the UI components, you can view them in the [online Shiny demo{blk}](https://lezhang.shinyapps.io/spsComps). Most but **not all** UI components work in a Rmarkdown document. Here we demostrate how you could use some of them in a Rmarkdown doc. The source code of this document is on [Github{blk}](https://raw." />
+<meta property="og:description" content="For most of the UI components, you can view them in the online Shiny demo{blk}. Most but not all UI components work in a Rmarkdown document. Here we demostrate how you could use some of them in a Rmarkdown doc. The source code of this document is on Github{blk}." />
 <meta property="og:type" content="article" />
 <meta property="og:url" content="/sps/dev/spscomps/ui/" />
 <meta property="article:modified_time" content="2021-05-09T15:49:11-07:00" /><meta property="og:site_name" content="sysPipe" />
 <meta itemprop="name" content="UI components">
-<meta itemprop="description" content="\#\# UI Components For most of the UI components, you can view them in the [online Shiny demo{blk}](https://lezhang.shinyapps.io/spsComps). Most but **not all** UI components work in a Rmarkdown document. Here we demostrate how you could use some of them in a Rmarkdown doc. The source code of this document is on [Github{blk}](https://raw.">
+<meta itemprop="description" content="For most of the UI components, you can view them in the online Shiny demo{blk}. Most but not all UI components work in a Rmarkdown document. Here we demostrate how you could use some of them in a Rmarkdown doc. The source code of this document is on Github{blk}.">
 <meta itemprop="dateModified" content="2021-05-09T15:49:11-07:00" />
-<meta itemprop="wordCount" content="1137">
+<meta itemprop="wordCount" content="1134">
 
 
 
 <meta itemprop="keywords" content="" />
 <meta name="twitter:card" content="summary"/>
 <meta name="twitter:title" content="UI components"/>
-<meta name="twitter:description" content="\#\# UI Components For most of the UI components, you can view them in the [online Shiny demo{blk}](https://lezhang.shinyapps.io/spsComps). Most but **not all** UI components work in a Rmarkdown document. Here we demostrate how you could use some of them in a Rmarkdown doc. The source code of this document is on [Github{blk}](https://raw."/>
+<meta name="twitter:description" content="For most of the UI components, you can view them in the online Shiny demo{blk}. Most but not all UI components work in a Rmarkdown document. Here we demostrate how you could use some of them in a Rmarkdown doc. The source code of this document is on Github{blk}."/>
 
 
 <script type="application/javascript">
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../../"
   data-offline-search-max-results="10"
 >
@@ -1162,11 +1162,10 @@ <h1>UI components</h1>
 <link href="../../../../rmarkdown-libs/spsComps-css-loader-css/css/css_loader.css" rel="stylesheet" />
 <script src="../../../../rmarkdown-libs/spsComps-css-loader-js/js/sps_cssloader.js"></script>
 <link href="../../../../rmarkdown-libs/spsComps-css-loader-css/css/css_loader.css" rel="stylesheet" />
-\#\# UI Components
-For most of the UI components, you can view them in the [online Shiny demo{blk}](https://lezhang.shinyapps.io/spsComps).
-Most but **not all** UI components work in a Rmarkdown document. Here we demostrate
+<p>For most of the UI components, you can view them in the <a href="https://lezhang.shinyapps.io/spsComps">online Shiny demo{blk}</a>.
+Most but <strong>not all</strong> UI components work in a Rmarkdown document. Here we demostrate
 how you could use some of them in a Rmarkdown doc. The source code of this
-document is on [Github{blk}](https://raw.githubusercontent.com/systemPipeR/systemPipeR.github.io/main/content/en/sps/dev/spscomps/ui.Rmd).
+document is on <a href="https://raw.githubusercontent.com/systemPipeR/systemPipeR.github.io/main/content/en/sps/dev/spscomps/ui.Rmd">Github{blk}</a>.</p>
 <h2 id="load-package">load package</h2>
 <p>To start to use <code>spsComps</code>, load it in your Shiny app file or Rmarkdown file</p>
 <pre><code class="language-r">library(spsComps)
@@ -1229,46 +1228,46 @@ <h2 id="gallery"><code>gallery</code></h2>
   enlarge_method = &quot;modal&quot;
 )
 </code></pre>
-<div id="gallery1872830" class="col sps-gallery" style="">
+<div id="gallery2917503" class="col sps-gallery" style="">
 <p class="text-center h2" style="color: #0275d8;">Gallery</p>
-<div class="row" style="  margin: 10px;"><div  id=gallery1872830-1 class="col-sm-4 sps-tab-link" style="right: 1px;">
+<div class="row" style="  margin: 10px;"><div  id=gallery2917503-1 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/1.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-1")
+    onclick=galEnlarge("#gallery2917503-1")
   >
   <a href="https://github.com/lz100/spsComps/blob/master/img/1.jpg?raw=true" ><p class="text-center h4 ">p1</p></a>
-</div> <div  id=gallery1872830-2 class="col-sm-4 sps-tab-link" style="right: 1px;">
+</div> <div  id=gallery2917503-2 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/2.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-2")
+    onclick=galEnlarge("#gallery2917503-2")
   >
   <a href="https://github.com/lz100/spsComps/blob/master/img/2.jpg?raw=true" ><p class="text-center h4 ">p2</p></a>
-</div> <div  id=gallery1872830-3 class="col-sm-4 sps-tab-link" style="right: 1px;">
+</div> <div  id=gallery2917503-3 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/3.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-3")
+    onclick=galEnlarge("#gallery2917503-3")
   >
   <a  ><p class="text-center h4 gallery-nohover">&nbsp;</p></a>
-</div> <div  id=gallery1872830-4 class="col-sm-4 sps-tab-link" style="right: 1px;">
+</div> <div  id=gallery2917503-4 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/4.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-4")
+    onclick=galEnlarge("#gallery2917503-4")
   >
   <a href="https://github.com/lz100/spsComps/blob/master/img/4.jpg?raw=true" ><p class="text-center h4 ">p4</p></a>
-</div> <div  id=gallery1872830-5 class="col-sm-4 sps-tab-link" style="right: 1px;">
+</div> <div  id=gallery2917503-5 class="col-sm-4 sps-tab-link" style="right: 1px;">
   <img
     src="https://github.com/lz100/spsComps/blob/master/img/5.jpg?raw=true" class="img-gallery"
     height=300 width=400
     style="width: 100%;"
-    onclick=galEnlarge("#gallery1872830-5")
+    onclick=galEnlarge("#gallery2917503-5")
   >
   <a href="https://github.com/lz100/spsComps/blob/master/img/5.jpg?raw=true" ><p class="text-center h4 ">p5</p></a>
 </div></div>
@@ -1279,7 +1278,7 @@ <h2 id="gallery"><code>gallery</code></h2>
 <div class="gallery-caption"></div>
 </div>
 <!--/SHINY.SINGLETON[7fdec3ac87205a269983f4a367a45b9599c89d0b]-->
-<script>fixGalHeight("gallery1872830")</script>
+<script>fixGalHeight("gallery2917503")</script>
 </div>
 <p>You can show a gallery of plots you make in the Rmd and when people click it,
 it will be enlarged. You can also specify a link for each image.</p>
@@ -1344,7 +1343,7 @@ <h3 id="some-colorful-buttons-hreftab">Some colorful buttons <code>hrefTab</code
     text_colors = c(&quot;#caffc1&quot;, &quot;black&quot;, &quot;blue&quot;)
 )
 </code></pre>
-<div id="list-tab6143613" class="col">
+<div id="list-tab9709371" class="col">
 <p class="h4" style="color: #0275d8; text-align: left;">Different background and text colors</p>
 <div><a
   href="#" 
@@ -1380,7 +1379,7 @@ <h3 id="a-table-colorful-buttons-hreftable">A table colorful buttons <code>hrefT
     style = &quot;display: table;&quot;
 )
 </code></pre>
-<table id="list-table2346973" class="table table-hover table-href table-striped" style="display: table;">
+<table id="list-table9981148" class="table table-hover table-href table-striped" style="display: table;">
 <caption class="text-center h2" style="color: #0275d8;">Change button color and text color</caption>
 <thead>
   <tr class="info">
@@ -1429,7 +1428,7 @@ <h3 id="a-table-colorful-buttons-hreftable">A table colorful buttons <code>hrefT
     style = &quot;display: table;&quot;
 )
 </code></pre>
-<table id="list-table7826614" class="table table-hover table-href table-striped" style="display: table;">
+<table id="list-table7341719" class="table table-hover table-href table-striped" style="display: table;">
 <caption class="text-center h2" style="color: #0275d8;">Change row name colors and width</caption>
 <thead>
   <tr class="info">
@@ -1666,10 +1665,10 @@ <h4 id="default-loaders">Default loaders</h4>
 “ring,” “roller,” “default,” “ellipsis,” “grid,” “hourglass,” “ripple,” “spinner.”</p>
 <pre><code class="language-r">cssLoader(height = &quot;100px&quot;)
 </code></pre>
-<div id="spsloader-594362718" class="sps-cssloader" style="height: 100px; width: 100px; display: block; opacity: 1;">
+<div id="spsloader-857314629" class="sps-cssloader" style="height: 100px; width: 100px; display: block; opacity: 1;">
 <script>
   $(function(){
-    $("#spsloader-594362718").prepend(chooseLoader("spsloader-594362718", "default", "", "#337ab7", "100px", "100px"));
+    $("#spsloader-857314629").prepend(chooseLoader("spsloader-857314629", "default", "", "#337ab7", "100px", "100px"));
   });
 </script>
 </div>
@@ -1677,10 +1676,10 @@ <h4 id="default-loaders">Default loaders</h4>
 <p>customize it:</p>
 <pre><code class="language-r">cssLoader(type = &quot;grid&quot;, height = &quot;150px&quot;, color = &quot;orange&quot;)
 </code></pre>
-<div id="spsloader-917283456" class="sps-cssloader" style="height: 150px; width: 150px; display: block; opacity: 1;">
+<div id="spsloader-481753296" class="sps-cssloader" style="height: 150px; width: 150px; display: block; opacity: 1;">
 <script>
   $(function(){
-    $("#spsloader-917283456").prepend(chooseLoader("spsloader-917283456", "grid", "", "orange", "150px", "150px"));
+    $("#spsloader-481753296").prepend(chooseLoader("spsloader-481753296", "grid", "", "orange", "150px", "150px"));
   });
 </script>
 </div>
@@ -1694,10 +1693,10 @@ <h4 id="default-loaders">Default loaders</h4>
 )
 </code></pre>
 <button>
-<i id="spsloader-561273984" class="sps-cssloader" style="height: 1.5rem; width: 1.5rem; display: inline-block; opacity: 1;">
+<i id="spsloader-369175824" class="sps-cssloader" style="height: 1.5rem; width: 1.5rem; display: inline-block; opacity: 1;">
 <script>
   $(function(){
-    $("#spsloader-561273984").prepend(chooseLoader("spsloader-561273984", "default", "", "#3a7bd5", "1.5rem", "1.5rem"));
+    $("#spsloader-369175824").prepend(chooseLoader("spsloader-369175824", "default", "", "#3a7bd5", "1.5rem", "1.5rem"));
   });
 </script>
 </i>
diff --git a/public/sps/dev/spsutil/index.html b/public/sps/dev/spsutil/index.html
index 8282846b9..b46778f3d 100644
--- a/public/sps/dev/spsutil/index.html
+++ b/public/sps/dev/spsutil/index.html
@@ -517,7 +517,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -549,7 +549,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/index.html b/public/sps/index.html
index 256ca25c3..d69368658 100644
--- a/public/sps/index.html
+++ b/public/sps/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/install/index.html b/public/sps/install/index.html
index 7ca743659..0cf8ae512 100644
--- a/public/sps/install/index.html
+++ b/public/sps/install/index.html
@@ -514,7 +514,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -546,7 +546,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/intro/index.html b/public/sps/intro/index.html
index d6f25eba6..70c85bba0 100644
--- a/public/sps/intro/index.html
+++ b/public/sps/intro/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/modules/ggplot/index.html b/public/sps/modules/ggplot/index.html
index 5bad77a06..8b22a2739 100644
--- a/public/sps/modules/ggplot/index.html
+++ b/public/sps/modules/ggplot/index.html
@@ -511,7 +511,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -543,7 +543,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/modules/index.html b/public/sps/modules/index.html
index cf9159068..62b9c6727 100644
--- a/public/sps/modules/index.html
+++ b/public/sps/modules/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/modules/rnaseq/index.html b/public/sps/modules/rnaseq/index.html
index 8316cc420..d5a850b0d 100644
--- a/public/sps/modules/rnaseq/index.html
+++ b/public/sps/modules/rnaseq/index.html
@@ -517,7 +517,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -549,7 +549,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/modules/workflow/index.html b/public/sps/modules/workflow/index.html
index 3a06feded..917c300af 100644
--- a/public/sps/modules/workflow/index.html
+++ b/public/sps/modules/workflow/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/quick_start/index.html b/public/sps/quick_start/index.html
index 41b5a7aa2..ffe71aaac 100644
--- a/public/sps/quick_start/index.html
+++ b/public/sps/quick_start/index.html
@@ -517,7 +517,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -549,7 +549,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/screenshots/index.html b/public/sps/screenshots/index.html
index 290c90277..3e705d8da 100644
--- a/public/sps/screenshots/index.html
+++ b/public/sps/screenshots/index.html
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/sps/sps_funcs/index.html b/public/sps/sps_funcs/index.html
index 68ae960c6..a29762d4f 100644
--- a/public/sps/sps_funcs/index.html
+++ b/public/sps/sps_funcs/index.html
@@ -25,7 +25,7 @@
 <meta itemprop="name" content="SPS Functions">
 <meta itemprop="description" content=".td-content li a { font-size: 1.5rem; }  SPS Function Reference Manuals There are many more useful functions one could use from SPS core and sub packages. We could not cover all the details in this manual. To take a look at these functions, click links below to go to the function reference manual of each package. Detailed function usage, example code, and example running results are shown inside.">
 <meta itemprop="dateModified" content="2021-03-02T23:05:00-08:00" />
-<meta itemprop="wordCount" content="85">
+<meta itemprop="wordCount" content="86">
 
 
 
@@ -508,7 +508,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -540,7 +540,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -1010,20 +1010,35 @@ <h2 id="sps-function-reference-manuals">SPS Function Reference Manuals</h2>
 We could not cover all the details in this manual. To take a look at these functions,
 click links below to go to the function reference manual of each package. Detailed
 function usage, example code, and example running results are shown inside.</p>
-<ul>
-<li>
-<p><a href="../../sps/funcs/sps/reference">systemPipeShiny &ndash; main package</a></p>
-</li>
-<li>
-<p><a href="../../sps/funcs/spscomps/reference">spsComps &ndash; SPS components</a></p>
-</li>
-<li>
-<p><a href="../../sps/funcs/drawer/reference">drawer &ndash; image editing tool</a></p>
-</li>
-<li>
-<p><a href="../../sps/funcs/spsutil/reference">spsUtil &ndash; utility functions </a></p>
-</li>
-</ul>
+<table>
+<thead>
+<tr>
+<th>Package</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><a href="../../sps/funcs/sps/reference">systemPipeShiny</a></td>
+<td>main package</td>
+</tr>
+<tr>
+<td><a href="../../sps/funcs/spscomps/reference">spsComps</a></td>
+<td>SPS components</td>
+</tr>
+<tr>
+<td><a href="../../sps/funcs/drawer/reference">drawer</a></td>
+<td>image editing tool</td>
+</tr>
+<tr>
+<td><a href="../../sps/funcs/spsutil/reference">spsUtil</a></td>
+<td>utility functions</td>
+</tr>
+</tbody>
+</table>
+<style>
+table {font-size: 1.5rem}
+</style>
 <hr>
 
 	
diff --git a/public/training/conferences/index.html b/public/training/conferences/index.html
index 8f0d92761..74203433d 100644
--- a/public/training/conferences/index.html
+++ b/public/training/conferences/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/training/index.html b/public/training/index.html
index 644c12312..14cd1ec7c 100644
--- a/public/training/index.html
+++ b/public/training/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../"
   data-offline-search-max-results="10"
 >
diff --git a/public/training/presentations/index.html b/public/training/presentations/index.html
index 4ed7c6428..83f33e0e4 100644
--- a/public/training/presentations/index.html
+++ b/public/training/presentations/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
diff --git a/public/training/videos/index.html b/public/training/videos/index.html
index ba04d20b5..2171d51b2 100644
--- a/public/training/videos/index.html
+++ b/public/training/videos/index.html
@@ -503,7 +503,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >
@@ -535,7 +535,7 @@
   aria-label="Search this site…"
   autocomplete="off"
   
-  data-offline-search-index-json-src="../../offline-search-index.e37415c6011b86533435a63f46962da6.json"
+  data-offline-search-index-json-src="../../offline-search-index.8b789866927612b8a1270b2bdea1abcb.json"
   data-offline-search-base-href="../../"
   data-offline-search-max-results="10"
 >