Skip to content

daroczig/CEU-DV4

2021/2022
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 

This is the R script/materials repository of the "Data Visualization 4: Data Visualization in Production with Shiny" course in the 2021/2022 Spring term, part of the MSc in Business Analytics at CEU. For the previous edition, see 2019/2020 Spring and 2020/2021 Spring.

Table of Contents

Syllabus

Please find in the syllabus folder of this repository.

Schedule

2 x 2 x 150 mins between on April 20 and May 2, 2022.

Day 1

Introduction to Shiny by Mihaly Orsos: https://github.com/misrori/DV4

Day 2

Overview on the previous week's materials by Mihaly Orsos:

  • basics of ui.R and server.R
  • HTML tags
  • reactive functions
  • action button
  • renderUI
  • UI layouts
    • basic UI elements
    • Shiny themes
    • shinydashboard
  • Password-protected applications
  • Lottery app

Countdown-timer app

This week, we build a countdown timer app, similar to http://count-down-timer.tk, and deploy using ShinyProxy.io

  1. App wireframe with a static UI:

    ui.R
    library(shiny)
    
    ui <- fluidPage(
        h1('Data Visualization 4'),
        h2('Data Visualization in Production with Shiny')
    )
    server.R
    library(shiny)
    server <- function(input, output) {
    
    }
  2. Move content generation to the backend: f0b53ee

  3. Add current time (to be later used as the baseline for the countdown timer): 5dd6724

  4. Actual countdown from the scheduled time: 5f91927

  5. Add background: 79fbe20

  6. CSS tweaks to center the timer: d1df52d

  7. Colorize timer when scheduled time is due: b74aead

  8. Show time-zone information: c23d1fe

  9. Move defaults to reactive values and let user updated via a modal window at d9cfb6c and get rid of the settingsModal function and just pass the modalDialog to showModal: acb815d

  10. Add time picker for the scheduled time: acb815d

  11. Better design for the settings button: 4b6e7e3..d0efbe7

  12. Pass settings as URL parameters: dbb3ed8

  13. Simplify ui.R by a single call to renderUI: 2d468e7

  14. Fix subtitle and schedule settings update: b416dd5

  15. Add support for timezone setting: 8a9be0e

  16. Ask for consent before using the app: 798dd68

Final app:

ui.R
library(shiny)
library(shinyWidgets)
library(particlesjs)

ui <- basicPage(

    tags$head(
        tags$link(rel = "stylesheet", type = "text/css", href = "app.css")
    ),
    uiOutput('app')

)
www/app.css
.center {
    position: absolute;
    top: 50%;
    left: 50%;
    transform: translate(-50%, -50%);
    text-align: center;
    background-color: #00000042;
    padding: 25px 50px;
    border-radius: 25px;
}

.datepicker{
    z-index:1151 !important;
}

#settings_show {
    position: absolute;
    top: 25px;
    right: 25px;
    color: black;
}
server.R
library(shiny)
library(shinyWidgets)
library(lubridate)

server <- function(input, output, session) {

    settings <- reactiveValues(
        title = 'Data Visualization 4',
        subtitle = 'Data Visualization in Production with Shiny',
        schedule = '2022-05-02 13:30:00',
        timezone = Sys.timezone()
    )

    output$countdown <- renderUI({
        invalidateLater(500)
        schedule <- ymd_hms(settings$schedule, tz = settings$timezone)
        color <- ifelse(schedule > Sys.time(), 'black', 'red')
        remaining <- span(
            round(as.period(abs(schedule - Sys.time()))),
            style = paste('color', color, sep = ':'))
        div(
            h1(settings$title),
            h2(settings$subtitle),
            h3('starts in'),
            h1(tags$b(remaining)),
            h4(paste('at', settings$schedule, settings$timezone)),
            class = 'center')
    })

    ## load settings from URL query params
    observe({
        query <- parseQueryString(session$clientData$url_search)
        for (v in c('title', 'subtitle', 'schedule', 'timezone')) {
            if (!is.null(query[[v]])) {
                settings[[v]] <- query[[v]]
            }
        }
    })

    ## override settings from modal
    observeEvent(input$settings_show, {
        showModal(modalDialog(
            textInput(
                'title', 'Title',
                value = settings$title),
            textInput(
                'subtitle', 'Subtitle',
                value = settings$subtitle),
            airDatepickerInput(
                'schedule', 'Time',
                value = as.POSIXct(settings$schedule),
                timepicker = TRUE),
            selectInput(
                'timezone', 'Timezone',
                choices = OlsonNames(),
                selected = settings$timezone),
            footer = tagList(actionButton('settings_update', 'Update'))
        ))
    })
    observeEvent(input$settings_update, {
        settings$title <- input$title
        settings$subtitle <- input$subtitle
        settings$schedule <- input$schedule
        settings$timezone <- input$timezone
        removeModal()
    })

    ## gdpr
    showModal(modalDialog(
        p('Click the below button you consent to ...'),
        footer = tagList(actionButton('consent', 'OK'))
    ))
    observeEvent(input$consent, {
        output$app <- renderUI({
            list(
                particles(),
                actionBttn('settings_show', 'Settings',
                           icon = icon('gear'),
                           style = 'material-circle'),
                uiOutput('countdown')
            )
        })
        removeModal()
    })

}

Further ideas to improve the app:

  • get timezone from visitor's browser setting / locale
  • configure theme (colors, layout, background etc)
  • ads!!!
Shiny Server
  1. 💪 Install R packages as a system user:

     ## install as a binary when possible
     sudo apt-get install r-cran-dplyr r-cran-quantmod r-cran-xml r-cran-tidyr r-cran-igraph r-cran-lubridate r-cran-psych r-cran-broom r-cran-yaml r-cran-htmlwidgets r-cran-shiny
    
     ## install from CRAN when binary is not available
     sudo R -e "withr::with_libpaths(new = '/usr/local/lib/R/site-library', install.packages(c('highcharter', 'shinyWidgets'), repos='https://cran.rstudio.com/'))"
    
     ## some R packages are not even on CRAN, so let's install from GitHub
     sudo Rscript -e "library(devtools);withr::with_libpaths(new = '/usr/local/lib/R/site-library', install_github('dreamRs/particlesjs', upgrade_dependencies = FALSE))"
    
  2. 💪 Install Shiny Server from https://rstudio.com/products/shiny/download-server/ubuntu/:

     sudo apt-get install gdebi-core
     wget https://download3.rstudio.org/ubuntu-18.04/x86_64/shiny-server-1.5.18.987-amd64.deb
     sudo gdebi shiny-server-1.5.18.987-amd64.deb
    
  3. 💪 Edit site_dir in shiny-server.conf to point to the /home/ceu folder via

     sudo mcedit /etc/shiny-server/shiny-server.conf
     sudo systemctl restart shiny-server
    
  4. Visit Shiny Server on port 3838 from your browser

  5. 💪 Always keep logs -- set this in the Shiny Server config & restart service as per https://docs.rstudio.com/shiny-server/#logging-and-analytics:

     preserve_logs true;
    

    Optionally, redirect all logs to the same file by injecting an environment variable in /etc/systemd/system/shiny-server.service by adding this line below the other Environment= line:

     Environment="SHINY_LOG_STDERR=1"
    
  6. To keep an eye on logs (test with making a typo in the app on purpose):

     sudo tail -f /var/log/shiny-server.log
    
  7. Add ui.R and server.R files (along with global.R and other stuff in the www folder) to directories created in /home/ceu

Note, that Shiny Server has some limitations (eg scaling to multiple users, some headers removed by the Node.js wrapper) -- so you might consider either the Pro version, other RStudio products or eg the below-mentioned Shiny app manager daemon for using Shiny in production at scale.

  1. 💪 Run behind a proxy to be able to access on the standard HTTP port:

    http {
    
      map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
      }
    
      server {
        listen 80;
    
        rewrite ^/shiny$ $scheme://$http_host/shiny/ permanent;
        location /shiny/ {
          rewrite ^/shiny/(.*)$ /$1 break;
          proxy_pass http://localhost:3838;
          proxy_redirect / $scheme://$http_host/shiny/;
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection $connection_upgrade;
          proxy_read_timeout 20d;
          proxy_buffering off;
        }
      }
    }

The above steps can be covered by starting a new t3.micro instance using the dv4 AMI and dv4 security group.

Shinyproxy.io
  1. Get familiar with Docker:

  2. 💪 Install Docker

    ## get dependencies
    sudo apt-get install -y apt-transport-https ca-certificates curl gnupg-agent software-properties-common
    
    ## import Docker's official GPG key
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
    
    ## add the external, official Docker apt repo for most recent release
    sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
    
    ## download list of available packages and install Docker
    sudo apt update
    sudo apt install -y docker-ce docker-ce-cli containerd.io
  3. Test Docker

    sudo docker run --rm hello-world
    sudo docker image rm hello-world
  4. 💪 ShinyProxy needs to connect to the Docker daemon, so let's open up a port for that

    • check /etc/systemd/system/docker.service.d/override.conf eg via sudo mcedit and paste the below content if not already there

      [Service]
      ExecStart=
      ExecStart=/usr/bin/dockerd -H unix:// -D -H tcp://127.0.0.1:2375
    • restart Docker

      sudo systemctl daemon-reload
      sudo systemctl restart docker
  5. 💪 Make sure Java is installed:

    sudo apt install -y openjdk-8-jdk-headless
  6. 💪 Install ShinyProxy

    wget -O /tmp/shinyproxy.deb https://www.shinyproxy.io/downloads/shinyproxy_2.6.1_amd64.deb
    sudo dpkg -i /tmp/shinyproxy.deb
  7. 💪 Configure ShinyProxy at /etc/shinyproxy/application.yml

    proxy:
      title: CEU Business Analytics Shiny Proxy
      logo-url: https://www.ceu.edu/sites/default/files/media/user-5/ceulogo_0_1.jpg
      landing-page: /
      heartbeat-rate: 10000
      heartbeat-timeout: 60000
      port: 8080
      docker:
        cert-path: /home/none
        url: http://localhost:2375
        port-range-start: 20000
      specs:
      - id: 01_hello
        display-name: Hello Application
        description: Application which demonstrates the basics of a Shiny app
        container-cmd: ["R", "-e", "shinyproxy::run_01_hello()"]
        container-image: openanalytics/shinyproxy-demo
    logging:
      file:
        /var/log/shinyproxy.log

    Optionally make that editable by the ceu user for easier access from RStudio for the time being:

    sudo chown ceu:ceu /etc/shinyproxy/application.yml
  8. 💪 Restart ShinyProxy

    sudo systemctl restart shinyproxy
  9. 💪 Set up yet-another-proxy so that the apps can be accessed over the standard HTTP/80 port

    1. Install nginx

      sudo apt install -y nginx
    2. Then we need to edit the main site's configuration at /etc/nginx/sites-enabled/default

      server {
          listen 80;
          location / {
              proxy_pass http://127.0.0.1:8080/;
              proxy_http_version 1.1;
              proxy_set_header Upgrade $http_upgrade;
              proxy_set_header Connection "upgrade";
              proxy_read_timeout 600s;
              proxy_redirect off;
              proxy_set_header Host $http_host;
              proxy_set_header X-Real-IP $remote_addr;
              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_set_header X-Forwarded-Protocol $scheme;
          }
      }
      
    3. Restart nginx

      sudo systemctl restart nginx
    4. Note that if you want to deploy under a specific path instead of the root path, you need to set server.servlet.context-path in the Shinyproxy configuration, then update the above Nginx config as well.

  10. Visit your EC2 box's IP address at http://your.ip.address

  11. Need to download the Docker image specified in the application.yml for the example app

    sudo docker pull openanalytics/shinyproxy-demo
  12. Let's build a new Docker image for our application! For this end, we need to define the build instructions in a Dockerfile placed in /home/ceu/countdown/Dockerfile

    FROM rocker/shiny
    
    RUN install2.r shinyWidgets lubridate remotes
    RUN installGithub.r -u FALSE dreamRs/particlesjs
    
    RUN mkdir /app
    COPY *.R /app/
    
    CMD ["R", "-e", "shiny::runApp('/app')"]

    And then build the Docker image based on the above:

    ```sh
    sudo docker build -t countdown .
    ```
    

    Now we can run a Docker container based on this image on the command-line:

    ```sh
    sudo docker run --rm -ti countdown
    ```
    
  13. Update the ShinyProxy config to include the above Dockerized app

    - id: countdown
      display-name: Countdown Timer
      description: Yeah, this is a countdown timer
      container-image: countdown
  14. Debug why it's not running?

    sudo chown shinyproxy:shinyproxy /var/log/shinyproxy.log
    sudo systemctl restart shinyproxy

    Need to update the Dockerfile (or config) to expose on the right port:

    EXPOSE 3838
    CMD ["R", "-e", "shiny::runApp('/app', port = 3838, host = '0.0.0.0')"]
  15. Don't forget to also add the www folder with the required CSS!

    COPY www/app.css /app/www/
  16. Authentication as per https://www.shinyproxy.io/documentation/configuration/#simple-authentication

...
proxy:
  authentication: simple
  ...
  users:
  - name: ceu
    password: ceudata
    group: users
  ...
  specs:
  - ...
    groups: users

My subjective "Best Damn Packages for Shiny" list

For the user interface:

  • shinyjs
  • shinyWidgets
  • DT
  • dygraphs
  • ggiraph

For the back-end:

  • logger
  • dbr
  • botor

:)

Home Assignment

Create and deploy a Shiny application as described below. You can use any dataset you like. If you have no preferred data source(s) to use, check the below ideas. In case of any question, reach out on Slack or open a GitHub ticket.

Minimal project (for up to "B" grade):

  • You need to create a Shiny application with shinydashboard layout using a sidebar menu
  • Write at least one helper function and put into the global.R file to make it accessible for both ui.R and server.R
  • Add at least 3 input elements using at least 2 different methods (eg dropdown, slider, checkbox)
  • Add at least 3 different UI elements (eg text, table, map, UI, info and valueboxes)
  • Use at least 2 menu items
  • Use reactive values
  • The change of the input elements have to change the UI elements showed to the user
  • Deploy the app into shinyapps.io, upload the code into Moodle with a reference to your shinyapps.io application

Suggested project (for up to "B" grade):

  • Implement the application described in the above steps
  • Add at least 5 (instead of 3) different UI elements, including at least one one from htmlwidgets.org
  • Deploy the app via shinyproxy (covered in the second day of the course)

Some ideas for data source:

Deadline: May 23, 2022

Suggested Reading

Home Assignment

To be uploaded.

Contact

File a GitHub ticket.

About

Materials for the "Data Visualization 4" class at CEU

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published