How to combine LDAvis output into a shiny app

[cho7tom]

Dear Carson,

I recently discovered the LDAvis package and I am highly interested in the outputs you have been able to generate in a browser via the serVis function (I am no js knowledge so it's great to be able to leverage such tools in my analysis).
Working part of a data scientist team, I would like to be able to integrate the serVis output into some of our own shiny apps, in order to enrich our text mining capabilities.
Is that feasible? If yes, which render* and which *Output should be used?
Congratulations again for this great package!

Best regards,

Thomas

[cpsievert]

Hi Thomas. I'm glad you brought this up.

The JavaScript code isn't very modular, which makes it hard to extract/reuse components of the viz, but if you just want to embed the entire output in a shiny app, there is hope.

That being said, if you want the LDAvis output to react to shiny inputs, it probably won't be very quick to respond simply because a good amount of computation is required in order to createJSON().

Anyway, I started this pull request to make it easy to embed LDAvis in a shiny app (or interactive rmarkdown document). It'd be great if you could help us test it out and provide feedback.

devtools::install_github("cpsievert/LDAvis@shiny")
rmarkdown::run(system.file("examples/rmarkdown.Rmd", package = "LDAvis"))

[cpsievert]

This is now possible @cho7tom. See here

[cho7tom]

Dear Carson,

Thank you very much for your quick solution!
All the best,

Regards

[mfost]

Quick question:
How do visOutput() and renderVis() work?
I'm trying to do something like the following, but R isn't recognizing the two functions. Any suggestions?

ui.R:
shinyUI(
fluidPage(visOutput('myChart')))

server.R:
library(LDAvis)
library(shiny)
shinyServer(function(input, output, session) {
output$myChart <- renderVis({
with(TwentyNewsgroups,
createJSON(phi, theta, doc.length, vocab, term.frequency,
R = input$nTerms))})
}

I appreciate any help you can offer.

[cpsievert]

You're missing a couple things in your code @mfost (most importantly the nTerms input).

I just put up a mini example here

You can run this shiny app directly by doing

shiny::runGitHub("LDAvis", "cpsievert", subdir = "inst/examples/shiny")

[mfost]

Thank you for your help.
I realized that I had switched to the static version of LDAvis instead of the developmental version.

[ghost]

Error in normalizePath(directoryPath, mustWork = TRUE) :
path[1]="": No such file or directory

I got this error when I run the app. Hope you could help me with this.

[vamshibandari]

any comment on the above even am getting the same error

[mnfost]

This problem has been addressed in different R visualization packages.
ramnathv/rCharts#188

It seems that you should make sure the library names you are referencing are in the correct case.
Example:

library(LDAvis)

In server.R:
output$myChart <- renderVis({createJSON(...)})

In ui.R:
visOutput('myChart')

[vamshibandari]

server.r

library(LDAvis)
library(shiny)
shinyServer(function(input, output, session) {
output$a <- renderVis({
with(Earning_call,
createJSON(phi = Earning_call$phi,
theta = Earning_call$theta,
doc.length = Earning_call$doc.length,
vocab = Earning_call$vocab,
term.frequency = Earning_call$term.frequency,
R = input$nTerms))})
})

Ui.r

library(LDAvis)
shinyUI(
fluidPage(
sliderInput("nTerms", "Number of terms to display", min = 20, max = 40, value = 30),
visOutput("a")
)
)

still after looking in to the library names(correcting cases) am getting the same error.

ERROR: path[1]="": The filename, directory name, or volume label syntax is incorrect.

earning call is the output of

Earning_call <- list(phi = phi,
theta = theta,
doc.length = doc.length,
vocab = vocab,
term.frequency = term.frequency)

[mnfost]

Unfortunately, I am unable to replicate your issue.

A few other threads mention to use the development version of the package devtools::install_github("cpsievert/LDAvis")

This also solved my issue from several months ago, so I suggest trying that and also restarting R.

[dieko95]

I got the following error (No idea of the solution):

Warning: Error in stats::cmdscale: NA values not allowed in 'd'
Stack trace (innermost first):
    81: stats::cmdscale
    80: mds.method
    79: createJSON
    78: func [C:\Users\Diego\Dropbox\R\Connexa\lda_vis/server.R#31]
    77: origRenderFunc
    76: output$distPlot
     1: runApp

Server.R

library(shiny)
library(lda)
library(LDAvis)
library(dplyr)


shinyServer(function(input, output) {
  
  
  output$distPlot <- renderVis({
    q1DcastLDA<-read.csv("q1DcastLDA.csv")
    doc.length<-read.csv("doc.length.csv",stringsAsFactors = FALSE)
    
    modelo<-LDA(q1DcastLDA,k = input$num)
    
    phi <- as.matrix(posterior(modelo)$terms) 
    theta <- as.matrix(posterior(modelo)$topics)
    vocab <- colnames(phi)
    doc.length.final <- sapply(strsplit(doc.length$name, "\\s+"),length)
    freq_matrix<-(colSums(q1DcastLDA))
    
    createJSON(phi = as.matrix(posterior(modelo)$terms) , theta = theta,
               vocab = vocab,
               doc.length = doc.length.final,
               term.frequency = freq_matrix)
  })
  
  
})

ui.R

library(shiny)
library(LDAvis)


shinyUI(fluidPage(
  img(src="connexa.png", height = 100, width = 400, align = "center"),
  # Application title
  h1("Recomendaciones de Grupos"),
  
    mainPanel(
      sliderInput("num",label = "Numero de Topicos", min = 3, max = 20, value = 13),
      # numericInput("num", label = h3("Numeric input"), value = 13),
      visOutput("distPlot")
      )
      
    )
)

Thanks for any commentary !!