performance times and memory usage?

[ellisp]

With a more complex treemap - say one with 4000 rows - the d3tree2 function takes impractibly long to convert from a treemap object. The final result renders fine - and not too slowly (just) on screen - but for practical purposes you want to create it once-off and save it; you certainly couldn't create it on the fly in a Shiny app. Also, it seems to take a surprisingly large amount of RAM, which makes me wonder if there is some kind of efficiency to be found somewhere. Example below is similar in size to a real world dataset for which the end result is ok, just very slow to generate. On my (not particularly high powered) machine the example below takes 66 seconds to create the original treemap object, 30 minutes to convert it to a d3 tree, and about 12 seconds to actually render it in a browser. The memory utilised shoots straight up to 4GB as soon as d3tree2() starts doing its thing, and then stays at that high level until R is shut down even though there are no objects of anywhere near that size in the R workspace.

library(treemap)
library(d3treeR)
library(dplyr)

Var1 <- letters[1:26]
Var2 <- LETTERS[1:26]
Var3 <- c("Tiger", "Lion", "Bear", "Penguin", "Eagle", "Aardvark")

biggish <- expand.grid(Var1, Var2, Var3)
n <- nrow(biggish)
biggish$Size <- abs(rnorm(n, 12, 3))
biggish$Colour <- rnorm(n, 0, 1)

system.time(
   tm <- treemap(biggish, index = c("Var1", "Var2", "Var3"), 
            vSize = "Size", vColor = "Colour",
            type = "value", palette = "Spectral",
            fun.aggregate="weighted.mean")
   )

system.time(
   tmpd3 <- d3tree2(tm)
   )   

[timelyportfolio]

This can definitely be a problem, since now that I also extract the legend with treemap we get two grid draws. I will try to flesh out the data.frame, data.table, data.tree integration for bigger data but then we will lose all the treemap niceness. I'll also peek a little more into the treemap code to see if draw can be separated from the data preparation.

[timelyportfolio]

Using lineprof on the treemap piece, we can easily see the bottleneck.

[timelyportfolio]

So, now that @mtennekes has added a draw argument, we can speed up your example considerably. Now, the big hangup on the d3treeR is the data.tree conversion. Not real sure how to speed that up, but I'll dig a little. Also, the extract_legend function in d3treeR will no longer fail when a legend does not exist (see 862afbb).

library(treemap)
library(d3treeR)
library(dplyr)
#library(lineprof)

Var1 <- letters[1:26]
Var2 <- LETTERS[1:26]
Var3 <- c("Tiger", "Lion", "Bear", "Penguin", "Eagle", "Aardvark")

biggish <- expand.grid(Var1, Var2, Var3)
n <- nrow(biggish)
biggish$Size <- abs(rnorm(n, 12, 3))
biggish$Colour <- rnorm(n, 0, 1)

#lp<-lineprof(
tm <- treemap(biggish, index = c("Var1", "Var2", "Var3"), 
              vSize = "Size", vColor = "Colour",
              type = "value", palette = "Spectral",
              fun.aggregate="weighted.mean",draw = FALSE)
#)

#lp2 <- lineprof(
  d3tree2(tm)
#)

#shine(lp)
#shine(lp2)

[ellisp]

great, this is much better. From my perspective you could close this issue now if you want. Of course, faster would always be nicer, but the basic un-usability is fixed.

[timelyportfolio]

Yep, faster would be better, but think only way would be to leverage Rcpp, which I think is out of the scope of this project at least. I'll close for now. Thanks again for all your feedback.