Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

factors and dates are slow #235

Closed
etiennebr opened this issue Nov 8, 2019 · 12 comments
Closed

factors and dates are slow #235

etiennebr opened this issue Nov 8, 2019 · 12 comments
Assignees

Comments

@etiennebr
Copy link

I'm experimenting with large datasets in mapdeck and it's amazingly smooth. Thanks for that! However, it seems that adding a tooltip to a large number of points really slows down the map rendering. RStudio becomes completly unreactive if N increases.

library(mapdeck)
    
    N <- 2e4
    data <- data.frame(
        longitude = runif(N, -180, 180),
        latitude = runif(N, -180, 180),
        tooltip = sample(LETTERS, N, replace = TRUE)
    )
    
    system.time(
        mapdeck() %>% 
            add_scatterplot(
                data = data,
                lon = "longitude",
                lat = "latitude",
                fill_colour = "tooltip"
            )
    )
#>    user  system elapsed 
#>   0.077   0.012   0.090
    
    system.time(
        mapdeck() %>% 
            add_scatterplot(
                data = data,
                lon = "longitude",
                lat = "latitude",
                fill_colour = "tooltip",
                tooltip = "tooltip"
            )
    )
#>    user  system elapsed 
#>   3.676   0.468   4.145

Created on 2019-11-08 by the reprex package (v0.2.1)

Session info
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.1 (2019-07-05)
#>  os       Ubuntu 18.04.3 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language en_CA                       
#>  collate  en_CA.UTF-8                 
#>  ctype    en_CA.UTF-8                 
#>  tz       America/Toronto             
#>  date     2019-11-08                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version  date       lib source                             
#>  assertthat    0.2.1    2019-03-21 [2] CRAN (R 3.6.0)                     
#>  backports     1.1.4    2019-04-10 [2] CRAN (R 3.6.0)                     
#>  callr         3.2.0    2019-03-15 [2] CRAN (R 3.6.0)                     
#>  cli           1.1.0    2019-03-19 [2] CRAN (R 3.6.0)                     
#>  crayon        1.3.4    2017-09-16 [2] CRAN (R 3.6.0)                     
#>  desc          1.2.0    2018-05-01 [2] CRAN (R 3.6.0)                     
#>  devtools      2.0.2    2019-04-08 [2] CRAN (R 3.6.0)                     
#>  digest        0.6.20   2019-07-04 [1] CRAN (R 3.6.0)                     
#>  evaluate      0.13     2019-02-12 [2] CRAN (R 3.6.0)                     
#>  fs            1.3.0    2019-05-02 [2] CRAN (R 3.6.0)                     
#>  glue          1.3.1    2019-03-12 [2] CRAN (R 3.6.0)                     
#>  highr         0.8      2019-03-20 [2] CRAN (R 3.6.0)                     
#>  htmltools     0.4.0    2019-10-04 [2] CRAN (R 3.6.1)                     
#>  htmlwidgets   1.5.1    2019-10-08 [2] CRAN (R 3.6.1)                     
#>  jsonify       1.0.0    2019-11-03 [2] CRAN (R 3.6.1)                     
#>  knitr         1.22     2019-03-08 [2] CRAN (R 3.6.0)                     
#>  magrittr      1.5      2014-11-22 [2] CRAN (R 3.6.0)                     
#>  mapdeck     * 0.2.1006 2019-11-08 [1] Github (symbolixau/mapdeck@d7f5421)
#>  memoise       1.1.0    2017-04-21 [2] CRAN (R 3.6.0)                     
#>  pkgbuild      1.0.3    2019-03-20 [2] CRAN (R 3.6.0)                     
#>  pkgload       1.0.2    2018-10-29 [2] CRAN (R 3.6.0)                     
#>  prettyunits   1.0.2    2015-07-13 [2] CRAN (R 3.6.0)                     
#>  processx      3.3.0    2019-03-10 [2] CRAN (R 3.6.0)                     
#>  ps            1.3.0    2018-12-21 [2] CRAN (R 3.6.0)                     
#>  R6            2.4.0    2019-02-14 [2] CRAN (R 3.6.0)                     
#>  Rcpp          1.0.2    2019-07-25 [1] CRAN (R 3.6.0)                     
#>  remotes       2.0.4    2019-04-10 [2] CRAN (R 3.6.0)                     
#>  rlang         0.4.1    2019-10-24 [2] CRAN (R 3.6.1)                     
#>  rmarkdown     1.12     2019-03-14 [2] CRAN (R 3.6.0)                     
#>  rprojroot     1.3-2    2018-01-03 [2] CRAN (R 3.6.0)                     
#>  sessioninfo   1.1.1    2018-11-05 [2] CRAN (R 3.6.0)                     
#>  stringi       1.4.3    2019-03-12 [2] CRAN (R 3.6.0)                     
#>  stringr       1.4.0    2019-02-10 [2] CRAN (R 3.6.0)                     
#>  testthat      2.1.1    2019-04-23 [2] CRAN (R 3.6.0)                     
#>  usethis       1.5.0    2019-04-07 [2] CRAN (R 3.6.0)                     
#>  withr         2.1.2    2018-03-15 [2] CRAN (R 3.6.0)                     
#>  xfun          0.6      2019-04-02 [2] CRAN (R 3.6.0)                     
#>  yaml          2.2.0    2018-07-25 [2] CRAN (R 3.6.0)                     
#> 
#> [1] /home/etienne/R-dev
#> [2] /home/etienne/R/x86_64-pc-linux-gnu-library/3.6
#> [3] /usr/local/lib/R/site-library
#> [4] /usr/lib/R/site-library
#> [5] /usr/lib/R/library

I experimented a bit and apparently time is growing exponentially.

@dcooley
Copy link
Collaborator

dcooley commented Nov 8, 2019

To confirm, are you saying the issue is in the R code before the map is rendered, or the map itself becomes unresponsive?

@etiennebr
Copy link
Author

Yes, sorry. The problem occurs before the map is rendered, so probably the R code. For large data, I can't say if the map becomes unresponsive with tooltips because I killed the process before.

@dcooley
Copy link
Collaborator

dcooley commented Nov 8, 2019

ok. And you've only noticed this when you add the tooltip argument, when it's just fill_colour the large data isn't an issue?

@etiennebr
Copy link
Author

etiennebr commented Nov 8, 2019

Exactly. I left the fill_colour because I thought it probably passed the same data to the map. But the problem is the same without the fill_colour.

@dcooley
Copy link
Collaborator

dcooley commented Nov 8, 2019

ok I can replicate the issue. I'll look into it.

@dcooley
Copy link
Collaborator

dcooley commented Nov 8, 2019

It seems to be related to "tooltip" being a factor. If I convert it to character or numeric then it's quick again.

@SymbolixAU
Copy link
Collaborator

I think the slow-down comes from jsonify::to_json when factors_as_string = TRUE (i.e, converting factors to string, which is the default for mapdeck)

@etiennebr
Copy link
Author

Just noticed that date is even worse, but converted to character it's fine.

library(mapdeck)

N <- 2e4
data <- data.frame(
  longitude = runif(N, -180, 180),
  latitude = runif(N, -180, 180),
  tooltip = sample(LETTERS, N, replace = TRUE),
  date = Sys.Date(),
  date_txt = as.character(Sys.Date())
)

system.time(
  mapdeck() %>% 
    add_scatterplot(
      data = data,
      lon = "longitude",
      lat = "latitude",
      fill_colour = "tooltip",
      tooltip = "date"
    )
)

#>    user  system elapsed 
#> 344.012   0.000 343.854 


system.time(
  mapdeck() %>% 
    add_scatterplot(
      data = data,
      lon = "longitude",
      lat = "latitude",
      fill_colour = "tooltip",
      tooltip = "date_txt"
    )
)
#>    user  system elapsed 
#>   0.076   0.000   0.077

@SymbolixAU
Copy link
Collaborator

yep. This is not strictly related to the jsonify issue, but that was / is a contributing factor. Will dig further.

@SymbolixAU SymbolixAU changed the title Adding tooltip slows execution factors and dates are slow Nov 12, 2019
SymbolixAU pushed a commit that referenced this issue Nov 12, 2019
@SymbolixAU
Copy link
Collaborator

Currently on a branch, but I think I've found the slowness

N <- 2e4
data <- data.frame(
	longitude = runif(N, -180, 180),
	latitude = runif(N, -180, 180),
	tooltip = sample(LETTERS, N, replace = TRUE),
	date = Sys.Date(),
	date_txt = as.character(Sys.Date())
)

system.time(
	mapdeck() %>% 
		add_scatterplot(
			data = data,
			lon = "longitude",
			lat = "latitude",
			fill_colour = "tooltip",
			tooltip = "date"
		)
)

# user  system elapsed 
# 0.092   0.006   0.098

system.time(
	mapdeck() %>% 
		add_scatterplot(
			data = data,
			lon = "longitude",
			lat = "latitude",
			fill_colour = "tooltip",
			tooltip = "date_txt"
		)
)

# user  system elapsed 
# 0.065   0.004   0.069

dcooley added a commit that referenced this issue Nov 12, 2019
@dcooley
Copy link
Collaborator

dcooley commented Nov 12, 2019

If you update the dependencies, then mapdeck, you should now see the improvements

devtools::install_github("SymbolixAU/jsonify")
devtools::install_github("SymbolixAU/spatialwidget")
devtools::install_github("SymbolixAU/mapdeck")

@etiennebr
Copy link
Author

Thanks, indeed it is a dramatic improvement!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants