Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

factors and dates are slow #235

Closed
etiennebr opened this issue Nov 8, 2019 · 12 comments
Closed

factors and dates are slow #235

etiennebr opened this issue Nov 8, 2019 · 12 comments
Assignees

Comments

@etiennebr
Copy link

@etiennebr etiennebr commented Nov 8, 2019

I'm experimenting with large datasets in mapdeck and it's amazingly smooth. Thanks for that! However, it seems that adding a tooltip to a large number of points really slows down the map rendering. RStudio becomes completly unreactive if N increases.

library(mapdeck)
    
    N <- 2e4
    data <- data.frame(
        longitude = runif(N, -180, 180),
        latitude = runif(N, -180, 180),
        tooltip = sample(LETTERS, N, replace = TRUE)
    )
    
    system.time(
        mapdeck() %>% 
            add_scatterplot(
                data = data,
                lon = "longitude",
                lat = "latitude",
                fill_colour = "tooltip"
            )
    )
#>    user  system elapsed 
#>   0.077   0.012   0.090
    
    system.time(
        mapdeck() %>% 
            add_scatterplot(
                data = data,
                lon = "longitude",
                lat = "latitude",
                fill_colour = "tooltip",
                tooltip = "tooltip"
            )
    )
#>    user  system elapsed 
#>   3.676   0.468   4.145

Created on 2019-11-08 by the reprex package (v0.2.1)

Session info
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.1 (2019-07-05)
#>  os       Ubuntu 18.04.3 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language en_CA                       
#>  collate  en_CA.UTF-8                 
#>  ctype    en_CA.UTF-8                 
#>  tz       America/Toronto             
#>  date     2019-11-08                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version  date       lib source                             
#>  assertthat    0.2.1    2019-03-21 [2] CRAN (R 3.6.0)                     
#>  backports     1.1.4    2019-04-10 [2] CRAN (R 3.6.0)                     
#>  callr         3.2.0    2019-03-15 [2] CRAN (R 3.6.0)                     
#>  cli           1.1.0    2019-03-19 [2] CRAN (R 3.6.0)                     
#>  crayon        1.3.4    2017-09-16 [2] CRAN (R 3.6.0)                     
#>  desc          1.2.0    2018-05-01 [2] CRAN (R 3.6.0)                     
#>  devtools      2.0.2    2019-04-08 [2] CRAN (R 3.6.0)                     
#>  digest        0.6.20   2019-07-04 [1] CRAN (R 3.6.0)                     
#>  evaluate      0.13     2019-02-12 [2] CRAN (R 3.6.0)                     
#>  fs            1.3.0    2019-05-02 [2] CRAN (R 3.6.0)                     
#>  glue          1.3.1    2019-03-12 [2] CRAN (R 3.6.0)                     
#>  highr         0.8      2019-03-20 [2] CRAN (R 3.6.0)                     
#>  htmltools     0.4.0    2019-10-04 [2] CRAN (R 3.6.1)                     
#>  htmlwidgets   1.5.1    2019-10-08 [2] CRAN (R 3.6.1)                     
#>  jsonify       1.0.0    2019-11-03 [2] CRAN (R 3.6.1)                     
#>  knitr         1.22     2019-03-08 [2] CRAN (R 3.6.0)                     
#>  magrittr      1.5      2014-11-22 [2] CRAN (R 3.6.0)                     
#>  mapdeck     * 0.2.1006 2019-11-08 [1] Github (symbolixau/mapdeck@d7f5421)
#>  memoise       1.1.0    2017-04-21 [2] CRAN (R 3.6.0)                     
#>  pkgbuild      1.0.3    2019-03-20 [2] CRAN (R 3.6.0)                     
#>  pkgload       1.0.2    2018-10-29 [2] CRAN (R 3.6.0)                     
#>  prettyunits   1.0.2    2015-07-13 [2] CRAN (R 3.6.0)                     
#>  processx      3.3.0    2019-03-10 [2] CRAN (R 3.6.0)                     
#>  ps            1.3.0    2018-12-21 [2] CRAN (R 3.6.0)                     
#>  R6            2.4.0    2019-02-14 [2] CRAN (R 3.6.0)                     
#>  Rcpp          1.0.2    2019-07-25 [1] CRAN (R 3.6.0)                     
#>  remotes       2.0.4    2019-04-10 [2] CRAN (R 3.6.0)                     
#>  rlang         0.4.1    2019-10-24 [2] CRAN (R 3.6.1)                     
#>  rmarkdown     1.12     2019-03-14 [2] CRAN (R 3.6.0)                     
#>  rprojroot     1.3-2    2018-01-03 [2] CRAN (R 3.6.0)                     
#>  sessioninfo   1.1.1    2018-11-05 [2] CRAN (R 3.6.0)                     
#>  stringi       1.4.3    2019-03-12 [2] CRAN (R 3.6.0)                     
#>  stringr       1.4.0    2019-02-10 [2] CRAN (R 3.6.0)                     
#>  testthat      2.1.1    2019-04-23 [2] CRAN (R 3.6.0)                     
#>  usethis       1.5.0    2019-04-07 [2] CRAN (R 3.6.0)                     
#>  withr         2.1.2    2018-03-15 [2] CRAN (R 3.6.0)                     
#>  xfun          0.6      2019-04-02 [2] CRAN (R 3.6.0)                     
#>  yaml          2.2.0    2018-07-25 [2] CRAN (R 3.6.0)                     
#> 
#> [1] /home/etienne/R-dev
#> [2] /home/etienne/R/x86_64-pc-linux-gnu-library/3.6
#> [3] /usr/local/lib/R/site-library
#> [4] /usr/lib/R/site-library
#> [5] /usr/lib/R/library

I experimented a bit and apparently time is growing exponentially.

@dcooley
Copy link
Collaborator

@dcooley dcooley commented Nov 8, 2019

To confirm, are you saying the issue is in the R code before the map is rendered, or the map itself becomes unresponsive?

@etiennebr
Copy link
Author

@etiennebr etiennebr commented Nov 8, 2019

Yes, sorry. The problem occurs before the map is rendered, so probably the R code. For large data, I can't say if the map becomes unresponsive with tooltips because I killed the process before.

@dcooley
Copy link
Collaborator

@dcooley dcooley commented Nov 8, 2019

ok. And you've only noticed this when you add the tooltip argument, when it's just fill_colour the large data isn't an issue?

@etiennebr
Copy link
Author

@etiennebr etiennebr commented Nov 8, 2019

Exactly. I left the fill_colour because I thought it probably passed the same data to the map. But the problem is the same without the fill_colour.

@dcooley
Copy link
Collaborator

@dcooley dcooley commented Nov 8, 2019

ok I can replicate the issue. I'll look into it.

@dcooley
Copy link
Collaborator

@dcooley dcooley commented Nov 8, 2019

It seems to be related to "tooltip" being a factor. If I convert it to character or numeric then it's quick again.

@SymbolixAU
Copy link
Owner

@SymbolixAU SymbolixAU commented Nov 11, 2019

I think the slow-down comes from jsonify::to_json when factors_as_string = TRUE (i.e, converting factors to string, which is the default for mapdeck)

@etiennebr
Copy link
Author

@etiennebr etiennebr commented Nov 11, 2019

Just noticed that date is even worse, but converted to character it's fine.

library(mapdeck)

N <- 2e4
data <- data.frame(
  longitude = runif(N, -180, 180),
  latitude = runif(N, -180, 180),
  tooltip = sample(LETTERS, N, replace = TRUE),
  date = Sys.Date(),
  date_txt = as.character(Sys.Date())
)

system.time(
  mapdeck() %>% 
    add_scatterplot(
      data = data,
      lon = "longitude",
      lat = "latitude",
      fill_colour = "tooltip",
      tooltip = "date"
    )
)

#>    user  system elapsed 
#> 344.012   0.000 343.854 


system.time(
  mapdeck() %>% 
    add_scatterplot(
      data = data,
      lon = "longitude",
      lat = "latitude",
      fill_colour = "tooltip",
      tooltip = "date_txt"
    )
)
#>    user  system elapsed 
#>   0.076   0.000   0.077
@SymbolixAU
Copy link
Owner

@SymbolixAU SymbolixAU commented Nov 11, 2019

yep. This is not strictly related to the jsonify issue, but that was / is a contributing factor. Will dig further.

@SymbolixAU SymbolixAU mentioned this issue Nov 11, 2019
1 of 1 task complete
@SymbolixAU SymbolixAU changed the title Adding tooltip slows execution factors and dates are slow Nov 12, 2019
SymbolixAU added a commit that referenced this issue Nov 12, 2019
@SymbolixAU
Copy link
Owner

@SymbolixAU SymbolixAU commented Nov 12, 2019

Currently on a branch, but I think I've found the slowness

N <- 2e4
data <- data.frame(
	longitude = runif(N, -180, 180),
	latitude = runif(N, -180, 180),
	tooltip = sample(LETTERS, N, replace = TRUE),
	date = Sys.Date(),
	date_txt = as.character(Sys.Date())
)

system.time(
	mapdeck() %>% 
		add_scatterplot(
			data = data,
			lon = "longitude",
			lat = "latitude",
			fill_colour = "tooltip",
			tooltip = "date"
		)
)

# user  system elapsed 
# 0.092   0.006   0.098

system.time(
	mapdeck() %>% 
		add_scatterplot(
			data = data,
			lon = "longitude",
			lat = "latitude",
			fill_colour = "tooltip",
			tooltip = "date_txt"
		)
)

# user  system elapsed 
# 0.065   0.004   0.069
dcooley added a commit that referenced this issue Nov 12, 2019
updated dependencies for #235
@dcooley
Copy link
Collaborator

@dcooley dcooley commented Nov 12, 2019

If you update the dependencies, then mapdeck, you should now see the improvements

devtools::install_github("SymbolixAU/jsonify")
devtools::install_github("SymbolixAU/spatialwidget")
devtools::install_github("SymbolixAU/mapdeck")
@etiennebr
Copy link
Author

@etiennebr etiennebr commented Nov 12, 2019

Thanks, indeed it is a dramatic improvement!

@etiennebr etiennebr closed this Nov 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.