Encoding issues

[shrektan]

Hi, first of all, thanks for this great package. It's very powerful.

However, I encounter an issue related to the encoding in windows. I'm not familiar with htmlwidgets and not be able to figure out what the problem is. So, I come here to file this issue and hopefully can get the help from you, thanks.

Here's the minimal reproducible example

library(magrittr)
r <- data.frame(a = 1:3, b = c("a", "b", "c"))
highcharter::highchart() %>%
  highcharter::hc_chart(type = "column") %>%
  highcharter::hc_xAxis(categories = r$b) %>%
  highcharter::hc_add_series(data = r$a, name = "权益类")

The Error msg

Error in gsub("</", "\\u003c/", payload, fixed = TRUE) : 
  invalid multibyte string at '<bb>"<7d>]},"theme":null,"conf_opts":{"global":{"Date":null,"VMLRadialGradientURL":"http =//code.highcharts.com/list(version)/gfx/vml-radial-gradient.png","canvasToolsURL":"http =//code.highcharts.com/list(version)/modules/canvas-tools.js","getTimezoneOffset":null,"timezoneOffset":0,"useUTC":true},"lang":{"contextButtonTitle":"Chart context menu","decimalPoint":".","downloadJPEG":"Download JPEG image","downloadPDF":"Download PDF document","downloadPNG":"Download PNG image","downloadSVG":"Download SVG vector image","drillUpText":"Back to {series.name}","invalidDate":null,"loading":"Loading...","months":["January","February","March","April","May","June","July","August","September","October","November","December"],"noData":"No data to display","numericSymbols":["k","M","G","T","P","E"],"printChart":"Print chart","resetZoom":"Reset zoom","resetZoomTitle":"Reset zoom level 1:1","shortMonths":["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct",

devtools::session_info()

> devtools::session_info()
Session info -------------------------------------------------------------------------------
 setting  value                                              
 version  R version 3.2.2 (2015-08-14)                       
 system   i386, mingw32                                      
 ui       RStudio (0.99.879)                                 
 language (EN)                                               
 collate  Chinese (Simplified)_People's Republic of China.936
 tz       Asia/Taipei                                        
 date     2016-02-29                                         

Packages -----------------------------------------------------------------------------------
 package     * version    date       source                                
 assertthat    0.1        2013-12-06 CRAN (R 3.1.1)                        
 chron         2.3-47     2015-06-24 CRAN (R 3.2.1)                        
 data.table    1.9.7      2015-11-04 Github (Rdatatable/data.table@819ac6f)
 DBI           0.3.1.9008 2015-07-17 Github (rstats-db/DBI@f5bd553)        
 devtools      1.10.0     2016-01-23 CRAN (R 3.2.3)                        
 digest        0.6.9      2016-01-08 CRAN (R 3.2.3)                        
 dplyr         0.4.3.9000 2016-01-06 Github (hadley/dplyr@4f2d7f8)         
 highcharter   0.2.0      2016-02-25 CRAN (R 3.2.3)                        
 htmltools     0.3        2015-12-29 CRAN (R 3.2.3)                        
 htmlwidgets   0.6        2016-02-29 Github (ramnathv/htmlwidgets@9557407) 
 jsonlite      0.9.19     2015-11-28 CRAN (R 3.2.3)                        
 lattice       0.20-33    2015-07-14 CRAN (R 3.2.2)                        
 magrittr    * 1.5        2014-11-22 CRAN (R 3.1.2)                        
 memoise       1.0.0      2016-01-29 CRAN (R 3.2.3)                        
 purrr         0.2.1      2016-02-16 Github (hadley/purrr@da96161)         
 quantmod      0.4-5      2015-07-24 CRAN (R 3.2.1)                        
 R6            2.1.2      2016-01-26 CRAN (R 3.2.3)                        
 Rcpp          0.12.3     2016-01-10 CRAN (R 3.2.3)                        
 rlist         0.4.5.1    2015-09-09 CRAN (R 3.2.3)                        
 rstudioapi    0.5        2016-01-24 CRAN (R 3.2.3)                        
 stringi       1.0-1      2015-10-22 CRAN (R 3.2.2)                        
 stringr       1.0.0      2015-04-30 CRAN (R 3.2.0)                        
 tidyr         0.4.1      2016-02-05 CRAN (R 3.2.3)                        
 TTR           0.23-0     2015-07-10 CRAN (R 3.2.1)                        
 viridisLite   0.1.1      2015-12-31 CRAN (R 3.2.3)                        
 xts           0.9-7      2014-01-02 CRAN (R 3.1.1)                        
 zoo           1.7-12     2015-03-16 CRAN (R 3.1.3) 

sessionInfo()

> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: i386-w64-mingw32/i386 (32-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=Chinese (Simplified)_People's Republic of China.936 
[2] LC_CTYPE=Chinese (Simplified)_People's Republic of China.936   
[3] LC_MONETARY=Chinese (Simplified)_People's Republic of China.936
[4] LC_NUMERIC=C                                                   
[5] LC_TIME=Chinese (Simplified)_People's Republic of China.936    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] magrittr_1.5

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.3       rstudioapi_0.5    devtools_1.10.0   viridisLite_0.1.1
 [5] lattice_0.20-33   R6_2.1.2          stringr_1.0.0     TTR_0.23-0       
 [9] highcharter_0.2.0 dplyr_0.4.3.9000  tools_3.2.2       xts_0.9-7        
[13] parallel_3.2.2    quantmod_0.4-5    grid_3.2.2        data.table_1.9.7 
[17] DBI_0.3.1.9008    htmltools_0.3     assertthat_0.1    digest_0.6.9     
[21] purrr_0.2.1       tidyr_0.4.1       htmlwidgets_0.6   rlist_0.4.5.1    
[25] memoise_1.0.0     stringi_1.0-1     jsonlite_0.9.19   chron_2.3-47     
[29] zoo_1.7-12 

[shrektan]

I guess it's something related to the handling of R object to JSON.

Note, if I remove the last Chinese character 类, the code works fine, i.e.:

library(magrittr)
r <- data.frame(a = 1:3, b = c("a", "b", "c"))
highcharter::highchart() %>%
  highcharter::hc_chart(type = "column") %>%
  highcharter::hc_xAxis(categories = r$b) %>%
  highcharter::hc_add_series(data = r$a, name = "权益")

[shrektan]

I guess it's not an error related to your package, because I find that the DT can't work as well.

DT::datatable(data.frame(name = "权益类"))

So, feel free to close this issue if you end up with the same conclusion. Thanks!

[shrektan]

@jbkunst Sorry, after I reboot my PC, the issue is gone... Sometimes, the best solution in Windows is rebooting... So, I'll close this issue.

[jbkunst]

😆 Sometimes? Always!

Regards

[glfeng318]

i got the same problem on centos.

[jbkunst]

Hi @Funnng,

If you have the same message/error:

Error in gsub("</", "\\u003c/", payload, fixed = TRUE) :

That line belogs to the htmlwidgets package (see line here)

So I'm not sure how I can help you. Have you tried removing special characters?

[shrektan]

@Funnng covert the UTF8 character to native might help

[glfeng318]

i tried the code below and it show nothing, and i check the console log in chrome and found this error.

highchart() %>% hc_add_series_treemap(treemap(data.frame(city=c('广州','广州','佛山','佛山'), zone=c('天河','海珠','顺达','禅城'), pay = c(563,235)), index = c('city','zone'), vSize='pay', vColor='pay', type='comp', palette = rev(viridis(9)), draw = F))

[glfeng318]

@jbkunst @shrektan i think it is not the same issue, the error i met is not like Error in gsub("</", "\\u003c/", payload, fixed = TRUE) :

> devtools::session_info()
Session info -------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.2.3 (2015-12-10)
 system   x86_64, linux-gnu           
 ui       RStudio (0.99.892)          
 language (EN)                        
 collate  en_US.UTF-8                 
 tz       <NA>                        
 date     2016-05-23                

[jbkunst]

Hi @Funnng

The issue is because internally the hc_add_series_treemap function is using a helper function str_to_id which try to remove spectial character to put the name of the value as id in the data series (check str_to_id("广州")).

By now, you can use a modify hc_add_series_treemap2 (see code below) until I check if there no problem if I remove some replacement in the str_to_id function.

library(dplyr)
library(purrr)

hc_add_series_treemap2 <- function(hc, tm, ...) {

  df <- tm$tm %>% 
    tbl_df() %>% 
    select_("-x0", "-y0", "-w", "-h", "-stdErr", "-vColorValue") %>% 
    rename_("value" = "vSize", "valuecolor" = "vColor") %>% 
    purrr::map_if(is.factor, as.character) %>% 
    data.frame(stringsAsFactors = FALSE) %>% 
    tbl_df()

  ndepth <- which(names(df) == "value") - 1

  ds <- map_df(seq(ndepth), function(lvl){ # lvl <- sample(seq(ndepth), size = 1)

    df2 <- df %>% 
      filter_(sprintf("level == %s", lvl)) %>% 
      rename_("name" = names(df)[lvl]) %>% 
      mutate_("id" = "name")

    if (lvl > 1) {
      df2 <- df2 %>% 
        mutate_("parent" = names(df)[lvl - 1],
                "parent" = "parent")
    } else {
      df2 <- df2 %>% 
        mutate_("parent" = NA)
    }

    df2 

  })

  ds <- list.parse3(ds)

  ds <- map(ds, function(x){
    if (is.na(x$parent))
      x$parent <- NULL
    x
  })

  hc %>% hc_add_series(data = ds, type = "treemap", ...)

}