Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fail uploading my shiny app #123

Closed
yenzichun opened this issue Sep 23, 2016 · 20 comments
Closed

fail uploading my shiny app #123

yenzichun opened this issue Sep 23, 2016 · 20 comments
Assignees
Labels
bug an unexpected problem or unintended behavior encoding 🌏

Comments

@yenzichun
Copy link

Hi, I was trying to publish my shinyapp to shinyapps.io.

At first, I got this problem, and sort of figured it out by setting the encoding of the R object loading from .rda to UTF-8.

But the same problem shows when I have some Chinese character in the app.R file.
In short, the following will work:
image

However, this will not (adding some Chinese character in the string):
image

and here's the messages on console:
image

(I have already set the whole app.R file to UTF-8, but it doesn't work.)

Can you help me with this?
Thanks!

@jmcphers
Copy link
Member

Thanks for reporting this! Can you paste the Chinese characters and/or the expression causing the problem into the issue? (We want to be sure we're testing with the same ones when we try to reproduce this, so we'll copy/paste from your text.)

@yenzichun
Copy link
Author

No problem!

I'll provide an easy example below so that you can reproduce the same issue.
Here's the Chinese character string: "這是一個直方圖"

Note: If I run shinyApp(ui, server) on my local machine, then the Chinese characters can be rendered correctly. But when I use the deployApp() in rsconnect, it will show the error.
So I'm guessing as long as the app.R file contains Chinese character, the server can't parse it correctly (even the file is in UTF-8 encoding).

image

If you need any further help to test Chinese character, please let me know. :)

ui <- fluidPage(
  navbarPage( "My Project",
    #tabPanel("這是一個直方圖",          # Chinese character here, and it will show error message
    tabPanel("This is a highchart demo", # English character here, and it won't show error message
      wellPanel(
        highchartOutput("hcontainer1")
      )
    )
  )
)

server <- function(input, output, session) {
  output$hcontainer1 <- renderHighchart({
    highchart() %>% 
      hc_chart(type = "column") %>%
      #hc_title(text = "這是一個直方圖") %>%            # this will show error message
      hc_title(text = "A highcharter chart") %>%        # this won't show error message
      hc_xAxis(categories = 2012:2016) %>% 
      hc_add_series(data = c(3900,  4200,  5700,  8500, 11900),
        name = "Downloads")
  })
}

@jmcphers
Copy link
Member

jmcphers commented Oct 4, 2016

Unfortunately I was not able to reproduce this! I deployed your app from a Windows machine without any problems.

https://jonathan.shinyapps.io/shinychars/

Can you paste your RStudio version and sessionInfo()?

@yenzichun
Copy link
Author

yenzichun commented Oct 4, 2016

Absolutely, @jmcphers .

https://yenping.shinyapps.io/chinese_char_demo/

Here's my RStudio version and sessionInfo():

> version
               _                           
platform       x86_64-w64-mingw32          
arch           x86_64                      
os             mingw32                     
system         x86_64, mingw32             
status                                     
major          3                           
minor          3.1                         
year           2016                        
month          06                          
day            21                          
svn rev        70800                       
language       R                           
version.string R version 3.3.1 (2016-06-21)
nickname       Bug in Your Hair
> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=Chinese (Traditional)_Taiwan.950  LC_CTYPE=Chinese (Traditional)_Taiwan.950   
[3] LC_MONETARY=Chinese (Traditional)_Taiwan.950 LC_NUMERIC=C                                
[5] LC_TIME=Chinese (Traditional)_Taiwan.950    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] dplyr_0.5.0       highcharter_0.4.0 shiny_0.13.2     

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.6       plyr_1.8.4        bitops_1.0-6      tools_3.3.1       xts_0.9-7         digest_0.6.10    
 [7] jsonlite_1.0      lubridate_1.6.0   tibble_1.1        nlme_3.1-128      lattice_0.20-33   viridisLite_0.1.3
[13] psych_1.6.9       igraph_1.0.1      DBI_0.5-1         yaml_2.1.13       parallel_3.3.1    stringr_1.1.0    
[19] htmlwidgets_0.7   grid_3.3.1        data.table_1.9.6  R6_2.1.2          foreign_0.8-66    RJSONIO_1.3-0    
[25] tidyr_0.6.0       reshape2_1.4.1    purrr_0.2.2       TTR_0.23-1        magrittr_1.5      htmltools_0.3.5  
[31] rsconnect_0.4.3   rlist_0.4.6.1     assertthat_0.1    quantmod_0.4-6    mnormt_1.5-4      mime_0.5         
[37] xtable_1.8-2      httpuv_1.3.3      stringi_1.1.1     RCurl_1.95-4.8    broom_0.4.1       chron_2.3-47     
[43] zoo_1.7-13 

BTW, did you deploy the app successfully with one .R file (app.R), or deploy it with separate files (ui.R and server.R)?
I deployed with only one .R file instead of separate files as it did in the tutorial page.
Hope this information would help. Thanks!!

@jmcphers
Copy link
Member

jmcphers commented Oct 5, 2016

Looks like you're on an older version of the package -- rsconnect 0.4.5 is the latest. Do you still see the error with the newest version?

@yenzichun
Copy link
Author

Unfortunately the error still exists after installing the latest version of rsconnect ... :(

I loaded the shiny, dplyr, highcharter, and rsconnect packages, ran deployApp(),
and these are warning messages during deploying.

Where could I possibly do wrong?
I will appreciate your help with this situation..

> deployApp("/Github/hchart")
Failed to lint file 'app.R'
The linter failed with message:

invalid multibyte string at '<e9>�<99><e6>銝�<e5>�<e6><e5><9c><96>",'
Failed to lint file 'app.R'
The linter failed with message:

invalid multibyte string at '<e9>�<99><e6>銝�<e5>�<e6><e5><9c><96>",'
Preparing to deploy application...DONE
Uploading bundle for application: 129870...DONE
Deploying bundle: 578344 for application: 129870 ...
Waiting for task: 264390878
  building: Processing bundle: 578344
  building: Building image: 569261
  building: Installing packages
  building: Installing files
  building: Pushing image: 569261
  deploying: Starting instances
  rollforward: Activating new instances
  terminating: Stopping old instances
Application successfully deployed to https://yenping.shinyapps.io/hchart/
Warning messages:
1: In gregexpr("../", content, fixed = TRUE) :
  input string 7 is invalid in this locale
2: In gregexpr("../", content, fixed = TRUE) :
  input string 19 is invalid in this locale
3: In value[[3L]](cond) :
  Failed to parse C:/Users/Yenping/AppData/Local/Temp/RtmpmUPjOB/file362cf37607c/app.R ; dependencies in this file will not be discovered.

Here's my complete app.R file that was deployed to shinyapps.io:

library(shiny)
library(dplyr)
library(highcharter)

ui <- fluidPage(
  navbarPage( "My Project",
              tabPanel("這是一個直方圖",
                       wellPanel(
                         highchartOutput("hcontainer1")
                       )
              )
  )
)

server <- function(input, output, session) {
  output$hcontainer1 <- renderHighchart({
    highchart() %>% 
      hc_chart(type = "column") %>%
      hc_title(text = "這是一個直方圖") %>%
      hc_xAxis(categories = 2012:2016) %>% 
      hc_add_series(data = c(3900,  4200,  5700,  8500, 11900),
                    name = "Downloads")
  })
}

shinyApp(ui,server)

https://yenping.shinyapps.io/hchart/

sessionInfo():

> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=Chinese (Traditional)_Taiwan.950  LC_CTYPE=Chinese (Traditional)_Taiwan.950   
[3] LC_MONETARY=Chinese (Traditional)_Taiwan.950 LC_NUMERIC=C                                
[5] LC_TIME=Chinese (Traditional)_Taiwan.950    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rsconnect_0.4.5   highcharter_0.4.0 dplyr_0.5.0       shiny_0.13.2     

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.6       plyr_1.8.4        bitops_1.0-6      tools_3.3.1       xts_0.9-7         digest_0.6.10    
 [7] jsonlite_1.0      lubridate_1.6.0   tibble_1.1        nlme_3.1-128      lattice_0.20-33   viridisLite_0.1.3
[13] psych_1.6.9       igraph_1.0.1      DBI_0.5-1         parallel_3.3.1    stringr_1.1.0     htmlwidgets_0.7  
[19] grid_3.3.1        data.table_1.9.6  R6_2.1.2          foreign_0.8-66    RJSONIO_1.3-0     tidyr_0.6.0      
[25] reshape2_1.4.1    purrr_0.2.2       TTR_0.23-1        magrittr_1.5      htmltools_0.3.5   rlist_0.4.6.1    
[31] assertthat_0.1    quantmod_0.4-6    mnormt_1.5-4      mime_0.5          xtable_1.8-2      httpuv_1.3.3     
[37] stringi_1.1.1     RCurl_1.95-4.8    broom_0.4.1       chron_2.3-47      zoo_1.7-13

@tkomde
Copy link

tkomde commented Nov 11, 2016

Hi,

I have the same error (fail lint(), dependent libraries aren't installed) on Japanese UTF-8 characters. The errors occur both my file and following official script (https://github.com/rstudio/shiny-examples/tree/master/022-unicode-chinese). My file is attached (please rename .txt to .R).
app.txt

Regards,

> deployApp(account='xxx')
Failed to lint file 'app.R'
The linter failed with message:

invalid multibyte string at '<87><e3><83>シ繧ソ繧貞叙蠕<97>')'
Failed to lint file 'app.R'
The linter failed with message:

invalid multibyte string at '<87><e3><83>シ繧ソ繧貞叙蠕<97>')'
Preparing to deploy application...DONE
Uploading bundle for application: 136990...DONE
Deploying bundle: 611667 for application: 136990 ...
Waiting for task: 292478887
  building: Processing bundle: 611667
  building: Parsing manifest
  building: Installing packages
  building: Installing files
  building: Pushing image: 602320
  deploying: Starting instances
  rollforward: Activating new instances
  terminating: Stopping old instances
Application successfully deployed to https://xxx.shinyapps.io/shinyappsio_jpn/
Warning messages:
1: In gregexpr("../", content, fixed = TRUE) :
  input string 16 is invalid in this locale
2: In value[[3L]](cond) :
  Failed to parse C:/Users/02026/AppData/Local/Temp/RtmpGMsJ1H/file3474d8f4e84/app.R ; dependencies in this file will not be discovered.

> version
               _                           
platform       x86_64-w64-mingw32          
arch           x86_64                      
os             mingw32                     
system         x86_64, mingw32             
status                                     
major          3                           
minor          3.1                         
year           2016                        
month          06                          
day            21                          
svn rev        70800                       
language       R                           
version.string R version 3.3.1 (2016-06-21)
nickname       Bug in Your Hair            
> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=Japanese_Japan.932  LC_CTYPE=Japanese_Japan.932    LC_MONETARY=Japanese_Japan.932
[4] LC_NUMERIC=C                   LC_TIME=C                     

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] rsconnect_0.5.0-2    TTR_0.23-1           leaflet_1.0.1        scales_0.4.0         reshape2_1.4.1      
 [6] ggplot2_2.1.0        jsonlite_0.9.22      caTools_1.17.1       dplyr_0.5.0          lubridate_1.5.6     
[11] httr_1.2.0           shinydashboard_0.5.1 shiny_0.14.2         RevoUtilsMath_8.0.3 

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.5      git2r_0.15.0     plyr_1.8.4       bitops_1.0-6     tools_3.3.1      xts_0.9-7       
 [7] packrat_0.4.7-1  digest_0.6.9     memoise_1.0.0    tibble_1.0       gtable_0.2.0     lattice_0.20-33 
[13] DBI_0.4-1        curl_0.9.7       yaml_2.1.13      withr_1.0.2      stringr_1.0.0    devtools_1.12.0 
[19] RevoUtils_10.0.1 htmlwidgets_0.6  grid_3.3.1       R6_2.1.2         RJSONIO_1.3-0    magrittr_1.5    
[25] htmltools_0.3.5  assertthat_0.1   mime_0.4         xtable_1.8-2     colorspace_1.2-6 httpuv_1.3.3    
[31] labeling_0.3     stringi_1.1.1    RCurl_1.95-4.8   lazyeval_0.2.0   munsell_0.4.3    zoo_1.7-13      

@kevinushey kevinushey self-assigned this Nov 11, 2016
@kevinushey
Copy link
Contributor

Can you try explicitly marking the encoding as UTF-8? You should be able to do so with:

options(encoding = "UTF-8")
deployApp(...)

I can reproduce the errors seen when working within a Japanese locale + the default encoding option set (native.enc); changing the default encoding to UTF-8 seems to resolve the issue. Nonetheless, we should see if we can make this more explicit in the rsconnect APIs.

@tkomde
Copy link

tkomde commented Nov 14, 2016

Hi, Kevin

Thank you for your reply.
By applying ’options(encoding = "UTF-8")’, I can upload project without error!

As I have seen following article, I didn't try setting options(encoding).
http://shiny.rstudio.com/articles/unicode.html

Better to guide the following might be helpful.
tmp.enc <- options()$encoding
options(encoding = "UTF-8")
deployApp()
options(encoding = tmp.enc)

Regards,

@kevinushey
Copy link
Contributor

I wonder if rsconnect should operate in the same way as shiny; that is, assume that any files to be deployed will using UTF-8 encoding?

Perhaps we can have an option rsconnect.encoding, defaulting to UTF-8, and users can reset back to e.g. native.enc if so desired?

@tkomde
Copy link

tkomde commented Nov 14, 2016

Yes, that helps much.
When I develop shiny applications using Japanese characters, I change the encoding to UTF-8 every times. So "rsconnect.encoding, defaulting to UTF-8" is welcome for me, and I will use default setting every time :) .

@kevinushey
Copy link
Contributor

@jin-tkomoda: one other option that might be useful here -- what if you try setting options(encoding = "UTF-8") in your .Rprofile? (Ie, in an R file at ~/.Rprofile).

This will tell R to assume that files are saved with UTF-8 encoding, and so the default behavior of the various 'read' routines will assume the files are saved with UTF-8 encoding (as opposed to the system encoding).

Of course, R's handling of UTF-8 strings has some pitfalls on Windows so this will be a bandaid more than anything.

@tkomde
Copy link

tkomde commented Nov 16, 2016

Normally (of my usecase), all .R files are UTF-8. But some CSVs and some text-stream contain japanese local encodings, so it is unsuited for default setting.

@cderv
Copy link
Contributor

cderv commented Aug 10, 2017

Hi,

@kevinushey is there any news on this ?

I encountered the same issue. It seems to come from packrat, used by rsconnect. The function which parses R files for dependencies is at the bottompackrat:::fileDependencies.R and use parse(file, n = -1L). Error comes from there.

base::parse seems to use R option encoding and when set to native.enc the default, it is not working, even if the file is UTF-8.

You could try parse("app.R") to reproduce the parsing error with your previous example. If you open your file explicitly with UTF-8, it should work parse(file("app.R", encoding = "UTF-8"), n = -1L).
Will work on a reprex to help.

Not sure how to deal with this. Encoding issue is not easy. I will open an issue in packrat too because the cause is there.

@kevinushey
Copy link
Contributor

I think we want to specify the encoding directly in the parse() call, not necessarily by creating a file() connection with the encoding supplied.

Note that options(encoding = "UTF-8") does not means "assume text content from connections is UTF-8"; rather, it means "attempt to re-encode text content from connections to UTF-8". This is separate from the behavior of parse(..., encoding = "UTF-8") where it does mean "assume strings are UTF-8".

Note that the default argument we have is parse(encoding = "unknown"); that is, we don't use getOption("encoding") here. This partially reflects the difference in behaviors.

Here's a test script that illustrates this:

options(encoding = "native.enc")

# write some UTF-8 encoded content to a file, preserving
# the UTF-8 encoding (don't attempt to re-encode)
contents <- "\"這是一個直方圖\""
tempfile <- tempfile()
writeLines(contents, con = tempfile, useBytes = TRUE)

# try parsing the file
parse(tempfile)
parse(tempfile)[[1]]
parse(tempfile, encoding = "UTF-8")
parse(tempfile, encoding = "UTF-8")[[1]]

# try parsing from a connection, but set the encoding
parse(file(tempfile))
parse(file(tempfile))[[1]]
parse(file(tempfile, encoding = "UTF-8"))
parse(file(tempfile, encoding = "UTF-8"))[[1]]

# convince ourselves we did write the content as UTF-8
input <- readChar(tempfile, file.info(tempfile)$size, useBytes = TRUE)
Encoding(input) <- "UTF-8"
input

And the output I see:

> options(encoding = "native.enc")
> 
> # write some UTF-8 encoded content to a file, preserving
> # the UTF-8 encoding (don't attempt to re-encode)
> contents <- "\"這是一個直方圖\""
> tempfile <- tempfile()
> writeLines(contents, con = tempfile, useBytes = TRUE)
> 
> # try parsing the file
> parse(tempfile)
expression("這是一個直方圖")
> parse(tempfile)[[1]]
[1] "這是一個直方圖"
> parse(tempfile, encoding = "UTF-8")
expression("這是一個直方圖")
> parse(tempfile, encoding = "UTF-8")[[1]]
[1] "這是一個直方圖"
> 
> # try parsing from a connection, but set the encoding
> parse(file(tempfile))
expression("這是一個直方圖")
> parse(file(tempfile))[[1]]
[1] "這是一個直方圖"
> parse(file(tempfile, encoding = "UTF-8"))
Error in parse(file(tempfile, encoding = "UTF-8")) : 
  1:1: unexpected INCOMPLETE_STRING
1: "
    ^
In addition: Warning message:
In parse(file(tempfile, encoding = "UTF-8")) :
  invalid input found on input connection 'C:\Users\kevin\AppData\Local\Temp\RtmpamHn4R\file1b6431855b3e'
> parse(file(tempfile, encoding = "UTF-8"))[[1]]
Error in parse(file(tempfile, encoding = "UTF-8")) : 
  1:1: unexpected INCOMPLETE_STRING
1: "
    ^
In addition: Warning message:
In parse(file(tempfile, encoding = "UTF-8")) :
  invalid input found on input connection 'C:\Users\kevin\AppData\Local\Temp\RtmpamHn4R\file1b6431855b3e'
> 
> # convince ourselves we did write the content as UTF-8
> input <- readChar(tempfile, file.info(tempfile)$size, useBytes = TRUE)
> Encoding(input) <- "UTF-8"
> input
[1] "\"這是一個直方圖\"\r\n"

tl;dr: Packrat should probably try parsing with parse(encoding = "UTF-8").

@cderv
Copy link
Contributor

cderv commented Aug 11, 2017

When I try that with French on Windows, things are different as the file is not written initially in UTF-8 if I play your example

options(encoding = "native.enc")

# write some content in French with special french character
contents <- "\"Sélectionner des pâtes dans sa tête\""
tempfile <- tempfile()
writeLines(contents, con = tempfile, useBytes = TRUE)

# check the encoding
input <- readChar(tempfile, file.info(tempfile)$size, useBytes = TRUE)
Encoding(input) <- "UTF-8"
input
Encoding(input) <- "Latin1"
input

# try parsing the file
parse(tempfile)
parse(tempfile)[[1]]
parse(tempfile, encoding = "UTF-8")
parse(tempfile, encoding = "UTF-8")[[1]]
parse(tempfile, encoding = "latin1")
parse(tempfile, encoding = "latin1")[[1]]

# try parsing from a connection, but set the encoding
parse(file(tempfile))
parse(file(tempfile))[[1]]
parse(file(tempfile, encoding = "UTF-8"))
parse(file(tempfile, encoding = "UTF-8"))[[1]]
parse(file(tempfile, encoding = "latin1"))
parse(file(tempfile, encoding = "latin1"))[[1]]

# try parsing with readLines
readLines(tempfile)
readLines(tempfile, encoding = "UTF-8")
readLines(tempfile, encoding = "Latin1")
results
> options(encoding = "native.enc")
> 
> # write some content in French with special french character
> contents <- "\"Sélectionner des pâtes dans sa tête\""
> tempfile <- tempfile()
> writeLines(contents, con = tempfile, useBytes = TRUE)
> 
> # check the encoding
> input <- readChar(tempfile, file.info(tempfile)$size, useBytes = TRUE)
> Encoding(input) <- "UTF-8"
> input
[1] "\"S\xe9lectionner des p\xe2tes dans sa t\xeate\"\r\n"
> Encoding(input) <- "Latin1"
> input
[1] "\"Sélectionner des pâtes dans sa tête\"\r\n"
> 
> # try parsing the file
> parse(tempfile)
expression("Sélectionner des pâtes dans sa tête")
> parse(tempfile)[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> parse(tempfile, encoding = "UTF-8")
expression("Sélectionner des pâtes dans sa tête")
> parse(tempfile, encoding = "UTF-8")[[1]]
[1] "S\xe9lectionner des p\xe2tes dans sa t\xeate"
> parse(tempfile, encoding = "latin1")
expression("Sélectionner des pâtes dans sa tête")
> parse(tempfile, encoding = "latin1")[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> 
> # try parsing from a connection, but set the encoding
> parse(file(tempfile))
expression("Sélectionner des pâtes dans sa tête")
> parse(file(tempfile))[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> parse(file(tempfile, encoding = "UTF-8"))
Error in parse(file(tempfile, encoding = "UTF-8")) : 
  1:1: unexpected INCOMPLETE_STRING
1: "S
    ^
In addition: Warning message:
invalid input found on input connection 'D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c464495cdf' 
> parse(file(tempfile, encoding = "UTF-8"))[[1]]
Error in parse(file(tempfile, encoding = "UTF-8")) : 
  1:1: unexpected INCOMPLETE_STRING
1: "S
    ^
In addition: Warning message:
invalid input found on input connection 'D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c464495cdf' 
> parse(file(tempfile, encoding = "latin1"))
expression("Sélectionner des pâtes dans sa tête")
> parse(file(tempfile, encoding = "latin1"))[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> 
> # try parsing with readLines
> readLines(tempfile)
[1] "\"Sélectionner des pâtes dans sa tête\""
> readLines(tempfile, encoding = "UTF-8")
[1] "\"S\xe9lectionner des p\xe2tes dans sa t\xeate\""
> readLines(tempfile, encoding = "Latin1")
[1] "\"Sélectionner des pâtes dans sa tête\""

If I try to force writing to UTF-8 to create a UTF-8 file as app.R in shiny apps which needs UTF-8 file, I think the parse(encoding = "UTF-8") should work in packrat

tempfile <- tempfile()
contents_utf8 <- iconv(contents, to = "UTF-8")
Encoding(contents_utf8)
writeLines(contents_utf8, con = tempfile, useBytes = T)
input <- readChar(tempfile, file.info(tempfile)$size, useBytes = TRUE)
Encoding(input) <- "UTF-8"
input

# try parsing the file - nothing work
parse(tempfile)
parse(tempfile)[[1]]
parse(tempfile, encoding = "UTF-8")
parse(tempfile, encoding = "UTF-8")[[1]]
parse(tempfile, encoding = "latin1")
parse(tempfile, encoding = "latin1")[[1]]

# try parsing from a connection, but set the encoding
parse(file(tempfile))
parse(file(tempfile))[[1]]
parse(file(tempfile, encoding = "UTF-8"))
parse(file(tempfile, encoding = "UTF-8"))[[1]]
parse(file(tempfile, encoding = "latin1"))
parse(file(tempfile, encoding = "latin1"))[[1]]
results
> tempfile <- tempfile()
> contents_utf8 <- iconv(contents, to = "UTF-8")
> Encoding(contents_utf8)
[1] "UTF-8"
> writeLines(contents_utf8, con = tempfile, useBytes = T)
> input <- readChar(tempfile, file.info(tempfile)$size, useBytes = TRUE)
> Encoding(input) <- "UTF-8"
> input
[1] "\"Sélectionner des pâtes dans sa tête\"\r\n"
> parse(tempfile)
expression("Sélectionner des pâtes dans sa tête")
> parse(tempfile)[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> parse(tempfile, encoding = "UTF-8")
expression("Sélectionner des pâtes dans sa tête")
> parse(tempfile, encoding = "UTF-8")[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> parse(tempfile, encoding = "latin1")
expression("Sélectionner des pâtes dans sa tête")
> parse(tempfile, encoding = "latin1")[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> parse(file(tempfile))
expression("Sélectionner des pâtes dans sa tête")
> parse(file(tempfile))[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> parse(file(tempfile, encoding = "UTF-8"))
expression("Sélectionner des pâtes dans sa tête")
> parse(file(tempfile, encoding = "UTF-8"))[[1]]
[1] "Sélectionner des pâtes dans sa tête"
> parse(file(tempfile, encoding = "latin1"))
expression("Sélectionner des pâtes dans sa tête")
> parse(file(tempfile, encoding = "latin1"))[[1]]
[1] "Sélectionner des pâtes dans sa tête"

Initially I was thinking of using file() with encoding because I tested it on a french app.r file with accentuated words in it (including column names). This is an example where parse(encoding = "UTF-8") did not manage to parse

# Create a file with french comments and column name
contents <- c("library(tibble)", "#Création table", "tab <- tibble(scénario = 1:3, résultat = 1:3)")
tempfile_default <- tempfile()
# By default on my system, will be latin1
writeLines(contents, con = tempfile_default)
parse(tempfile_default)
parse(tempfile_default)[[1]]
parse(tempfile_default, encoding = "UTF-8")
parse(tempfile_default, encoding = "UTF-8")
# force UTF-8 for shiny requirement for example
tempfile_utf8 <- tempfile()
writeLines(iconv(contents, to = "UTF-8"), con = tempfile_utf8, useBytes = T)
parse(tempfile_utf8)
parse(tempfile_utf8)[[1]]
parse(tempfile_utf8, encoding = "UTF-8")
parse(tempfile_utf8, encoding = "UTF-8")
# nothing is working except if I defined file with encoding.
parse(file(tempfile_utf8, encoding = "UTF-8"))
parse(file(tempfile_utf8, encoding = "UTF-8"), encoding = "UTF-8")
Results
> # Create a file with french comments and column name
> contents <- c("library(tibble)", "#Création table", "tab <- tibble(scénario = 1:3, résultat = 1:3)")
> tempfile_default <- tempfile()
> # By default on my system, will be latin1
> writeLines(contents, con = tempfile_default)
> parse(tempfile_default)
Error in parse(tempfile_default) : 
  D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c45b8abe9:3:20: unexpected '>'
2: #Cr<e9>ation table
3: tab <- tibble(sc<e9>
                      ^
> parse(tempfile_default)[[1]]
Error in parse(tempfile_default) : 
  D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c45b8abe9:3:20: unexpected '>'
2: #Cr<e9>ation table
3: tab <- tibble(sc<e9>
                      ^
> parse(tempfile_default, encoding = "UTF-8")
Error in parse(tempfile_default, encoding = "UTF-8") : 
  D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c45b8abe9:3:20: unexpected '>'
2: #Cr<e9>ation table
3: tab <- tibble(sc<e9>
                      ^
> parse(tempfile_default, encoding = "UTF-8")
Error in parse(tempfile_default, encoding = "UTF-8") : 
  D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c45b8abe9:3:20: unexpected '>'
2: #Cr<e9>ation table
3: tab <- tibble(sc<e9>
                      ^
> # force UTF-8 for shiny requirement for example
> tempfile_utf8 <- tempfile()
> writeLines(iconv(contents, to = "UTF-8"), con = tempfile_utf8, useBytes = T)
> parse(tempfile_utf8)
Error in parse(tempfile_utf8) : 
  D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c44c3a7735:3:18: unexpected input
2: #Création table
3: tab <- tibble(scé
                    ^
> parse(tempfile_utf8)[[1]]
Error in parse(tempfile_utf8) : 
  D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c44c3a7735:3:18: unexpected input
2: #Création table
3: tab <- tibble(scé
                    ^
> parse(tempfile_utf8, encoding = "UTF-8")
Error in parse(tempfile_utf8, encoding = "UTF-8") : 
  D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c44c3a7735:3:16: unexpected input
2: #Création table
3: tab <- tibble(scé
                  ^
> parse(tempfile_utf8, encoding = "UTF-8")
Error in parse(tempfile_utf8, encoding = "UTF-8") : 
  D:\Users\dervieuxchr\AppData\Local\Temp\RtmpqUQ761\file16c44c3a7735:3:16: unexpected input
2: #Création table
3: tab <- tibble(scé
                  ^
> # nothing is working except if I defined file with encoding.
> parse(file(tempfile_utf8, encoding = "UTF-8"))
expression(library(tibble), tab <- tibble(scénario = 1:3, résultat = 1:3))
> parse(file(tempfile_utf8, encoding = "UTF-8"), encoding = "UTF-8")
expression(library(tibble), tab <- tibble(scénario = 1:3, résultat = 1:3))

file is being parsed. packrat can then guess dependencies.

In this case, I need file(encoding = "UTF-8"). Not sure to understand every bit of how R deals with encoding.

It could be an edge case, and I think the main solution is to set in packrat parse(encoding = "UTF-8) or to be able to pass encoding param through.

Session Info
sessionInfo()
#> R version 3.3.2 (2016-10-31)
#> Platform: x86_64-w64-mingw32/x64 (64-bit)
#> Running under: Windows 7 x64 (build 7601) Service Pack 1
#> 
#> locale:
#> [1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252   
#> [3] LC_MONETARY=French_France.1252 LC_NUMERIC=C                  
#> [5] LC_TIME=French_France.1252    
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] backports_1.1.0 magrittr_1.5    rprojroot_1.2   tools_3.3.2    
#>  [5] htmltools_0.3.6 yaml_2.1.14     Rcpp_0.12.11    stringi_1.1.5  
#>  [9] rmarkdown_1.6   knitr_1.17      stringr_1.2.0   digest_0.6.12  
#> [13] evaluate_0.10.1

@Wang-CODEPROJECT
Copy link

Hi, I have a error when I connect my RStudio with shinyapp.io . It tells me:

Error in curl::curl_fetch_memory(url, handle = handle) :
schannel: CertGetNameString() failed to match connection hostname (api.shinyapps.io) against server certificate names
In addition: Warning message:
In readLines(certificateFile, warn = FALSE) :
invalid input found on input connection 'C:/Program Files/R/R-3.6.1/library/rsconnect/cert/cacert.pem'
Timing stopped at: 0.01 0.02 0.94

How can I solve this?

@BroVic
Copy link

BroVic commented Sep 18, 2019

Hello all, let me join in the fray.

I have seen this warning

In fileDependencies.R(file) :
  Failed to parse C:/Users/Admn/Documents/blah/blah/blah/SomeFile.r ; dependencies in this file will not be discovered.

Now, I know that SomeFile.r has certain non-ASCII characters in it and my code parses this file with scan(). Furthermore, I am in packrat::on() and equally have a Shiny app in the same project. However, I suspect that packrat may be the issue as the warnings were thrown after a call to packrat::status() - 21 warnings in all and one for each file with the strange characters.

@seabass20
Copy link

Hi! Any updates on the best way to do this? My ShinyApp reads in several csv files, most of them with non-ASCII characters (german Umlaute). I have saved all of the relevant r-scripts and csv files in UTF-8 encoding, and have also set options(encoding = "UTF-8") before deploying the App, but I still get this error message.

image

@hadley hadley added the bug an unexpected problem or unintended behavior label Feb 21, 2023
@hadley
Copy link
Member

hadley commented Feb 24, 2023

Should be fixed by rstudio/packrat#647

@hadley hadley closed this as completed Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug an unexpected problem or unintended behavior encoding 🌏
Projects
None yet
Development

No branches or pull requests

9 participants