Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warning: Error in gsub: input string 1 is invalid UTF-8 #206

Open
PieterMaertens opened this issue Jul 2, 2018 · 2 comments

Comments

Projects
None yet
3 participants
@PieterMaertens
Copy link

commented Jul 2, 2018

When recording a test with the shinytest package using recordTest(), everything works fine.
But when I rerun the test with the following statement:
testApp(".")
I get an error message:
Warning: Error in gsub: input string 1 is invalid UTF-8
The encoding of all shiny files (global.R, server.R and ui.R) is UTF-8. The ui + server files contain special characters.
I run the shinytest package 1.3.0 on a windows 10 pc with R version 3.5.0 and Rstudio version 1.1.453 and the test json file is:

{
"input": {
"action0": 0,
"algemeen": "Herkomst",
"bevolking": "Ja",
"CMI": "Globaal",
"CMIKeuze": "Alle",
"jaren1": "2017",
"jaren2": "2016",
"jaren2T": "2016",
"jaren3": "2017",
"jaren3T": "2017",
"jaren4": "2017",
"jaren4T": "2017",
"jaren5T": "2017",
"jaren6": "2017",
"jaren6T": "2017",
"jaren7": "2017",
"KenletterKeuze": "Alle",
"kilometer": 5,
"MA": "Globaal",
"MID": "Globaal",
"MIDVBL": "Globaal",
"mzgJaren2": [
2016,
2017
],
"NHPPD": "BB per Oper VTE",
"niveau": "0",
"show1": 0,
"show18": 0,
"show19": 0,
"show2": 0,
"show21": 0,
"show3": 0,
"show4": 0,
"show40": 0,
"show5": 0,
"show6": 0,
"show7": 0,
"VBL": "Globaal",
"ZR": "provincie"
},
"output": {
"Contact2": "Uw contactpersoon is Test",
"Disclaimer": "test",
"herkomstTitel": "<h4 align="center">Herkomst patiënten</h4>",
"mymap": {
"x": {
"options": {
"zoomControl": false
}
},
"evals": [

  ],
  "jsHooks": [

  ],
  "deps": [

  ]
},
"semester1": {
  "html": "Opgelet: enkel 1ste semester!",
  "deps": [

  ]
},
"show": {
  "message": "",
  "call": "NULL",
  "type": [
    "shiny.silent.error",
    "validation"
  ]
},
"showdetails": {
  "message": "",
  "call": "NULL",
  "type": [
    "shiny.silent.error",
    "validation"
  ]
},
"tekstzkh": "test"

},
"export": {

}
}

@zappingseb

This comment has been minimized.

Copy link

commented Dec 13, 2018

There is a pretty easy solution working under Linux & Mac OS:

https://stackoverflow.com/questions/29465612/how-to-detect-invalid-utf8-unicode-binary-in-a-text-file

I did it by

sed s/\n/\\n/g 001.json > output002.json

and

grep -axv '.*' output002.json

I got the lines of code that contain non-utf8 and removed them.

For me it was a JavaScript file I imported

@felipegerard

This comment has been minimized.

Copy link

commented Apr 29, 2019

I'm getting a similar error when comparing current results to expected ones. I also have non-ASCII characters in my code, but in my view the solution shouldn't be to just remove them. What if I need them for some filters for instance?

I found that if I manually open the JSON files as the correct encoding (ISO-8859-1 in my case) and overwrite them as UTF-8, the comparison works fine. Therefore the problem lies in the fact that the JSON files get saved as something different from UTF-8 but then get opened as UTF-8, which produces invalid characters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.