Error while parsing image content with vcr rawToChar() error

[Rekyt]

Hi @sckott 👋 ,

I am currently working on a small API package called bnfimage which goal is to be an R client for the French National Library Image database.

I thought I could use vcr for the tests of the package, but I had trouble when recording the queries. The query returns an image (a png image in the example below), I can perfectly read it. But whenever I try to record it with vcr I have the error below because of the rawToChar() function. I can reproduce it by making the query and directly feeding it to rawToChar().

I know it is not directly due to vcr, but is there a solution in case of API that return images? I looked through the different API packages you developed but couldn't find a similar example.

Reprex:

# Actual Query
a = httr::GET("https://gallica.bnf.fr/iiif/ark:/12148/btv1b9055204k/f1/full/1500,750/0/native.png")

# Query retrieved the image
image(httr::content(a))

# Also works with magick
magick::image_read(a$content)

# Impossible to parse with rawToChar() -> thus impossible to use vcr
rawToChar(a$content)
#> Error in rawToChar(a$content): caractère nul au milieu de la chaîne : '\x89PNG\r\n\032\n\0\0\0\rIHDR\0\0\005\xdc\0\0\002\xee\020\0\0\0\0\xc1\xdbW\xd3\0\0\0\tpHYs\0\0=\x84\0\0=\x84\001լ\xaft\0\0\x80\0IDATx\xda$\x9c\005X\024\xef\xfb\xf5-vg\xe6\x89\xe9ؤKi\004\005\xbb\v\x83\026\003\xb1\xbb\x83\xee\020\020\005DQ\f\024\f\xec\xee\xee\xee\xee\xee\xee\xfe\xea\xfb\xfc\xfe\xef\xce\xe5\xa5+\xbb3O\xdc\xf79\x9f\xb3A\xadG\x93\xee\035\xbe\xb2{\xd7ܢ\u0608\x82\xa0\xa0\xc6g\x83\005ϩN\x9f\034\xbb8F8\x975\xa8r\016\xb3\xdfb\xff\xd2\t:\xb7v\xae\xe5\xee\xe9zڕq\xdb\xe6>\xca\xe9{\xfd\x89.\xe9>\x8e\xde\r\xeaӮ\xf7\xeb'{\035\xf42{\xfa{"/\xde'\xc8\x9f\xffU\xff\xe9\xbe\017|\xd7\xf8\xd1\036\xc3\032\xa6\006T\004\xce\f\030\0378)h\x99\xff<\xef獶5\xb2\xfa-\xf5~\xef\xeb\xee\x9d\xe0\xb3?\xb0]Ö>\u05fd\x8e{\x85x3\r\xc66(\xf4\xfc\xe0k\xf0[\xe4\xb3\xc1㉧λ\x9fW\x8d\xb7\xbd\xef\\\xff\0\x9fJ\xdfu\xbe+\xbcZ{\xae\xf5>\xec\xd5\xdf3\xcc\xe7\x8a\177\x8e\xefj\xefF^k\xbd\xee{Ż\xc5zFzA\xaf\xee>\xad<\xcfz{{\x86\xb9\x8foP\xe2\xbd\xc0;\xd7}\xba\017\xf0\x

^{Created on 2019-10-12 by the reprex package (v0.3.0)}

Thank you so much!
Matthias

Session info

devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.0 (2019-04-26)
#>  os       macOS Sierra 10.12.6        
#>  system   x86_64, darwin15.6.0        
#>  ui       X11                         
#>  language (EN)                        
#>  collate  fr_FR.UTF-8                 
#>  ctype    fr_FR.UTF-8                 
#>  tz       Europe/Paris                
#>  date     2019-10-12                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version date       lib source        
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 3.6.0)
#>  backports     1.1.5   2019-10-02 [1] CRAN (R 3.6.0)
#>  callr         3.3.2   2019-09-22 [1] CRAN (R 3.6.0)
#>  cli           1.1.0   2019-03-19 [1] CRAN (R 3.6.0)
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 3.6.0)
#>  curl          4.2     2019-09-24 [1] CRAN (R 3.6.0)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.6.0)
#>  devtools      2.2.1   2019-09-24 [1] CRAN (R 3.6.0)
#>  digest        0.6.21  2019-09-20 [1] CRAN (R 3.6.0)
#>  ellipsis      0.3.0   2019-09-20 [1] CRAN (R 3.6.0)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 3.6.0)
#>  fs            1.3.1   2019-05-06 [1] CRAN (R 3.6.0)
#>  glue          1.3.1   2019-03-12 [1] CRAN (R 3.6.0)
#>  highr         0.8     2019-03-20 [1] CRAN (R 3.6.0)
#>  htmltools     0.4.0   2019-10-04 [1] CRAN (R 3.6.0)
#>  httr          1.4.1   2019-08-05 [1] CRAN (R 3.6.0)
#>  knitr         1.25    2019-09-18 [1] CRAN (R 3.6.0)
#>  magick        2.2     2019-08-26 [1] CRAN (R 3.6.0)
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 3.6.0)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.6.0)
#>  mime          0.7     2019-06-11 [1] CRAN (R 3.6.0)
#>  pkgbuild      1.0.6   2019-10-09 [1] CRAN (R 3.6.0)
#>  pkgload       1.0.2   2018-10-29 [1] CRAN (R 3.6.0)
#>  png           0.1-7   2013-12-03 [1] CRAN (R 3.6.0)
#>  prettyunits   1.0.2   2015-07-13 [1] CRAN (R 3.6.0)
#>  processx      3.4.1   2019-07-18 [1] CRAN (R 3.6.0)
#>  ps            1.3.0   2018-12-21 [1] CRAN (R 3.6.0)
#>  R6            2.4.0   2019-02-14 [1] CRAN (R 3.6.0)
#>  Rcpp          1.0.2   2019-07-25 [1] CRAN (R 3.6.0)
#>  remotes       2.1.0   2019-06-24 [1] CRAN (R 3.6.0)
#>  rlang         0.4.0   2019-06-25 [1] CRAN (R 3.6.0)
#>  rmarkdown     1.16    2019-10-01 [1] CRAN (R 3.6.0)
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 3.6.0)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.6.0)
#>  stringi       1.4.3   2019-03-12 [1] CRAN (R 3.6.0)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 3.6.0)
#>  testthat      2.2.1   2019-07-25 [1] CRAN (R 3.6.0)
#>  usethis       1.5.1   2019-07-04 [1] CRAN (R 3.6.0)
#>  vcr         0.3.0.9100 2019-10-12 [1] Github (ropensci/vcr@fb90ac1)
#>  webmockr    0.4.0      2019-08-09 [1] CRAN (R 3.6.0)               
#>  withr         2.1.2   2018-03-15 [1] CRAN (R 3.6.0)
#>  xfun          0.10    2019-10-01 [1] CRAN (R 3.6.0)
#>  xml2          1.2.2   2019-08-09 [1] CRAN (R 3.6.0)
#>  yaml          2.2.0   2018-07-25 [1] CRAN (R 3.6.0)
#> 
#> [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library

[sckott]

thx for this detailed report! However, I don't see the vcr and webmockr versions you're using, can you include that.

[Rekyt]

I've updated my comment above but here are the versions of the two packages:

#>  vcr         0.3.0.9100 2019-10-12 [1] Github (ropensci/vcr@fb90ac1)
#>  webmockr    0.4.0      2019-08-09 [1] CRAN (R 3.6.0)       

[sckott]

thanks! will have a look

[sckott]

If you haven't tried yet, can you try using this option https://ropenscilabs.github.io/http-testing-book/vcr-configuration.html#config-preserve-exact-body-bytes and see if that helps.

[Rekyt]

I tried but I get the same error:

library("vcr")
#> Registered S3 method overwritten by 'httr':
#>   method                 from
#>   as.character.form_file crul

vcr::use_cassette("simple_test",{
  httr::GET("https://gallica.bnf.fr/iiif/ark:/12148/btv1b9055204k/f1/full/1500,750/0/native.png")
}, preserve_exact_body_bytes = TRUE)
#> CrulAdapter enabled!
#> HttrAdapter enabled!
#> net connect allowed
#> Error in rawToChar(x$content): caractère nul au milieu de la chaîne : '‰PNG\r\n\032\n\0\0\0\rIHDR\0\0\005Ü\0\0\002î\020\0\0\0\0ÁÛWÓ\0\0\0\tpHYs\0\0=„\0\0=„\001Õ¬¯t\0\0\200\0IDATxÚ$œ\005X\024ïûõ-vgæ‰éؤKi\004\005»\vƒ\026\003±»ƒî\020\020\005DQ\f\024\fìîîîîîîþêûüþïÎå¥+»3OÜ÷9Ÿ³A­G“î\035¾²{×Ü¢Ø\210‚  Ægƒ\005Ï©NŸ\034»8F8—5¨r\016³ßbÿÒ\t:·v®åîézÚ•qÛæ>Êé{ý‰.é>ŽÞ\rêÓ®÷ë'{\035ô2{ú{"/Þ'Èï¡ŸŸÿUÿé¾\017|×øÑ\036Ã\032¦\006T\004Î\f\030\0378)h\231ÿ<ïç\215¶5²ú-õ~ïëî\235à³?°]Ö>×½Ž{…x3\rÆ6(ôüàkð[ä³Á㉧λŸW\215·½ï\\ÿ\0ŸJßu¾+¼Z{®õ>ìÕß3ÌçŠ\177ŽïjïF^k½î{Å»ÅzFzA¯î>­<Ïz{{†¹\217oPâ½À;×}º\017ðÜë±×½\235ë†úÈeVýÚîbƒ‡\rN{TøÖ©oã|¹\201¾~yý¿ž~žË\\î¹'x.õ\230îÁyÕöþæE{\032¼ÿzOö¼ãáW_ñ|êiòÈh\220FÆciÐÄù®[\003ç\b'ÎùBýÞ>>ž”waƒïn›ëW{g6èåZÇCï1·~K·nõí|§4\230Ô Ì7Íû {•kvýî¾Çɹ"ê_kÐ˽бÔ+ÑiµÃOÇR‡\036¶\rl§;]²½b\215±«çpÌi‹Kmç$ÇT§^\016\025¶ý\0354ÇQNþŽM\235\032;\025Û‹†QNÛíV[Ê\034»¸\025ºrN\025î\017\035z:û:ç8‡¸Üp²s¸å²Õ!ß±“s\210ƒ`imwÞjc<e9m[`p5µ´Ýb{Ý6ÉvŠ­Þ:À¹ÌIt\nrhoß̶¶öÃî†\235§å“e\235u°õ°µ•íuÇEN\201ö׬ª9Þ.Ù¶­Ý\b;ÆöµÝ\006ûFÖŸ\016#í\220S\200ó\033»\231Úr«‡Ý\037û^vKìg;6sLµ
#> ejecting cassette: simple_test
#> CrulAdapter disabled!
#> HttrAdapter disabled!

^{Created on 2019-10-16 by the reprex package (v0.3.0)}

[sckott]

okay, thanks for trying that.

hmm, i wouldn't think rawToChar is what you'd want for a png. So you get the image and then you pass the bytes to rawToChar()?

[sckott]

ahhhh, i see now. tried running vcr on bi_image in your pkg, and the error comes from vcr in the traceback.

[sckott]

working on branch fix-raw-response-handling

[sckott]

@Rekyt changes on master now - reinstall - let me know if it works now

[Rekyt]

It does work now! Thank you so much!