tidy_bib_file does not find references

[Robinlovelace]

I noticed this issue a while back but thought it may have been a quirk with the particular .Rmd file I was using. However, I've reproduced the error message on many files since.

Please test the reproducible example below and see if you can reproduce the error:

# aim: test tidy_bib 

# install dev version
devtools::install_github("crsh/citr")
#> Skipping install of 'citr' from a github remote, the SHA1 (ecdb245b) has not changed since last install.
#>   Use `force = TRUE` to force installation

# download an Rmd file that contains references
download.file("https://github.com/ITSLeeds/stats19/raw/master/README.Rmd", "test.Rmd")

# download a bibliography
download.file("https://github.com/ITSLeeds/stats19/raw/master/vignettes/references.bib", "references.bib")

# clean the 'messy' bibliography
citr::tidy_bib_file(rmd_file = "test.Rmd", messy_bibliography = "references.bib")
#> Error in citr::tidy_bib_file(rmd_file = "test.Rmd", messy_bibliography = "references.bib"): Found no references in test.Rmd

^{Created on 2019-02-06 by the reprex package (v0.2.1)}

Session info

devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.5.1 (2018-07-02)
#>  os       Debian GNU/Linux 9 (stretch)
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       Etc/UTC                     
#>  date     2019-02-06                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version date       lib source                    
#>  assertthat    0.2.0   2017-04-11 [1] CRAN (R 3.5.1)            
#>  backports     1.1.2   2017-12-13 [1] CRAN (R 3.5.1)            
#>  base64enc     0.1-3   2015-07-28 [1] CRAN (R 3.5.1)            
#>  callr         3.0.0   2018-08-24 [1] CRAN (R 3.5.1)            
#>  citr          0.3.0   2019-02-06 [1] Github (crsh/citr@ecdb245)
#>  cli           1.0.1   2018-09-25 [1] CRAN (R 3.5.1)            
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 3.5.1)            
#>  curl          3.2     2018-03-28 [1] CRAN (R 3.5.1)            
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.5.1)            
#>  devtools      2.0.1   2018-10-26 [1] CRAN (R 3.5.1)            
#>  digest        0.6.18  2018-10-10 [1] CRAN (R 3.5.1)            
#>  evaluate      0.12    2018-10-09 [1] CRAN (R 3.5.1)            
#>  fs            1.2.6   2018-08-23 [1] CRAN (R 3.5.1)            
#>  glue          1.3.0   2018-07-17 [1] CRAN (R 3.5.1)            
#>  htmltools     0.3.6   2017-04-28 [1] CRAN (R 3.5.1)            
#>  httpuv        1.4.5   2018-07-19 [1] CRAN (R 3.5.1)            
#>  knitr         1.20    2018-02-20 [1] CRAN (R 3.5.1)            
#>  later         0.7.5   2018-09-18 [1] CRAN (R 3.5.1)            
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 3.5.1)            
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.5.1)            
#>  mime          0.6     2018-10-05 [1] CRAN (R 3.5.1)            
#>  miniUI        0.1.1.1 2018-05-18 [1] CRAN (R 3.5.1)            
#>  pkgbuild      1.0.2   2018-10-16 [1] CRAN (R 3.5.1)            
#>  pkgload       1.0.2   2018-10-29 [1] CRAN (R 3.5.1)            
#>  prettyunits   1.0.2   2015-07-13 [1] CRAN (R 3.5.1)            
#>  processx      3.2.1   2018-12-05 [1] CRAN (R 3.5.1)            
#>  promises      1.0.1   2018-04-13 [1] CRAN (R 3.5.1)            
#>  ps            1.2.1   2018-11-06 [1] CRAN (R 3.5.1)            
#>  R6            2.3.0   2018-10-04 [1] CRAN (R 3.5.1)            
#>  Rcpp          1.0.0   2018-11-07 [1] CRAN (R 3.5.1)            
#>  remotes       2.0.2   2018-10-30 [1] CRAN (R 3.5.1)            
#>  rlang         0.3.0.1 2018-10-25 [1] CRAN (R 3.5.1)            
#>  rmarkdown     1.10    2018-06-11 [1] CRAN (R 3.5.1)            
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 3.5.1)            
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.5.1)            
#>  shiny         1.2.0   2018-11-02 [1] CRAN (R 3.5.1)            
#>  stringi       1.2.4   2018-07-20 [1] CRAN (R 3.5.1)            
#>  stringr       1.3.1   2018-05-10 [1] CRAN (R 3.5.1)            
#>  testthat      2.0.1   2018-10-13 [1] CRAN (R 3.5.1)            
#>  usethis       1.4.0   2018-08-14 [1] CRAN (R 3.5.1)            
#>  withr         2.1.2   2018-03-15 [1] CRAN (R 3.5.1)            
#>  xtable        1.8-3   2018-08-29 [1] CRAN (R 3.5.1)            
#>  yaml          2.2.0   2018-07-25 [1] CRAN (R 3.5.1)            
#> 
#> [1] /usr/local/lib/R/site-library
#> [2] /usr/local/lib/R/library

[crsh]

Thanks for the excellent reproducible example. This should now be fixed. Let me know if you find other examples where it doesn't work as expected.

[Robinlovelace]

Confirmed: it now works again. Great work!

devtools::install_github("crsh/citr")
#> Skipping install of 'citr' from a github remote, the SHA1 (aab8006e) has not changed since last install.
#>   Use `force = TRUE` to force installation
download.file("https://github.com/ITSLeeds/stats19/raw/master/README.Rmd", "test.Rmd")
download.file("https://github.com/ITSLeeds/stats19/raw/master/vignettes/references.bib", "references.bib")
citr::tidy_bib_file(rmd_file = "test.Rmd", messy_bibliography = "references.bib")
#> Warning in do_read_bib(file, encoding = .Encoding, srcfile): encoding
#> 'native.enc' will be ignored
#> Removing 4 unneeded bibliography entries.
#> Writing 2 Bibtex entries ... OK
#> Results written to file 'references.bib'

^{Created on 2019-02-06 by the reprex package (v0.2.1)}

[Robinlovelace]

Heads-up @crsh, sorry, but I've just found another edge case. Please try this:

devtools::install_github("crsh/citr")
#> Skipping install of 'citr' from a github remote, the SHA1 (aab8006e) has not changed since last install.
#>   Use `force = TRUE` to force installation
download.file("https://github.com/ITSLeeds/TDS/blob/master/slides/2-software.Rmd", "test.Rmd")
download.file("https://github.com/ITSLeeds/stats19/raw/master/vignettes/references.bib", "references.bib")
citr::tidy_bib_file(rmd_file = "test.Rmd", messy_bibliography = "references.bib")
#> Error in citr::tidy_bib_file(rmd_file = "test.Rmd", messy_bibliography = "references.bib"): Found no references in test.Rmd

^{Created on 2019-02-06 by the reprex package (v0.2.1)}

Note: if I move the reference from line 105 to line 30, for example, it no longer claims there are no references in there. Please try to reproduce and let me know. Thanks!

[crsh]

Thanks, I'll take a look.

[Robinlovelace]

Great. I realised the file was changing. Have re-added a reference that is not found - the test should work now. Thanks!

[crsh]

This was due to overly aggressive sanitizing of the R Markdown file. I had not come across this issue because I hadn't tried it on xaringan slides. It should work now, but I get an error that reference was not found in the look-up bibliography. Thanks again for the excellent reports. They made debugging a breeze. 👍

[Robinlovelace]

Great fixes. They teach me lots about regex!