read_csv does not close unsuccessful connections

[dpprdan]

read_csv and friends trigger a closing unused connection warning whenever an attempt to read over a connection is not successful.

spelling error in url, so attempt is aborted.

readr::read_csv('https://cloud.r-project.org/CRAN_mirrorsZ.csv')
#> Error in open.connection(con, "rb"): HTTP error 404.

The timing of the warning is a bit erratic, i.e. it sometimes shows up after executing one more command, sometimes it takes a few more commands. I've often had success with something like:

readr::read_csv('https://cloud.r-project.org/CRAN_mirrors.csv', col_types = "ccccccccc")
#> # A tibble: 99 x 9
#>    Name   Country  City   URL    Host    Maintainer   OK    CountryCode Comment 
#>    <chr>  <chr>    <chr>  <chr>  <chr>   <chr>        <chr> <chr>       <chr>   
#>  1 0-Clo~ 0-Cloud  0-Clo~ https~ Automa~ winston # s~ 1     us          secure_~
#> # output shortened
#> Warning message:
#> In for (i in seq_len(n)) { :
#>   closing unused connection 3 (https://cloud.r-project.org/CRAN_mirrorsZ.csv)

Session info

devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.6.1 (2019-07-05)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language en                          
#>  collate  German_Germany.1252         
#>  ctype    German_Germany.1252         
#>  tz       Europe/Berlin               
#>  date     2019-12-05                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version    date       lib source                          
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 3.6.1)                  
#>  backports     1.1.5      2019-10-02 [1] CRAN (R 3.6.1)                  
#>  callr         3.3.2      2019-09-22 [1] CRAN (R 3.6.1)                  
#>  cli           1.1.0      2019-03-19 [1] CRAN (R 3.6.1)                  
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 3.6.1)                  
#>  curl          4.3        2019-12-02 [1] CRAN (R 3.6.1)                  
#>  desc          1.2.0      2018-05-01 [1] CRAN (R 3.6.1)                  
#>  devtools      2.2.1      2019-09-24 [1] CRAN (R 3.6.1)                  
#>  digest        0.6.23     2019-11-23 [1] CRAN (R 3.6.1)                  
#>  ellipsis      0.3.0      2019-09-20 [1] CRAN (R 3.6.1)                  
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 3.6.1)                  
#>  fansi         0.4.0      2018-10-05 [1] CRAN (R 3.6.1)                  
#>  fs            1.3.1      2019-05-06 [1] CRAN (R 3.6.1)                  
#>  glue          1.3.1      2019-03-12 [1] CRAN (R 3.6.1)                  
#>  highr         0.8        2019-03-20 [1] CRAN (R 3.6.1)                  
#>  hms           0.5.2      2019-10-30 [1] CRAN (R 3.6.1)                  
#>  htmltools     0.4.0      2019-10-04 [1] CRAN (R 3.6.1)                  
#>  knitr         1.26       2019-11-12 [1] CRAN (R 3.6.1)                  
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 3.6.1)                  
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 3.6.1)                  
#>  pillar        1.4.2      2019-06-29 [1] CRAN (R 3.6.1)                  
#>  pkgbuild      1.0.6      2019-10-09 [1] CRAN (R 3.6.1)                  
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 3.6.1)                  
#>  pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.6.1)                  
#>  prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.6.1)                  
#>  processx      3.4.1      2019-07-18 [1] CRAN (R 3.6.1)                  
#>  ps            1.3.0      2018-12-21 [1] CRAN (R 3.6.1)                  
#>  R6            2.4.1      2019-11-12 [1] CRAN (R 3.6.1)                  
#>  Rcpp          1.0.3      2019-11-08 [1] CRAN (R 3.6.1)                  
#>  readr         1.3.1.9000 2019-12-05 [1] Github (tidyverse/readr@1c045e5)
#>  remotes       2.1.0.9000 2019-07-22 [1] Github (r-lib/remotes@6e9eaa9)  
#>  rlang         0.4.2      2019-11-23 [1] CRAN (R 3.6.1)                  
#>  rmarkdown     1.18       2019-11-27 [1] CRAN (R 3.6.1)                  
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.6.1)                  
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.6.1)                  
#>  stringi       1.4.3      2019-03-12 [1] CRAN (R 3.6.0)                  
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 3.6.1)                  
#>  testthat      2.3.1      2019-12-01 [1] CRAN (R 3.6.1)                  
#>  tibble        2.1.3      2019-06-06 [1] CRAN (R 3.6.1)                  
#>  usethis       1.5.1      2019-07-04 [1] CRAN (R 3.6.1)                  
#>  utf8          1.1.4      2018-05-24 [1] CRAN (R 3.6.1)                  
#>  vctrs         0.2.0      2019-07-05 [1] CRAN (R 3.6.1)                  
#>  withr         2.1.2      2018-03-15 [1] CRAN (R 3.6.1)                  
#>  xfun          0.11       2019-11-12 [1] CRAN (R 3.6.1)                  
#>  yaml          2.2.0      2018-07-25 [1] CRAN (R 3.6.0)                  
#>  zeallot       0.1.0      2018-01-28 [1] CRAN (R 3.6.1)                  
#> 
#> [1] C:/Users/daniel/Documents/.R/win-library
#> [2] C:/Program Files/R/R-3.6.1/library

[cderv]

I can reproduce with this minimal reprex

con <- curl::curl('https://cloud.r-project.org/CRAN_mirrorsZ.csv')
open(con, 'rb')
rm(con)
gc()
con <- curl::curl('https://cloud.r-project.org/CRAN_mirrorsZ.csv')
readr:::read_connection(con)
rm(con)
gc()

removing the connection object and garbage collecting will trigger the warnings.
I believe this happens because the error prevent the connection to be closed properly.

readr/R/source.R

Lines 106 to 115 in 192cb1c

	read_connection <- function(con) {
	stopifnot(is.connection(con))
	if (!isOpen(con)) {
	open(con, "rb")
	on.exit(close(con), add = TRUE)
	}
	read_connection_(con, tempfile())
	}

Even if the close(con) is inside the on.exit

If we close, no warnings

# does no warn
con <- curl::curl('https://cloud.r-project.org/CRAN_mirrorsZ.csv')
readr:::read_connection(con)
close(con)
rm(con)
gc()

I think the on.exit(close(con), add = TRUE) should be before the open.

hope it helps.

[dpprdan]

Thanks @cderv for looking into it. I had the same thoughts, namely that the erratic behaviour might be due to garbage collection and that the on.exit() does not work in case the connection fails. I did not notice that on.exit() does not get triggered when open() fails, though 🤦‍♂. So obvious and clever at the same time. 🎉

[lock]

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/