Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

read_csv does not close unsuccessful connections #1050

Closed
dpprdan opened this issue Dec 5, 2019 · 3 comments · Fixed by #1051
Closed

read_csv does not close unsuccessful connections #1050

dpprdan opened this issue Dec 5, 2019 · 3 comments · Fixed by #1051

Comments

@dpprdan
Copy link

dpprdan commented Dec 5, 2019

read_csv and friends trigger a closing unused connection warning whenever an attempt to read over a connection is not successful.

spelling error in url, so attempt is aborted.

readr::read_csv('https://cloud.r-project.org/CRAN_mirrorsZ.csv')
#> Error in open.connection(con, "rb"): HTTP error 404.

The timing of the warning is a bit erratic, i.e. it sometimes shows up after executing one more command, sometimes it takes a few more commands. I've often had success with something like:

readr::read_csv('https://cloud.r-project.org/CRAN_mirrors.csv', col_types = "ccccccccc")
#> # A tibble: 99 x 9
#>    Name   Country  City   URL    Host    Maintainer   OK    CountryCode Comment 
#>    <chr>  <chr>    <chr>  <chr>  <chr>   <chr>        <chr> <chr>       <chr>   
#>  1 0-Clo~ 0-Cloud  0-Clo~ https~ Automa~ winston # s~ 1     us          secure_~
#> # output shortened
#> Warning message:
#> In for (i in seq_len(n)) { :
#>   closing unused connection 3 (https://cloud.r-project.org/CRAN_mirrorsZ.csv)
Session info
devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.6.1 (2019-07-05)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language en                          
#>  collate  German_Germany.1252         
#>  ctype    German_Germany.1252         
#>  tz       Europe/Berlin               
#>  date     2019-12-05                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version    date       lib source                          
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 3.6.1)                  
#>  backports     1.1.5      2019-10-02 [1] CRAN (R 3.6.1)                  
#>  callr         3.3.2      2019-09-22 [1] CRAN (R 3.6.1)                  
#>  cli           1.1.0      2019-03-19 [1] CRAN (R 3.6.1)                  
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 3.6.1)                  
#>  curl          4.3        2019-12-02 [1] CRAN (R 3.6.1)                  
#>  desc          1.2.0      2018-05-01 [1] CRAN (R 3.6.1)                  
#>  devtools      2.2.1      2019-09-24 [1] CRAN (R 3.6.1)                  
#>  digest        0.6.23     2019-11-23 [1] CRAN (R 3.6.1)                  
#>  ellipsis      0.3.0      2019-09-20 [1] CRAN (R 3.6.1)                  
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 3.6.1)                  
#>  fansi         0.4.0      2018-10-05 [1] CRAN (R 3.6.1)                  
#>  fs            1.3.1      2019-05-06 [1] CRAN (R 3.6.1)                  
#>  glue          1.3.1      2019-03-12 [1] CRAN (R 3.6.1)                  
#>  highr         0.8        2019-03-20 [1] CRAN (R 3.6.1)                  
#>  hms           0.5.2      2019-10-30 [1] CRAN (R 3.6.1)                  
#>  htmltools     0.4.0      2019-10-04 [1] CRAN (R 3.6.1)                  
#>  knitr         1.26       2019-11-12 [1] CRAN (R 3.6.1)                  
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 3.6.1)                  
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 3.6.1)                  
#>  pillar        1.4.2      2019-06-29 [1] CRAN (R 3.6.1)                  
#>  pkgbuild      1.0.6      2019-10-09 [1] CRAN (R 3.6.1)                  
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 3.6.1)                  
#>  pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.6.1)                  
#>  prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.6.1)                  
#>  processx      3.4.1      2019-07-18 [1] CRAN (R 3.6.1)                  
#>  ps            1.3.0      2018-12-21 [1] CRAN (R 3.6.1)                  
#>  R6            2.4.1      2019-11-12 [1] CRAN (R 3.6.1)                  
#>  Rcpp          1.0.3      2019-11-08 [1] CRAN (R 3.6.1)                  
#>  readr         1.3.1.9000 2019-12-05 [1] Github (tidyverse/readr@1c045e5)
#>  remotes       2.1.0.9000 2019-07-22 [1] Github (r-lib/remotes@6e9eaa9)  
#>  rlang         0.4.2      2019-11-23 [1] CRAN (R 3.6.1)                  
#>  rmarkdown     1.18       2019-11-27 [1] CRAN (R 3.6.1)                  
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.6.1)                  
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.6.1)                  
#>  stringi       1.4.3      2019-03-12 [1] CRAN (R 3.6.0)                  
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 3.6.1)                  
#>  testthat      2.3.1      2019-12-01 [1] CRAN (R 3.6.1)                  
#>  tibble        2.1.3      2019-06-06 [1] CRAN (R 3.6.1)                  
#>  usethis       1.5.1      2019-07-04 [1] CRAN (R 3.6.1)                  
#>  utf8          1.1.4      2018-05-24 [1] CRAN (R 3.6.1)                  
#>  vctrs         0.2.0      2019-07-05 [1] CRAN (R 3.6.1)                  
#>  withr         2.1.2      2018-03-15 [1] CRAN (R 3.6.1)                  
#>  xfun          0.11       2019-11-12 [1] CRAN (R 3.6.1)                  
#>  yaml          2.2.0      2018-07-25 [1] CRAN (R 3.6.0)                  
#>  zeallot       0.1.0      2018-01-28 [1] CRAN (R 3.6.1)                  
#> 
#> [1] C:/Users/daniel/Documents/.R/win-library
#> [2] C:/Program Files/R/R-3.6.1/library
@cderv
Copy link
Contributor

cderv commented Dec 5, 2019

I can reproduce with this minimal reprex

con <- curl::curl('https://cloud.r-project.org/CRAN_mirrorsZ.csv')
open(con, 'rb')
rm(con)
gc()
con <- curl::curl('https://cloud.r-project.org/CRAN_mirrorsZ.csv')
readr:::read_connection(con)
rm(con)
gc()

removing the connection object and garbage collecting will trigger the warnings.
I believe this happens because the error prevent the connection to be closed properly.

readr/R/source.R

Lines 106 to 115 in 192cb1c

read_connection <- function(con) {
stopifnot(is.connection(con))
if (!isOpen(con)) {
open(con, "rb")
on.exit(close(con), add = TRUE)
}
read_connection_(con, tempfile())
}

Even if the close(con) is inside the on.exit

If we close, no warnings

# does no warn
con <- curl::curl('https://cloud.r-project.org/CRAN_mirrorsZ.csv')
readr:::read_connection(con)
close(con)
rm(con)
gc()

I think the on.exit(close(con), add = TRUE) should be before the open.

hope it helps.

@dpprdan
Copy link
Author

dpprdan commented Dec 6, 2019

Thanks @cderv for looking into it. I had the same thoughts, namely that the erratic behaviour might be due to garbage collection and that the on.exit() does not work in case the connection fails. I did not notice that on.exit() does not get triggered when open() fails, though 🤦‍♂. So obvious and clever at the same time. 🎉

@lock
Copy link

lock bot commented Jun 24, 2020

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/

@lock lock bot locked and limited conversation to collaborators Jun 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants