install_version() fails in rocker container depending on value of argument repos

[jdblischak]

On my local machine running Ubuntu 18.04, install_version() is able to install a package from various CRAN mirrors. All of the below work:

remotes::install_version("rstudioapi", "0.1", repos = "https://cloud.r-project.org")
remotes::install_version("rstudioapi", "0.1", repos = "https://cran.rstudio.com")
remotes::install_version("rstudioapi", "0.1", repos = "https://cran.case.edu")

Session information for local machine

R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3
LAPACK: /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3.10.3

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.6.1 tools_3.6.1    remotes_2.1.0  packrat_0.5.0

In contrast, in the Docker image rocker/r-base, install_version() only works for some CRAN mirrors:

remotes::install_version("rstudioapi", "0.1", repos = "https://cloud.r-project.org")
## Error in package_find_repo(package, repos) : 
##   couldn't find package 'rstudioapi'
remotes::install_version("rstudioapi", "0.1", repos = "https://cran.rstudio.com")
## Error in package_find_repo(package, repos) : 
##   couldn't find package 'rstudioapi'
remotes::install_version("rstudioapi", "0.1", repos = "https://cran.case.edu")
## Downloading package from url: https://cran.case.edu/src/contrib/Archive/rstudioapi/rstudioapi_0.1.tar.gz
## <installation output ommitted>

Session information for rocker/r-base

R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux bullseye/sid

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.8.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.8.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.6.1 tools_3.6.1    remotes_2.1.0 

[aronatkins]

Some CRAN mirrors do not respond with gzip encoded responses. Could this be related?

Using curl -I <url> for some quick experimentation:

The https://cran.case.edu/src/contrib/Meta/archive.rds URL responds with:

Content-Encoding: x-gzip

The //cran.rstudio.com/src/contrib/Meta/archive.rds URL responds without a Content-Encoding header.

I can use readRDS to reach each file after using curl to download the archive.rds file.

curl -o rstudio.rds https://cran.rstudio.com/src/contrib/Meta/archive.rds
R --slave -e 'foo <- readRDS("rstudio.rds")'

curl -o case.rds https://cran.case.edu/src/contrib/Meta/archive.rds
R --slave -e 'foo <- readRDS("case.rds")'

Directly attempting to read the URL succeeds when the content-encoding header is present (cran.case.edu) but fails when not present (cran.rstudio.com):

R --slave -e 'foo <- readRDS(gzcon(url("https://cran.case.edu/src/contrib/Meta/archive.rds")))'

R --slave -e 'foo <- readRDS(gzcon(url("https://cran.rstudio.com/src/contrib/Meta/archive.rds")))'
# Error in readRDS(gzcon(url("https://cran.rstudio.com/src/contrib/Meta/archive.rds"))) :
#   ReadItem: unknown type 65, perhaps written by later version of R

[gaborcsardi]

R --slave -e 'foo <- readRDS(gzcon(url("https://cran.rstudio.com/src/contrib/Meta/archive.rds")))'

works for me on macOS Mojave, with R 3.6.2.

Maybe it depends on the curl library version? I have

❯ otool -L /Library/Frameworks/R.framework/Versions/Current/Resources/modules/internet.so
/Library/Frameworks/R.framework/Versions/Current/Resources/modules/internet.so:
	internet.so (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 8.0.0)
	/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libR.dylib (compatibility version 3.6.0, current version 3.6.2)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1259.11.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

[jimhester]

It does not work for me on macOS Catalina, my versions are nearly the same as yours.

> R --slave -e 'foo <- readRDS(gzcon(url("https://cran.rstudio.com/src/contrib/Meta/archive.rds")))'
Error in readRDS(gzcon(url("https://cran.rstudio.com/src/contrib/Meta/archive.rds"))) :
  unknown input format
Execution halted
> otool -L /Library/Frameworks/R.framework/Versions/Current/Resources/modules/internet.so
/Library/Frameworks/R.framework/Versions/Current/Resources/modules/internet.so:
        internet.so (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 8.0.0)
        /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libR.dylib (compatibility version 3.6.0, current version 3.6.1)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1259.11.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

[gaborcsardi]

This is a 5 year old base R bug, that has always been there for url() + libcurl:
https://www.mail-archive.com/r-devel@r-project.org/msg41648.html

We'll need a workaround in remotes: avoid url() and download to a temp file.