build_site() fails behind a proxy

[Dripdrop12]

I've added http_proxy to .Renviron, but build_site() fails with the following error message:

Error in curl::curl_fetch_memory(url, handle = handle): Failed to connect to cloud.r-project.org port 443: Timed out

[jayhesselberth]

Try setting options(pkgdown.internet = FALSE) and then running build_site().

[Dripdrop12]

I got the same error :(

R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 16299)

Matrix products: default

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] pkgdown_1.1.0

[jayhesselberth]

You need the dev version of pkgdown for that option to have an effect.

[Dripdrop12]

I just installed 1.1.0.9000 and it didn't take. Is there a specific branch I can try?

[JiaxiangBU]

@jayhesselberth Hi, I reinstall package in the dev version, but the error happens again.

Here is the error message.

> devtools::install_github("r-lib/pkgdown")
Downloading GitHub repo r-lib/pkgdown@master
from URL https://api.github.com/repos/r-lib/pkgdown/zipball/master
Installing pkgdown
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 41627  100 41627    0     0  79903      0 --:--:-- --:--:-- --:--:-- 79898
Installing rematch2
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD  \
  INSTALL '/private/var/folders/yt/307v0c_945z1rh043jsdx1lh0000gn/T/Rtmp890agL/devtools14934ba6f999/rematch2'  \
  --library='/Users/vija/Downloads/180805_folder_01/tmp_jli/trans/projIN/add2evaluation/packrat/lib/x86_64-apple-darwin15.6.0/3.5.1'  \
  --install-tests 

* installing *binary* package ‘rematch2’ ...
* DONE (rematch2)
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD  \
  INSTALL  \
  '/private/var/folders/yt/307v0c_945z1rh043jsdx1lh0000gn/T/Rtmp890agL/devtools149327c265c3/r-lib-pkgdown-7f564dd'  \
  --library='/Users/vija/Downloads/180805_folder_01/tmp_jli/trans/projIN/add2evaluation/packrat/lib/x86_64-apple-darwin15.6.0/3.5.1'  \
  --install-tests 

* installing *source* package ‘pkgdown’ ...
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (pkgdown)
Reloading installed pkgdown
> 
> options(pkgdown.internet = FALSE)
> build_site()
| Building pkgdown site ══════════════════════════════════════════════
Reading from: '/Users/vija/Downloads/180805_folder_01/tmp_jli/trans/projIN/add2evaluation'
Writing to:   '/Users/vija/Downloads/180805_folder_01/tmp_jli/trans/projIN/add2evaluation/docs'
─ Initialising site ────────────────────────────────────────────────
Copying 'packrat/lib/x86_64-apple-darwin15.6.0/3.5.1/pkgdown/assets/favicon.ico' to 'favicon.ico'
Copying 'packrat/lib/x86_64-apple-darwin15.6.0/3.5.1/pkgdown/assets/pkgdown.css' to 'pkgdown.css'
Copying 'packrat/lib/x86_64-apple-darwin15.6.0/3.5.1/pkgdown/assets/pkgdown.js' to 'pkgdown.js'
─ Building home ──────────────────────────────────────────────────
It is recommended to use ‘given’ instead of ‘middle’.
It is recommended to use ‘given’ instead of ‘middle’.
It is recommended to use ‘given’ instead of ‘middle’.
It is recommended to use ‘given’ instead of ‘middle’.
Writing 'authors.html'
Reading 'LICENSE.md'
It is recommended to use ‘given’ instead of ‘middle’.
It is recommended to use ‘given’ instead of ‘middle’.
Writing 'LICENSE.html'
It is recommended to use ‘given’ instead of ‘middle’.
It is recommended to use ‘given’ instead of ‘middle’.
Writing 'LICENSE-text.html'
<simpleError in curl::curl_fetch_memory(url, handle = handle): Timeout was reached: Operation timed out after 10003 milliseconds with 0 out of 0 bytes received>
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Timeout was reached: Operation timed out after 10003 milliseconds with 0 out of 0 bytes received

"si"
#> [1] "si"

^{Created on 2018-11-09 by the reprex package (v0.2.1)}

Session info

devtools::session_info()
#> Session info -------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.5.1 (2018-07-02)
#>  system   x86_64, darwin15.6.0        
#>  ui       X11                         
#>  language (EN)                        
#>  collate  zh_CN.UTF-8                 
#>  tz       Asia/Shanghai               
#>  date     2018-11-09
#> Packages -----------------------------------------------------------------
#>  package   * version date       source        
#>  backports   1.1.2   2017-12-13 CRAN (R 3.5.0)
#>  base      * 3.5.1   2018-07-05 local         
#>  compiler    3.5.1   2018-07-05 local         
#>  datasets  * 3.5.1   2018-07-05 local         
#>  devtools    1.13.6  2018-06-27 CRAN (R 3.5.0)
#>  digest      0.6.18  2018-10-10 CRAN (R 3.5.0)
#>  evaluate    0.12    2018-10-09 CRAN (R 3.5.0)
#>  graphics  * 3.5.1   2018-07-05 local         
#>  grDevices * 3.5.1   2018-07-05 local         
#>  htmltools   0.3.6   2017-04-28 CRAN (R 3.5.0)
#>  knitr       1.20    2018-02-20 CRAN (R 3.5.0)
#>  magrittr    1.5     2014-11-22 CRAN (R 3.5.0)
#>  memoise     1.1.0   2017-04-21 CRAN (R 3.5.0)
#>  methods   * 3.5.1   2018-07-05 local         
#>  Rcpp        0.12.19 2018-10-01 CRAN (R 3.5.0)
#>  rmarkdown   1.10    2018-06-11 CRAN (R 3.5.0)
#>  rprojroot   1.3-2   2018-01-03 CRAN (R 3.5.0)
#>  stats     * 3.5.1   2018-07-05 local         
#>  stringi     1.2.4   2018-07-20 CRAN (R 3.5.0)
#>  stringr     1.3.1   2018-05-10 CRAN (R 3.5.0)
#>  tools       3.5.1   2018-07-05 local         
#>  utils     * 3.5.1   2018-07-05 local         
#>  withr       2.1.2   2018-03-15 CRAN (R 3.5.0)
#>  yaml        2.2.0   2018-07-25 CRAN (R 3.5.0)

[jayhesselberth]

Looks like internet connections are still an issue. Are proxies a special case?

[hadley]

This is low-priority for me, so I'm happy to review PRs, but I don't think it should block the 1.2 release.

[cderv]

After some search about this issue, (as I can test behind a proxy), I think I found where the issue comes from. I reproduced the error in an environment without internet.

build_site by default launch a new process (new_process = TRUE) and a new R is launch by callr::r(). In this new process, the options from the main process are not found.

options(pkgdown.internet = FALSE)
getOption("pkgdown.internet")
#> [1] FALSE
callr::r(function(...){getOption("pkgdown.internet")})
#> NULL

The option needs to be pass through like others

pkgdown/R/build.r

Line 315 in 9cbb675

options(crayon.enabled = crayon_enabled, crayon.colors = crayon_colors)

Also, we can confirm that it works, otherwise, with build_site(new_process = FALSE).

I'll make a PR for that.

However, related to internet connectivity, I think everything is not covered by the option. I found out that the internet connection can also be required by other package, and not only by packrat.
If we take the example of httr 📦 , some examples require internet. In build_reference_topic where examples from Rd files can be run, the option pkgdown.internet is not taken into account. It could cause error on 📦 where this is needed in example.
But for this we could assume that 📦 developer should deal with that in their examples (dontrun or skip if not internet) or use pkgdown with examples = FALSE argument.

This is the same for vignette. api-packages.Rmd in httr needs internet to build so, pkgdown fail if no internet, and the option change nothing.

The PR I will propose don't covers this.

[cderv]

@Dripdrop12 @JiaxiangBU you could try if with the PR, it is now correctly working for you

remotes::install_github("r-lib/pkgdown#877")

[JiaxiangBU]

@cderv Hi, thanks for your contribution, but I still get another error.

remotes::install_github("r-lib/pkgdown#877")
#> Skipping install of 'pkgdown' from a github remote, the SHA1 (012d46f5) has not changed since last install.
#>   Use `force = TRUE` to force installation
library(pkgdown)
build_site(new_process = FALSE)
#> Error: Can't find DESCRIPTION

^{Created on 2018-11-09 by the reprex package (v0.2.1)}

Session info

devtools::session_info()
#> Session info -------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.5.1 (2018-07-02)
#>  system   x86_64, darwin15.6.0        
#>  ui       X11                         
#>  language (EN)                        
#>  collate  zh_CN.UTF-8                 
#>  tz       Asia/Shanghai               
#>  date     2018-11-09
#> Packages -----------------------------------------------------------------
#>  package    * version    date       source                        
#>  backports    1.1.2      2017-12-13 CRAN (R 3.5.0)                
#>  base       * 3.5.1      2018-07-05 local                         
#>  commonmark   1.6        2018-09-30 CRAN (R 3.5.0)                
#>  compiler     3.5.1      2018-07-05 local                         
#>  curl         3.2        2018-03-28 CRAN (R 3.5.0)                
#>  datasets   * 3.5.1      2018-07-05 local                         
#>  devtools     1.13.6     2018-06-27 CRAN (R 3.5.0)                
#>  digest       0.6.18     2018-10-10 CRAN (R 3.5.0)                
#>  evaluate     0.12       2018-10-09 CRAN (R 3.5.0)                
#>  fs           1.2.6      2018-08-23 CRAN (R 3.5.0)                
#>  graphics   * 3.5.1      2018-07-05 local                         
#>  grDevices  * 3.5.1      2018-07-05 local                         
#>  htmltools    0.3.6      2017-04-28 CRAN (R 3.5.0)                
#>  knitr        1.20       2018-02-20 CRAN (R 3.5.0)                
#>  magrittr     1.5        2014-11-22 CRAN (R 3.5.0)                
#>  MASS         7.3-50     2018-04-30 CRAN (R 3.5.1)                
#>  memoise      1.1.0      2017-04-21 CRAN (R 3.5.0)                
#>  methods    * 3.5.1      2018-07-05 local                         
#>  pkgdown    * 1.1.0.9000 2018-11-09 Github (cderv/pkgdown@012d46f)
#>  R6           2.3.0      2018-10-04 CRAN (R 3.5.0)                
#>  Rcpp         0.12.19    2018-10-01 CRAN (R 3.5.0)                
#>  remotes      2.0.2      2018-10-30 CRAN (R 3.5.0)                
#>  rlang        0.3.0.1    2018-10-25 CRAN (R 3.5.0)                
#>  rmarkdown    1.10       2018-06-11 CRAN (R 3.5.0)                
#>  roxygen2     6.1.0      2018-07-27 CRAN (R 3.5.0)                
#>  rprojroot    1.3-2      2018-01-03 CRAN (R 3.5.0)                
#>  stats      * 3.5.1      2018-07-05 local                         
#>  stringi      1.2.4      2018-07-20 CRAN (R 3.5.0)                
#>  stringr      1.3.1      2018-05-10 CRAN (R 3.5.0)                
#>  tools        3.5.1      2018-07-05 local                         
#>  utils      * 3.5.1      2018-07-05 local                         
#>  withr        2.1.2      2018-03-15 CRAN (R 3.5.0)                
#>  xml2         1.2.0      2018-01-24 CRAN (R 3.5.0)                
#>  yaml         2.2.0      2018-07-25 CRAN (R 3.5.0)

When I add this arg force=T, pkgdown is successfully installled.

1. When use function build_site(new_process = FALSE), it works.
2. When use function build_site(), I get this error Error in pkgdown::build_site(...) : 参数没有用(mathjax = TRUE). It means build_site() forgets mathjax = TRUE.

[Dripdrop12]

@cderv - that fixed it on my end. I have not messed around with the settings, but a standard pkgdown::build_site() from your pull request works like a charm!