Server down? #169

rdatasculptor · 2020-10-02T09:14:38Z

I am trying to download maps, but the server does not respond properly. Is it down?

If I try e.g. one of the examples in the package:

getData('worldclim', var='tmin', res=0.5, lon=5, lat=45)

Then I get this error:

trying URL 'https://biogeo.ucdavis.edu/data/climate/worldclim/1_4/tiles/cur/tmin_16.zip'
Error in utils::download.file(url = aurl, destfile = fn, method = "auto",  : 
  cannot open URL 'https://biogeo.ucdavis.edu/data/climate/worldclim/1_4/tiles/cur/tmin_16.zip'

I am interested in this map in particular:

nld <- getData("GADM", country = "NLD", level = 1)

But then I get this error:

trying URL 'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_NLD_1_sp.rds'
Error in utils::download.file(url = aurl, destfile = fn, method = "auto",  : 
  cannot open URL 'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_NLD_1_sp.rds'

I hope it can be solved soon.
Thank you in advance!

The text was updated successfully, but these errors were encountered:

kellybarr · 2020-10-05T14:40:28Z

Downloads from worldclim also fail!

Robinlovelace · 2020-10-06T20:24:19Z

In case it helps, you can find copies of most of the data sources elsewhere online. This was my fix:

# old way:
# worldclim_prec = getData(name = "worldclim", var = "prec", res = 10)
# new way (after searching for the relevant dataset and finding it, on Zenodo or similar academic website, and re-uploading it):
if(!file.exists("prec_10m_bil.zip")) {
  u = "https://github.com/Robinlovelace/geocompr/releases/download/1.2/prec_10m_bil.zip"
  download.file(u, "prec_10m_bil.zip")
  unzip("prec_10m_bil.zip", exdir = "prec")
}
worldclim_prec = stack(list.files("prec", pattern = "bil", full.names = TRUE))

rhijmans · 2020-10-25T04:44:40Z

server is working again...

Robinlovelace · 2020-10-27T10:37:22Z

Hi @rhijmans great it's fixed. Running the code, it seems to be quite slow, e.g. taking ~15 seconds to download this file:

system.time(download.file("https://biogeo.ucdavis.edu/data/climate/worldclim/1_4/grid/cur/prec_10m_bil.zip", "prec_10m_bil.zip"))
trying URL 'https://biogeo.ucdavis.edu/data/climate/worldclim/1_4/grid/cur/prec_10m_bil.zip'
Content type 'application/zip' length 5118786 bytes (4.9 MB)
==================================================
downloaded 4.9 MB

   user  system elapsed 
  0.121   0.200  14.380

rdatasculptor · 2020-10-27T11:57:00Z

server is working again...

Thanks!

Robinlovelace · 2020-10-30T17:43:30Z

It's still causing issues on our tests, heads-up @Nowosad: https://github.com/Robinlovelace/geocompr/runs/1313968422#step:10:4425

Robinlovelace · 2020-10-30T17:59:47Z

Think it's fixed now, just a heads-up...

Robinlovelace · 2021-01-24T14:12:54Z

Heads-up @rhijmans I'm still seeing very slow responses from this endpoint: https://biogeo.ucdavis.edu/data/climate/worldclim/1_4/grid/cur/prec_10m_bil.zip

Flagging here as it caused our CI check to fail: https://github.com/Robinlovelace/geocompr/runs/1757308011#step:4:4421

Is there any way it can be hosted on a faster responding server? Very grateful for the data in any case, many thanks.

kuntalnr · 2021-08-01T21:55:00Z

Hello. I am still having this problem. I have posted my question in the link below.

my R session is

R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

Can anyone help me. My question is also posted on stackoverflow.

https://stackoverflow.com/questions/68614602/error-in-download-worldclim-data-using-raster-getdata-function?noredirect=1#comment121261135_68614602

rhijmans · 2021-08-03T08:17:21Z

The service has been restored

tatiana0918 · 2021-08-17T15:19:03Z

I HAVE THE SAME PROBLEM IN R WITH WALLACE

that is my error error in utils::download.file: download from 'https://biogeo.ucdavis.edu/data/climate/worldclim/1_4/grid/cur/bio_2-5m_bil.zip' failed for Mac
help me

cwh7 · 2021-08-27T20:27:44Z

Is the service down again? I am getting an error when running

getData("GADM", country = "usa", level = 2)

trying URL 'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_USA_2_sp.rds'
Error in utils::download.file(url = aurl, destfile = fn, method = "auto", :
cannot open URL 'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_USA_2_sp.rds'
In addition: Warning message:
In utils::download.file(url = aurl, destfile = fn, method = "auto", :
InternetOpenUrl failed: '.'

AlbertJML · 2021-10-02T19:34:59Z

It seems it is down again. I tried with
getData(name = "GADM", country="BRA", level=3 )
And I got the same error message ("trying URL.......cannot open URL....")
Hope the server is working again soon

rhijmans · 2021-10-02T19:49:41Z

Down for maintenance. Expected back by Monday.

JohnCoene · 2021-10-04T16:29:31Z

It looks like it's back up but with different permissions.

It now returns 403, e.g. https://data.biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_IND_2_sp.rds

ruthrivkin · 2021-10-04T19:01:34Z

Following up to say I also can't access the datasets, either directly on the WorldClim site or via R. Everything I try is returning a 403 Forbidden error message.

rhijmans · 2021-10-04T19:38:04Z

Update: The site should be back up sometime on Wednesday, October 6th

mickayla32 · 2021-10-07T00:33:59Z

Does anyone know if this issue is resolved? I'm still getting the following error message, and wasn't sure if it was because of the server being down still...
trying URL 'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_USA_2_sp.rds'
Error in utils::download.file(url = aurl, destfile = fn, method = "auto", :
cannot open URL 'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_USA_2_sp.rds'
In addition: Warning message:
In utils::download.file(url = aurl, destfile = fn, method = "auto", :
URL 'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_USA_2_sp.rds': status was 'Peer certificate cannot be authenticated with given CA certificates'

rhijmans · 2021-10-07T01:03:37Z

It has been working for me. I get

x = raster::getData("GADM", country="USA", level=2)
#trying URL 'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_USA_2_sp.rds'
#Content type '�' length 13956072 bytes (13.3 MB)
#downloaded 13.3 MB

rhijmans · 2021-10-07T01:04:19Z

And you can also click on the URL in your message...

melissademmitt · 2021-10-07T13:01:18Z

@mickayla32 Have you gotten yours to work? I am still getting the same error message as well, as of this morning.

mickayla32 · 2021-10-07T16:36:30Z

@melissademmitt No, mine is still giving me the same error message... I tried restarting R as well. Does anyone have any suggestions?

rhijmans · 2021-10-07T17:52:19Z

For now, a work-around...

library(raster)
f <-  'https://biogeo.ucdavis.edu/data/gadm3.6/Rsp/gadm36_USA_2_sp.rds'
b <- basename(f)
download.file(f, b, mode="wb")
x <- readRDS(b)

mickayla32 · 2021-10-07T19:39:40Z

@rhijmans Thank you for this! I tried this, and it seems like the problem must just be with my RStudio and/or computer, because I'm still getting the same error message when I run the download.file function

rhijmans · 2021-10-07T19:45:00Z

The error message is 'Peer certificate cannot be authenticated with given CA certificates suggests that somehow the https address cannot be verified on your computer. Perhaps something on the Internet or in your cache is out of sync; and that might go away in a couple of days (or restart?).

mickayla32 · 2021-10-07T20:46:29Z

So, I tried adding --
method = "curl"
to the download.file function, and got this error:
curl: (60) SSL certificate problem: certificate has expired

So then I added
method = "wget"
to the download.file function instead, and it worked! Do you know if this means theres something wrong with curl on my computer? Sorry, I'm not very experienced in coding!

rhijmans · 2021-10-08T03:04:04Z

Thanks, that is interesting. I do not know why this happens. Perhaps something is wrong with our certificate, but if it is only with curl it is perhaps curl related (where it checks the certificates). Perhaps that will fix itself (?) .
Also linking this discussion to a related issue

Having hard-coded `method="auto"` prevents users from changing the method with `options("download.file.method" = ...)`. However, if this option is not set, the `method = "auto"` is chosen anyway, so there wouldn't be any unexpected behaviour changes for users. This should help with errors like - rspatial#169 (comment) - since users would be able to call: ```r op <- options("download.file.method" = "wget", "download.file.extra" = "--no-check-certificate") getData(...) options(op) ```

mnazarov · 2021-10-28T16:07:49Z

@rhijmans I also experienced a similar error with expired CA certificates, but curiously in the opposite way than @mickayla32 - it works with curl, and doesn't work with wget. The manual steps are not very convenient in my case, since the package is used via another package EnvRtype, so I propose the change as in #230 so that the method can be modified via options

TheTams · 2021-11-03T12:00:17Z

I am having the same issue with curl (and am also not very experienced in coding)

curl: (60) SSL certificate problem: certificate has expired"

wget doesn't solve the problem for me.

download.file(url, destfile, method ="wget")
sh: wget: command not found
Error in download.file(url, destfile, method = "wget") :
'wget' call had nonzero exit status
In addition: Warning message:
In system(paste("wget", paste(extra, collapse = " "), shQuote(url), :
error in running command

The work around results in

download.file(url, destfile, method = "wb")
Error in match.arg(method, c("auto", "internal", "libcurl", "wget", "curl", :
'arg' should be one of “auto”, “internal”, “libcurl”, “wget”, “curl”, “lynx”

I can download manually, and that ok for now in my testing phase, but won't be when I want to run higher res logging into the research computer remotely, if it has the same errors.

gsadoti · 2022-04-07T00:10:33Z

Hi @rhijmans, is the Wordclim server down? I can't seem to get anything from within R, links don't appear to work from www.worldclim.org, and biogeo.ucdavis.edu appears to be offline. Cheers, Giancarlo

rhijmans · 2022-04-07T01:17:20Z

Yes, see #263

Nowosad mentioned this issue Oct 4, 2020

Vulnerability to external file geocompx/geocompr#576

Closed

rhijmans closed this as completed Oct 25, 2020

mnazarov mentioned this issue Oct 28, 2021

remove method="auto" from download.file call #230

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server down? #169

Server down? #169

rdatasculptor commented Oct 2, 2020 •

edited

kellybarr commented Oct 5, 2020

Robinlovelace commented Oct 6, 2020

rhijmans commented Oct 25, 2020

Robinlovelace commented Oct 27, 2020

rdatasculptor commented Oct 27, 2020

Robinlovelace commented Oct 30, 2020

Robinlovelace commented Oct 30, 2020

Robinlovelace commented Jan 24, 2021

kuntalnr commented Aug 1, 2021

rhijmans commented Aug 3, 2021

tatiana0918 commented Aug 17, 2021

cwh7 commented Aug 27, 2021

AlbertJML commented Oct 2, 2021

rhijmans commented Oct 2, 2021

JohnCoene commented Oct 4, 2021

ruthrivkin commented Oct 4, 2021

rhijmans commented Oct 4, 2021 •

edited

mickayla32 commented Oct 7, 2021

rhijmans commented Oct 7, 2021

rhijmans commented Oct 7, 2021

melissademmitt commented Oct 7, 2021

mickayla32 commented Oct 7, 2021

rhijmans commented Oct 7, 2021

mickayla32 commented Oct 7, 2021

rhijmans commented Oct 7, 2021

mickayla32 commented Oct 7, 2021

rhijmans commented Oct 8, 2021

mnazarov commented Oct 28, 2021

TheTams commented Nov 3, 2021 •

edited

gsadoti commented Apr 7, 2022

rhijmans commented Apr 7, 2022

Server down? #169

Server down? #169

Comments

rdatasculptor commented Oct 2, 2020 • edited

kellybarr commented Oct 5, 2020

Robinlovelace commented Oct 6, 2020

rhijmans commented Oct 25, 2020

Robinlovelace commented Oct 27, 2020

rdatasculptor commented Oct 27, 2020

Robinlovelace commented Oct 30, 2020

Robinlovelace commented Oct 30, 2020

Robinlovelace commented Jan 24, 2021

kuntalnr commented Aug 1, 2021

rhijmans commented Aug 3, 2021

tatiana0918 commented Aug 17, 2021

cwh7 commented Aug 27, 2021

AlbertJML commented Oct 2, 2021

rhijmans commented Oct 2, 2021

JohnCoene commented Oct 4, 2021

ruthrivkin commented Oct 4, 2021

rhijmans commented Oct 4, 2021 • edited

mickayla32 commented Oct 7, 2021

rhijmans commented Oct 7, 2021

rhijmans commented Oct 7, 2021

melissademmitt commented Oct 7, 2021

mickayla32 commented Oct 7, 2021

rhijmans commented Oct 7, 2021

mickayla32 commented Oct 7, 2021

rhijmans commented Oct 7, 2021

mickayla32 commented Oct 7, 2021

rhijmans commented Oct 8, 2021

mnazarov commented Oct 28, 2021

TheTams commented Nov 3, 2021 • edited

gsadoti commented Apr 7, 2022

rhijmans commented Apr 7, 2022

rdatasculptor commented Oct 2, 2020 •

edited

rhijmans commented Oct 4, 2021 •

edited

TheTams commented Nov 3, 2021 •

edited