broken api to download GFW data?

[Jo-Schie]

Hi @goergen95 . The packages download function seem to be broken.

library(raster)
library(igraph)
library(tibble)
library(sf)
library(magrittr)
library(dplyr)
library(tidyr)
library(ggplot2)
library(stringr)
library(mapme.forest)

# read in polygons of interest
aoi = st_read(system.file("extdata", "aoi_polys.gpkg", package = "mapme.forest"))

# download GFW data for the area of interest
raster_files = downloadfGFW(shape = aoi,
                            basename = "pkgTest",
                            dataset = "GFC-2018-v1.6",
                            outdir = "../data/",
                            keepTmpFiles = T,
                            .tmpdir = "../data/tmp")

gives this error message:

trying URL 'https://storage.googleapis.com/earthenginepartners-hansen/GFC-2018-v1.6/Hansen_GFC-2018-v1.6_treecover2000_20N_100E.tif'
Content type 'application/octet-stream' length 518834343 bytes (494.8 MB)
==================================================
downloaded 494.8 MB

trying URL 'https://storage.googleapis.com/earthenginepartners-hansen/GFC-2018-v1.6/Hansen_GFC-2018-v1.6_lossyear_20N_100E.tif'
Content type 'image/tiff' length 86248019 bytes (82.3 MB)
==================================================
downloaded 82.3 MB

trying URL 'http://gfw2-data.s3.amazonaws.com/climate/Hansen_emissions/2018_loss/per_pixel/20N_100E_tCO2_pixel_AGB_masked_by_loss.tif'
Error in download.file(urls[i], localname) : 
  cannot open URL 'http://gfw2-data.s3.amazonaws.com/climate/Hansen_emissions/2018_loss/per_pixel/20N_100E_tCO2_pixel_AGB_masked_by_loss.tif'
In addition: Warning message:
In download.file(urls[i], localname) :
  cannot open URL 'http://gfw2-data.s3.amazonaws.com/climate/Hansen_emissions/2018_loss/per_pixel/20N_100E_tCO2_pixel_AGB_masked_by_loss.tif': HTTP status was '403 Forbidden'

Any ideas how to quickfix this?

[goergen95]

I can confirm that the Biomass Loss Layer currently cannot be downloaded due missing access rights. Seems like Amazon has changed its policy... We might need to include a authorization function. The only option for a rather "quick fix" that I see is to look if the data is available from another server.

[Ohm-Np]

Hi @goergen95 , I need your help here.

While running this code:

# get the file paths to the raster files
treeCover = "../../datalake/mapme.protectedareas/input/global_forest_watch/treecover2000.tif"
lossYear = "../../datalake/mapme.protectedareas/input/global_forest_watch/lossyear.tif"
co2Layer = "../../datalake/mapme.protectedareas/input/global_forest_watch/co2_emission_.tif"

# read the region of interest
roi <- 
  st_read("../test/sampling_AOIs_2021-09-28.gpkg")
# transform roi to wgs84
roi <- 
  st_transform(roi, 
               "+proj=longlat +datum=WGS84 +no_defs")
# path to the installed grass
grass = "/usr/lib/grass78"

# zonal statistics
roi_stats = statsGRASS(grass = grass, 
                       addon_base = "./data-raw/addons", 
                       areas = roi[1, ], 
                       tree_cover = treeCover, 
                       tree_loss = lossYear, 
                       tree_co2 = co2Layer, 
                       idcol =  "UID", 
                       thresholdClump = 6, 
                       thresholdCover = 10, 
                       years = 2001:2020, 
                       saveRaster = T,
                       outdir = "../test/sampling_results/rasters/")

I got this error:

Error in execGRASS("g.extension", extension = "r.area", prefix = addon_base,  : 
  The command:
g.extension extension=r.area prefix=./data-raw/addons operation=add
produced an error (1) during execution:
Fetching <r.area> from GRASS GIS Addons repository (be patient)...
svn: E170000: URL 'https://github.com/OSGeo/grass-addons/trunk/grass7/raster/r.area' doesn't exist
ERROR: GRASS Addons <r.area> not found

Is it related to grass installation on our server or due to the addons provider? Also, there is problem in accessing raster files as mentioned by Johannes, however, for now, we have all the rasters required for LA in the datalake.

[goergen95]

Hi @Ohm-Np,
I think that issue occurred before on the server. g.extension seems to be unable to install the r.area extension so that the function fails with the above error. Please make sure to install subversion along with your GRASS installation. For Ubuntu systems that would be something along the following lines:

sudo apt-get install subversion

Hope that helps!

[Ohm-Np]

Thank you @goergen95, I will let you know if it works.

[Jo-Schie]

there is currently a quickfix implemented for the download issue. The processing issue was resolved by reinstalling subversion