Error in !out[a] : invalid argument type

[ndmsc]

Hi,
When downloading MOD11C3 data using getHdf function from MODIS package I ran into this error:
Error in !out[a] : invalid argument type

This is the command I used:
b1 <- getHdf(product = "MOD11C3", begin = "2000.02.01", quiet=TRUE)

It downloads the first file and then the error occurs.
Basically, I want to download all monthly land surface temperature from February 2000 till current date.
Can you please help me understanding the error and how to avoid it?

Thanks

[admccurdy]

I get the same error when I run the above code or any of the getHdf() examples in the documentation. Any ideas would be much appreciated.

[admccurdy]

To note the hdf files appear to download properly but then the command exits with the error.

[fdetsch]

Thanks @ndmsc and @admccurdy for reporting! And sorry for my late reply - I haven't been around for the last couple of weeks...

Unfortunately, I cannot reproduce said error:

# devtools::install_github("MatMatt/MODIS", ref = "develop")
library(MODIS)

b1 = getHdf("MOD11C3", begin = "2000.02.01", end = "2000.12.31", quiet = TRUE)
b1

not only downloads all required files, but also returns the list of output file names:

$MOD11C3.006
 [1] "E:/MODIS_ARC/MODIS/MOD11C3.006/200002~1.01/MOD11C3.A2000032.006.2015058203309.hdf"
 [2] "E:/MODIS_ARC/MODIS/MOD11C3.006/200003~1.01/MOD11C3.A2000061.006.2015058070048.hdf"
 [3] "E:/MODIS_ARC/MODIS/MOD11C3.006/200004~1.01/MOD11C3.A2000092.006.2015060123950.hdf"
 [4] "E:/MODIS_ARC/MODIS/MOD11C3.006/200005~1.01/MOD11C3.A2000122.006.2015062180458.hdf"
 [5] "E:/MODIS_ARC/MODIS/MOD11C3.006/200006~1.01/MOD11C3.A2000153.006.2015064110836.hdf"
 [6] "E:/MODIS_ARC/MODIS/MOD11C3.006/200007~1.01/MOD11C3.A2000183.006.2015066033219.hdf"
 [7] "E:/MODIS_ARC/MODIS/MOD11C3.006/200008~1.01/MOD11C3.A2000214.006.2015069162120.hdf"
 [8] "E:/MODIS_ARC/MODIS/MOD11C3.006/200009~1.01/MOD11C3.A2000245.006.2015071235605.hdf"
 [9] "E:/MODIS_ARC/MODIS/MOD11C3.006/200010~1.01/MOD11C3.A2000275.006.2015075090705.hdf"
[10] "E:/MODIS_ARC/MODIS/MOD11C3.006/200011~1.01/MOD11C3.A2000306.006.2015111040233.hdf"
[11] "E:/MODIS_ARC/MODIS/MOD11C3.006/200012~1.01/MOD11C3.A2000336.006.2015111162742.hdf"

This works both on Windows and Linux. Here's my session info:

> sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=German_Germany.1252  LC_CTYPE=German_Germany.1252    LC_MONETARY=German_Germany.1252
[4] LC_NUMERIC=C                    LC_TIME=German_Germany.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] MODIS_1.1.1   shiny_1.0.5   raster_2.6-7  sp_1.2-5      mapdata_2.2-6 maps_3.2.0   

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.14    magrittr_1.5    units_0.4-6     devtools_1.13.4 xtable_1.8-2    lattice_0.20-35
 [7] R6_2.2.2        ptw_1.9-12      udunits2_0.13   tools_3.4.3     parallel_3.4.3  rgdal_1.2-16   
[13] grid_3.4.3      e1071_1.6-8     DBI_0.7         withr_2.1.0     class_7.3-14    htmltools_0.3.6
[19] yaml_2.1.15     mapedit_0.3.2   digest_0.6.12   sf_0.5-5        nloptr_1.0.4    bitops_1.0-6   
[25] RCurl_1.95-4.8  memoise_1.1.0   mime_0.5        compiler_3.4.3  classInt_0.1-24 XML_3.98-1.9   
[31] httpuv_1.3.5

[ndmsc]

It's strange, I have installed R 3.4.3 and managed to get exactly the same environment as yours @fdetsch as shown by sessionInfo() hereafter. But the error is still there. MODIS_1.1.1 package is installed using devtools::install_github("MatMatt/MODIS", ref = "develop") otherwise install.packages() will install MODIS_1.1.0

Any help with this would be much appreciated.

sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 15063)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] shiny_1.0.5 MODIS_1.1.1 raster_2.6-7 sp_1.2-5 mapdata_2.2-6 maps_3.2.0

loaded via a namespace (and not attached):
[1] Rcpp_0.12.14 magrittr_1.5 units_0.4-6 devtools_1.13.4 xtable_1.8-2 lattice_0.20-35 R6_2.2.2 ptw_1.9-12
[9] udunits2_0.13 tools_3.4.3 parallel_3.4.3 rgdal_1.2-16 grid_3.4.3 e1071_1.6-8 DBI_0.7 withr_2.1.0
[17] class_7.3-14 htmltools_0.3.6 mapedit_0.3.2 digest_0.6.12 sf_0.5-5 nloptr_1.0.4 bitops_1.0-6 RCurl_1.95-4.8
[25] memoise_1.1.0 mime_0.5 compiler_3.4.3 classInt_0.1-24 XML_3.98-1.9 httpuv_1.3.5

[fdetsch]

Very strange indeed! What's your output of MODISoptions()? Here's mine:

STORAGE:
_______________
localArcPath : D:/programming/r/MODIS/MODIS_ARC/ 
outDirPath   : D:/programming/r/MODIS/MODIS_ARC/PROCESSED/ 


DOWNLOAD:
_______________
MODISserverOrder : LAADS, LPDAAC 
dlmethod         : wget 
stubbornness     : high 
wait             : 0.5 
quiet            : FALSE 


PROCESSING:
_______________
GDAL           : GDAL 2.2.1, released 2017/06/23 
MRT            : Version 4.1 (March 2011) 
pixelSize      : asIn 
outProj        : +init=epsg:4326 
resamplingType : NN 
dataFormat     : GTiff

[ndmsc]

Here is the output of MODISoptions()

MODISoptions()
'MRT_HOME' not set/found! MRT is NOT enabled! See: 'https://lpdaac.usgs.gov/tools/modis_reprojection_tool'

STORAGE:

---

localArcPath : C:/RDMS/data_backup/
outDirPath : C:/RDMS/tmp/

DOWNLOAD:

---

MODISserverOrder : LPDAAC
dlmethod : auto
stubbornness : high
wait : 0.5
quiet : FALSE

PROCESSING:

---

GDAL : Not available. Use 'MODIS:::checkTools('GDAL')' for more information!
MRT : Not available. Use 'MODIS:::checkTools('MRT')' for more information!
pixelSize : asIn
outProj : asIn
resamplingType : NN
dataFormat : GTiff

[fdetsch]

Seems like the package is not able to find GDAL on your system. Is it in your 'Path' variable? For example, on my machine, the required GDAL executables are in C:\Program Files\QGIS 2.18\bin (that's the 'Path' entry) and MODIS:::checkTools() returns

> MODIS:::checkTools()
Checking availability of MRT:
  'MRT_HOME' found: C:/Program Files (x86)/MODIS Reprojection Tool 
  'MRT_DATA_DIR' found: C:/Program Files (x86)/MODIS Reprojection Tool/data 
   MRT enabled, settings are fine!
Checking availability of 'FWTools/OSGeo4W' (GDAL with HDF4 support for Windows):
   OK, GDAL 2.2.1, released 2017/06/23 found!

[admccurdy]

Missing GDAL was the problem for me. I feel a bit sheepish that I didn't figure that out. Thanks for the help @fdetsch.

[fdetsch]

Glad to help. @ndmsc, does it work for you as well? See also #31 that was related to the same error.

[ndmsc]

Yes @fdetsch, GDAL was the problem and now it's working. What I don't understand is why getHdf function is looking for GDAL? I want to download MODIS data only without processing it ...

When I installed GDAL it caused tcltk package to fail. But this is another story.

Thanks for your help

[fdetsch]

Indeed, this makes little sense. I'll have a look into it within the next couple of days.

[MatMatt]

I think getHdf uses gdalinfo to do the integrity check. I'm not sure if switching off the check disables the requirement.

[fdetsch]

@ndmsc Did you try running getHdf(..., checkIntegrity = FALSE)? As pointed out by @MatMatt, this turns off the integrity check, meaning that gdalinfo is not invoked at all.

[ndmsc]

@fdetsch Actually no. I downloaded the files using wget on Linux machine !!!

[fdetsch]

Alright, then I just consider this issue as solved now.