Error running VAST example on Ubuntu 18.04

[Jason-Conner-NOAA]

Fresh install on Google Cloud Compute
Ubuntu 18.04
MRAN 4.02
VAST 3.6.1
FishStatsUtils 2.8.0
INLA 21.01.26
TMB 1.7.19

After successful install of all the above packages, running the basic example from the Wiki, I get this error during parameter estimation:

Error in optimHess(parameter_estimates$par, fn = fn, gr = gr) :
gradient in optim evaluated to length 1 not 75
In addition: Warning messages:
1: In sparseMatrix(i = index.i, j = index.j, x = Aij, dims = c(mesh$n, :
'giveCsparse' has been deprecated; setting 'repr = "T"' for you
2: In sparseMatrix(i = index.i, j = index.j, x = Aij, dims = c(mesh$n, :
'giveCsparse' has been deprecated; setting 'repr = "T"' for you
3: In sparseMatrix(i = index.i, j = index.j, x = Aij, dims = c(mesh$n, :
'giveCsparse' has been deprecated; setting 'repr = "T"' for you

Perhaps an issue with MRAN 4.02? I will try installing another R version next week and update this unless I hear from you.

[James-Thorson-NOAA]

could you clarify: is this the development branch of VAST and FishStatsUtils, or the latest numeric release of each?

[Jason-Conner-NOAA]

The latest released versions, installed with:
devtools::install_github("james-thorson/VAST", INSTALL_opts="--no-staged-install")

[Jason-Conner-NOAA]

The option to update all packages was selected on install.

[James-Thorson]

Thanks for reporting the issue. However, I can't replicate the behavior on my Windows machine using: MRAN 4.02 VAST 3.6.1 FishStatsUtils 2.8.0 INLA 21.01.26 TMB 1.7.19 (i.e., all the same stuff on Windows 10). Could you please try: 1. using the CRAN version of TMB for your version of MRAN (which I think is 1.7.16; 1.7.19 appears to be the GitHub version)? 2. Replicating the same settings on a Windows machine to see if its just an Ubuntu issue? 3. Try using the development branch of FishStatsUtils and VAST, where Cole Monnahan has recently helped me set of CI and this has confirmed that VAST can pass some basic build tests on Ubuntu? I'm guessing #3 is a good place to start. Sorry that I can't immediately identify the issue!

…

On Fri, Feb 19, 2021 at 4:17 PM Jason Conner ***@***.***> wrote: The option to update all packages was selected on install. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB46UTNAFTGBTSTNEDODVSLS735QVANCNFSM4X5GTBHQ> .

[Jason-Conner-NOAA]

1. TMB 1.7.16 results in the same error

2. Windows 10, R 4.02 using development branches for VAST and FishStatsUtils results in:
   Error in .spTransform_Polygon(input[[i]], to_args = to_args, from_args = from_args, : failure in Polygons 48 Polygon 2 points 330:331:332:333:334:335:336:337:338:339:340:341:342:343:344:345:346:347:348:349:350:351:352:353:354:355:356:357:358:359:360:361:362:363:364:365:366:367:368:369:370:371:372:373:374:375:376:377:378:379:380:381:382:383:384:385:386:387:388:389:390:391:392:393:394:395:396:397:398:399:400:401:402:403:404:405:406:407:408:409:410:411:412:413:414:415:418:419:420:421:422:423:424:425:426:427:428:429:430:431:432:433:434:435:436:437:438:439:440:441:442:443:444:448:449:450:451:452:453:454:455:456:457:458:459:460:461:462:463:464:465:466:467:468:469:470:471:472:473:474:475:476:477:478:479:480:481:482:483:484:485:486:487:488:489:490:491:492:493:494:495:496:497:498:499:500:501:502:503:504:505:506:507:508:509:510:511 In addition: Warning messages: 1: In spTransform(xSP, CRSobj, ...) : NULL source CRS comment, falling back to PROJ string 2: In wkt(obj) : CRS object has no comment 3: In .spTransform_Polygon(input[[i]], to_args = to_args, from_args = from_args, : 177 projected point(s) not finite

3. Development branches on previous Ubuntu config results in original error.

For reference, VAST example is run by sourcing attached code (minus the .txt).
vast_example.R.txt

[James-Thorson-NOAA]

Let's skip the Ubuntu for now. Can you clarify what function causes the error in Windows?

I have seen that same error during plot_results due to weird behavior of sp during plotting, and haven't had time to track it's cause down previously. If it's during plot_results there's a simple work-around I can share.

[Jason-Conner-NOAA]

Yes, in 2. it is the plot( fit ) function that causes the error. fit_model() worked fine, with a warning that: Note that getReportCovariance=FALSE causes an error in TMB::sdreport when no ADREPORTed variables are present

Here is the session info:
`R version 4.0.2 (2020-06-22)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17134)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] FishStatsUtils_2.9.0 VAST_3.7.0 TMB_1.7.19

loaded via a namespace (and not attached):
[1] tidyselect_1.1.0 remotes_2.2.0 purrr_0.3.4 sf_0.9-7 splines_4.0.2 lattice_0.20-41 rnaturalearth_0.1.0 generics_0.1.0 vctrs_0.3.6
[10] testthat_3.0.2 usethis_2.0.1 utf8_1.1.4 rlang_0.4.10 pkgbuild_1.2.0 e1071_1.7-4 pillar_1.5.0 glue_1.4.2 withr_2.4.1
[19] DBI_1.1.1 TMBhelper_1.3.0 sp_1.4-5 sessioninfo_1.1.1 lifecycle_1.0.0 devtools_2.3.2 memoise_2.0.0 callr_3.5.1 fastmap_1.1.0
[28] ps_1.5.0 curl_4.3 parallel_4.0.2 class_7.3-17 fansi_0.4.2 Rcpp_1.0.6 KernSmooth_2.23-17 INLA_21.02.23 classInt_0.4-3
[37] cachem_1.0.4 desc_1.2.0 pkgload_1.2.0 fs_1.5.0 RANN_2.6.1 dplyr_1.0.2 processx_3.4.5 ThorsonUtilities_1.0 splancs_2.01-40
[46] grid_4.0.2 rprojroot_2.0.2 rgdal_1.5-23 cli_2.3.1 tools_4.0.2 magrittr_2.0.1 tibble_3.0.6 crayon_1.4.1 pkgconfig_2.0.3
[55] ellipsis_0.3.1 Matrix_1.2-18 prettyunits_1.1.1 assertthat_0.2.1 rstudioapi_0.13 R6_2.5.0 rnaturalearthdata_0.1.0 units_0.6-7 compiler_4.0.2 `

[James-Thorson-NOAA]

OK, please try plot( fit, projargs='+proj=natearth +lon_0=X +units=km', country = "united states of america") where X is some central point in your domain. I'll explore sp to figure out what changed there eventually, but its just a plotting issue.

[Jason-Conner-NOAA]

Yes, that solves the issue in 2., for plotting in Win 10.

Do you have access to an Ubuntu server? Do you have a project in GCP?

[James-Thorson-NOAA]

I'm adding Ubuntu CI now with Cole, but no huge insights. You could try the development branch -- its working there?

Also, Andrew Allyn has some other insight he'll share here.

[Jason-Conner-NOAA]

It wasn't working with the dev branch as of yesterday (c635214). Today I will make sure all packages are versioned the same as in Win 10 above, but I suspect there is a system library somewhere that needs to be installed. Also, I will double check permissions on dir for copying cpp files.

[aallyn]

Hi Jason, I had a similar warning message (sparseMatrix) and error (optimHess) arise a little while ago. At the time, I was using a docker image from the rocker project that was built using the "latest" tag. When a new version of R was released (v4.0.4) and used, the warning and error appeared. I can replicate the same error now with the code above on MacOS Catalina v10.15.7 with R v4.0.4 and using development branches for FishStatsUtils and VAST with associated dependencies. I do not get the error if I adjust the package versions for Matrix and TMB explicitly before installing FishStatsUtils and VAST (example below). I got a bit turned around in my sleuthing, but everything seems to come back to the TMB version and associated Matrix version -- where v1.3.2 throws the warning and error (associated with TMB v1.7.19, maybe?) and Matrix v1.2.8 (associated with TMB v1.7.18) does not. Not sure exactly what is happening, but might be worth trying the explicit install of earlier Matrix and TMB versions and see if that works?

# Different install processes to produce the error or not. Restart R completely before trying one or the other!!!
produce_error<- TRUE
if(produce_error){
  install.packages("devtools")
  library(devtools)
  install.packages("Matrix")
  library(Matrix)
  devtools::install_github("James-Thorson-NOAA/FishStatsUtils", ref = "development", force = TRUE)
  devtools::install_github("James-Thorson-NOAA/VAST", ref = "development", force = TRUE)
  library(VAST)
  library(FishStatsUtils)
} else {
  install.packages("devtools")
  library(devtools)
  install_version("Matrix", version = "1.2.8")
  library(Matrix)
  install_version("TMB", "1.7.18")
  # Now VAST and FishStatsUtils, do NOT update TMB
  devtools::install_github("James-Thorson-NOAA/FishStatsUtils", ref = "development", force = TRUE)
  devtools::install_github("James-Thorson-NOAA/VAST", ref = "development", force = TRUE)
  library(VAST)
  library(FishStatsUtils)
}

[Jason-Conner-NOAA]

Thank you Andrew, the earlier version of Matrix installed fine, but I cannot install TMB 1.7.18 from either the MRAN or vanilla CRAN repos.
Error in download_version_url(package, version, repos, type) : version '1.7.18' is invalid for package 'TMB'

[Jason-Conner-NOAA]

The issue is clearly compatibility with Matrix versions, thank you Andrew!

The example runs with this config:
Ubuntu 18.04
MRAN 4.02
FishStatsUtils_2.9.0
VAST_3.7.0 (VAST_v12_0_0.cpp)
TMB_1.7.19
INLA_21.01.26
Matrix_1.2-8
TMBhelper_1.3.0

Thanks again for all of your help!