Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Help with BIOMOD_2-5-2 - Memory issue: Rstudio uses nearly all the memory, and is forced to stop (R4.4.1, RStudio: 2024.04.0, build 735) #485

Closed
zpmdal opened this issue Jul 30, 2024 · 2 comments
Labels
help wanted How to use biomod2 functions and outputs

Comments

@zpmdal
Copy link

zpmdal commented Jul 30, 2024

myBiomodModelOut <- BIOMOD_Modeling(bm.format = myBiomodData,
modeling.id = 'AllModels',
models = c('GAM', 'GLM', 'MAXENT', 'RF', 'MARS'),
CV.strategy = 'random',
CV.nb.rep = 2,
CV.perc = 0.8,
OPT.strategy = 'bigboss',
#OPT.strategy = 'tuned',
var.import = 3,
#metric.eval = c('TSS','ROC', 'BOYCE', 'KAPPA')
metric.eval = c('ROC', 'BOYCE', 'KAPPA')
)

The above works fine. However when I started the following for projection:
myBiomodProj <- BIOMOD_Projection(bm.mod = myBiomodModelOut,
proj.name = 'Current',
new.env = myExpl,
models.chosen = 'all',
metric.binary = 'all',
metric.filter = 'all',
build.clamping.mask = TRUE)

I am using laptop with 32 GB of RAM memory, and 24 GB of swap, 1TB of HDD, installed with Linux (Fedora 40). The memory usage becomes overshoot, it could reach 30G. Then the Rstudio is forced to quit. This happens at the step of building TSS binary/fitered ... .

The strange thing is that the same code run under the same version of R and RStudio on Windows 10 PC with 16 GB of RAM, and 500 GB of HDD run just fine. The memory usage reached around 10GB at most. I am not sure it's caused by Linux OS problem, or because of bugs in the code.

Thanks!

Zhi-Ping

@zpmdal zpmdal added the help wanted How to use biomod2 functions and outputs label Jul 30, 2024
@HeleneBlt
Copy link
Contributor

Hello Zhi-Ping !

Thanks for reporting 🙏
Indeed, this is strange !

You can try to run it with some other arguments :

myBiomodProj <- BIOMOD_Projection(bm.mod = myBiomodModelOut,
proj.name = 'Current',
new.env = myExpl,
models.chosen = 'all',
metric.binary = 'all',
metric.filter = 'all',
build.clamping.mask = TRUE,
on_0_1000 = TRUE, #The probability will be multiplied by 1000 to have integer
do.stack = FALSE ) #You will have several spatRaster 

Could you also try to separate your projections :

  • one without any metrics
  • one with only filtered metrics
  • one with only binary metrics

So it will be easier to see whether the problem comes from biomod2 or more of the memory allowed by Linux to RStudio. 🤔

Thanks !
Hélène

@zpmdal
Copy link
Author

zpmdal commented Jul 31, 2024

Thanks Helene for your quick response,

do.stack = FALSE solved the problem. The number of metrics does not seem to matter.

I am using the Fedora 40 Linux. As there is no problem, without "do.stack = False" on the Windows 10 PC with 16 GB of memory, I guess it's related to the way Linux handles the memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted How to use biomod2 functions and outputs
Projects
None yet
Development

No branches or pull requests

3 participants