Error in hM$rL[[r]]$s[levels(hM$dfPi[, r]), ] : subscript out of bounds

[rhettdharrison]

Hello again,

I am trying to run a model with spatial latent effects following Vignette 3 and the Manual, but getting this error. If I remove the spatial effect then the error disappears. I've checked the xycoords and it is fine - as far as I can see (I plotted it and checked the values). Below the error is the model code in full (and attached data). The xycoords are for plot locations. There are 2 samples per plot (at 1 m and at 18 m height).

m.pa.spatial = sampleMcmc(m.pa.spatial, thin = thin, samples = samples, transient = transient,

•               nChains = nChains, verbose = verbose)
  

Error in hM$rL[[r]]$s[levels(hM$dfPi[, r]), ] : subscript out of bounds

traceback()
2: computeDataParameters(hM)
1: sampleMcmc(m.pa.spatial, thin = thin, samples = samples, transient = transient,
nChains = nChains, verbose = verbose)

xycoords <- matrix(c(plot.char$UTM_E,plot.char$UTM_N),ncol=2)
colnames(xycoords) <- c("UTM_E","UTM_N")
rownames(xycoords) <- plot.id
xycoords <- apply(xycoords,2, function(x) sapply(x, function(m) m - min(x)))

studyDesign = data.frame(plot = plot.id, sample = sample.id)
rL.spatial = HmscRandomLevel(sData = xycoords)
rL.spatial = setPriors(rL.spatial,nfMin=1,nfMax=1)
rL2 = HmscRandomLevel(units = studyDesign$sample)

#set presence-absence model
m.pa.spatial <- Hmsc(Y = sign(lh_sp), XData = plot.char, XFormula = ~ HEIGHT + ELEV + SLOPE + LAI, distr = "probit", studyDesign = studyDesign, ranLevels = list("plot" = rL.spatial))

plot.char.txt
lh_sp.txt

[ovaskain]

I am guessing that the problem might be that you have the same plot.id repeatedly in xycoords. With this... studyDesign = data.frame(plot = plot.id, sample = sample.id) ...the same plot.id can be repeated (multiple samples from the same plot). But with this... rL.spatial = HmscRandomLevel(sData = xycoords) ...each plot.id should be only once in the xycoords.

…

On 17.6.2020 17.58, Rhett Harrison wrote: Hello again, I am trying to run a model with spatial latent effects following Vignette 3 and the Manual, but getting this error. If I remove the spatial effect then the error disappears. I've checked the xycoords and it is fine - as far as I can see (I plotted it and checked the values). Below the error is the model code in full (and attached data). The xycoords are for plot locations. There are 2 samples per plot (at 1 m and at 18 m height). m.pa.spatial = sampleMcmc(m.pa.spatial, thin = thin, samples = samples, transient = transient, * |nChains = nChains, verbose = verbose) | Error in hM$rL[[r]]$s[levels(hM$dfPi[, r]), ] : subscript out of bounds traceback() 2: computeDataParameters(hM) 1: sampleMcmc(m.pa.spatial, thin = thin, samples = samples, transient = transient, nChains = nChains, verbose = verbose) xycoords <- matrix(c(plot.char$UTM_E,plot.char$UTM_N),ncol=2) colnames(xycoords) <- c("UTM_E","UTM_N") rownames(xycoords) <- plot.id xycoords <- apply(xycoords,2, function(x) sapply(x, function(m) m - min(x))) studyDesign = data.frame(plot = plot.id, sample = sample.id) rL.spatial = HmscRandomLevel(sData = xycoords) rL.spatial = setPriors(rL.spatial,nfMin=1,nfMax=1) rL2 = HmscRandomLevel(units = studyDesign$sample) #set presence-absence model m.pa.spatial <- Hmsc(Y = sign(lh_sp), XData = plot.char, XFormula = ~ HEIGHT + ELEV + SLOPE + LAI, distr = "probit", studyDesign = studyDesign, ranLevels = list("plot" = rL.spatial)) plot.char.txt <https://github.com/hmsc-r/HMSC/files/4793330/plot.char.txt> lh_sp.txt <https://github.com/hmsc-r/HMSC/files/4793324/lh_sp.txt> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#47>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEIYMZRPKRVEF46YDKYT44LRXDKYHANCNFSM4OAV3QXQ>.

[rhettdharrison]

That is correct. However, I have tried re-specifying the model and cannot get it to work.

If I correct the xycoords by removing the repeats, I get the same error. If, alternatively, I assign the sample.id to the rownames of xycoords and remove the plot level from the design, I get the following error (I rechecked the data).

Error in chol.default(W) : 
  the leading minor of order 2 is not positive definite
> traceback()
4: chol.default(W)
3: chol(W)
2: computeDataParameters(hM)
1: sampleMcmc(m.ab.spatial, thin = thin, samples = samples, transient = transient, 
       nChains = nChains, verbose = verbose)

xycoords <- matrix(c(plot.char$UTM_E,plot.char$UTM_N),ncol=2)
colnames(xycoords) <- c("UTM_E","UTM_N")
rownames(xycoords) <- sample.id
xycoords <- apply(xycoords,2, function(x) sapply(x, function(m) m - min(x)))

studyDesign = data.frame(sample = sample.id)
rL.spatial = HmscRandomLevel(sData = xycoords)
rL.spatial = setPriors(rL.spatial,nfMin=1,nfMax=1)

#set presence-absence model
m.pa.spatial <- Hmsc(Y = sign(lh_sp), XData = plot.char, XFormula = ~ HEIGHT + ELEV + SLOPE + LAI,
             distr = "probit", studyDesign = studyDesign,
             ranLevels = list("sample" = rL.spatial))

[jarioksa]

I have tried to reproduce this, but failed. In other words: the commands work when I try them and I cannot trigger an error. So this is guessing: the error message you get is the same as earlier with chol (leading minor of order 2 is not positive definite) when you had non-numeric data. In this case the error comes from chol() of spatial weight matrix W which you find from your spatial coordinates. So check these first. In this case, I think you should first see what you get from dist(xycoords) and then how data are read into Hmsc. With your model the spatial coordinates should be in m.pa.spatial$rL["sample"]$s, and the spatial weights are W = dist(m.pa.spatial$rL["sample"]$s).

This is blind guessing though, as we do not have a reproducible example. Can you work out such an example with, e.g., TD data in Hmsc?

[rhettdharrison]

The data in xycoords was fine (plotted and checked the sum), but obviously coordinates of each of the pair of samples taken from the same plot were identical, which produced zeros in the distance matrix. I am not sure why that creates a problem, but adding 5 m to the second x of each pair coodinates (a trivial change when the plots are 100s to kms apart) corrected the error.

So it seems that it is not possible to specify a spatial model where multiple samples are taken from the sample sites, at least not without jittering the points.

[ovaskain]

There have been several questions about nested (spatial etc.) effects in Hmsc. I try to reply to all those concerns here with this more general explanation. First of all, it is possible to have any number of random effects, and they can be nested or non-nested in any way. Each random effects can be either non-structured (defined by units=...) or spatial (defined as sData=..."). Temporal random effect is a special case of spatial for one-dimensional space. Let's say that you have samples within plots. Then for most purposes you should define a random effect at the plot level to account for the study design (replicates samples from the same plot are not independent data points). You can also have the sample level as random effect, if you are interested in co-occurrence at that level. Otherwise it is not needed. If you wish to have plot as a non-structured random effect, then you define it with unit=levels(studyDesign$plot), so that the plot id:s do not repeat for the random effect (the rL object) but they do repeat in the studyDesign. If you wish to have plot as a spatial random effect, then you define it with sData = xy, where the rownames of the xy are the levels of studyDesign$plot. So also here, the plot id:s do not repeat for the random effect but they do repeat in studyDesign. This solve the error messages related to dimensions. The error messages related to the matrices being not positive definite are because of some coordinates being too close to each other, typically identical because of e.g. the plot id:s repeating in the random effect. So they coordinates should not be jittered (that solves the error message but results in a model that you probably don't want to define), but rather following the advice above. More thorough explanations found from the Ovaskainen and Abrego book that recently came out from Cambridge University Press, see e.g. here for more details and example scrtips: https://www.helsinki.fi/en/researchgroups/statistical-ecology/hmsc

…

On 18.6.2020 15.20, Rhett Harrison wrote: The data in xycoords was fine (plotted and checked the sum), but obviously coordinates of each of the pair of samples taken from the same plot were identical, which produced zeros in the distance matrix. I am not sure why that creates a problem, but adding 5 m to the second x of each pair coodinates (a trivial change when the plots are 100s to kms apart) corrected the error. So it seems that it is not possible to specify a spatial model where multiple samples are taken from the sample sites, at least not without jittering the points. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#47 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEIYMZSQ6JV7LMHW52753V3RXIBANANCNFSM4OAV3QXQ>.

[ewmorr]

Thanks for this explanation. This answers my questions in #49. I'll update that issue by way of comment and close.

[rhettdharrison]

Thanks from here too. Now very clear.

[smithja16]

Sorry about posting on a closed thread, but I wanted to post my own experience with this error (using Hmsc v3.0-13). It arose when I was doing my own form of cross-validation (i.e. fitting a model to a data subset).

The solution was to use droplevels() to ensure that the data subset did not reference levels of a random effect (here, 'Vessel') missing in that subset:
studyDesign = data.frame(sample=as.factor(1:nrow(train_data)), vessel=droplevels(train_data$Vessel))