question on interpretation of results

[yangwei1993]

Hi，
I note there is CVcorrelation and CVcorrelation p values in the results. I have read the meaning of CVcorrelation p in the documentation. However, the meaning of CVcorrelation p is not well explained. Dose it mean the correlation between the VLSM map and behavior? And a larger value refer to a strong relationship?

[dorianps]

It goes basically like this:
The model finds a set of voxel weights that together maximize the relationship with behavior. This is done only on one part of the data, by default 75%. Then it applies the weights on the remaining 25% of the subjects and computes a predicted behavioral score (reference code is here). This is done in turn until all 100% of the subjects have a prediction. Then it computes the Pearson correlation between real vs. predicted data on all subjects. The whole process is repeated a few times to make sure the outcome is not random. The average of all repetitions is CVcorrelation.stat (code here).

CVcorrelation.pval is simply the corresponding p-value of that Pearson correlation for a sample with N subjects (depending on your sample) (reference code here).

So, the answer to your questions is:
CVcorrelation.pval is the probability that the correlation between true and predicted behavioral scores obtained from out of sample cases is null, given the set of voxel weights you see on the map. The p-value is typically lower for stronger relationships (since it is less likely that the result is by chance), but p-values are not linear measures of relationship strength, and are not used for that purpose. The only way to use p-values is to decide when something is significant or not. If you want to comment on the strength of relationship, you can use CVcorrelation itself.

Note that LESYMAP conducts an arbitrary thresholding to eliminate voxel weights <10% of the maximal weight value, and then normalizes the outcome in 0-1 range (code here). This seemed to work well during simulations. However, CVcorrelation is computed on the real weights which are in rawWeights.img; those weights are much smaller value and there are many more voxels with very small weights.

Hope this clears a bit the confusion.

[yangwei1993]

font{ line-height: 1.6; } ul,ol{ padding-left: 20px; list-style-position: inside; } ***@***.***

…

---- Replied Message ---- From ***@***.***> Date 7/19/2022 12:39 To ***@***.***> Cc ***@***.***> , ***@***.***> Subject Re: [dorianps/LESYMAP] question on interpretation of results (Issue #33) It goes basically like this: The model finds a set of voxel weights that together maximize the relationship with behavior. This is done only on one part of the data, by default 75%. Then it applies the weights on the remaining 25% of the subjects and computed a predicted behavioral score (reference code is here). This is done in turn until all 100% of the subjects have a prediction. Then correlation is computed between real vs. predicted data on all subjects. This is repeated a few times to make sure the outcome is not random. The average of all repetitions is CVcorrelation.stat (code here). CVcorrelation.pval is simply the corresponding p-value of that Pearson correlation above for a sample with N subjects (depending on your sample) (reference code here). So, the answer to your questions is: CVcorrelation.pval is the probability that the correlation between true and predicted behavioral scores obtained from out of sample cases is null, given the set of voxel weights you see on the map. The p-value is typically lower for stronger relationships (since it is less likely that the result is by chance), but p-values are not linear measures of relationship strength, and are not used for that purpose. The only way to use p-values is to decide when something is significant or not. If you want to comment on the strength of relationship, you can use CVcorrelation itself. Note that LESYMAP conducts an arbitrary thresholding to eliminate voxel weights <10% of the maximal weight value, and then normalizes the outcome in 0-1 range (code here). This seemed to work well during simulations. However, CVcorrelation is computed on the real weights which are in rawWeights.img; those weights are much smaller value and there are many more voxels with very small weights. Hope this clears a bit the confusion. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>