Reference Dataset Subsampling Factor Parameters

This option defines a factor that determines the size of the Reference Dataset, in terms of the number of subjects, used in each permutation test.

For example:

• Consider that your final Reference Dataset contains 100 control subjects.
• Consider that you have defined 1,000 permutation tests.
• Consider that you have defined a Ref. Dataset subsampling factor of 1 (100%).

With a subsampling factor of 1, the entire Reference Dataset would be used in every test. Under a leave-one-out approach, only 100 unique subsets can be generated, each corresponding to the exclusion of one of the 100 subjects. Therefore, it would not be possible to perform 1000 unique tests using this configuration.

However, if you define a Ref. Dataset subsampling factor of 0.95 (95%), each test will use a unique subset of 95 subjects randomly selected from the original 100. The number of possible unique subsets in this case is:

C(100, 95) = 75287520

S²M ensures that the same subset of subjects is not used more than once. Therefore, with a subsampling factor of 0.95, there are more than enough possible unique subsets to perform 1000 permutation tests.

Obviously, as with most methodological choices in science, this approach involves trade-offs.

Reducing the Ref. Dataset subsampling factor increases the number of unique subsets that can be generated. In practice, this number increases very rapidly as the subset size decreases, making the number of possible combinations less of a limiting factor. However, smaller subsets also introduce greater variability between the Reference Datasets used in individual tests, which can be beneficial for characterizing the variability of the null distribution and strengthening the statistical inference.

Conversely, the smaller the subset, the less closely each individual test represents the full Reference Dataset. As a result, the empirical null distribution and the resulting statistical thresholds may become less representative of the actual comparison of your subject against the complete Reference Dataset of 100 controls. Excessive subsampling may therefore reduce the rigor of the inference and potentially increase the influence of random variability, noise, or false-positive findings.

Therefore, the Ref. Dataset subsampling factor should be chosen carefully, balancing the desired number of unique tests against the representativeness and stability of the Reference Dataset used in each test.