statistical.rst

Statistical Objective Functions

We implemented the Mann-Whitney U-test (MWUT) to calculate the error between experimental data and simulations. The U-test is a non-parametric statistical test that, within a confidence level, determine if a stochastic repeated measurements is identical or not to another repeated measurements.

1. We count how many times experimental data (exp_i) are larger than simulated values (sim_j):

for i in range(len(exp)):
  *for* j in range(len(sim)):
     *if* exp_i > sim_j:
        U_exp ← U_exp + 1.0
     *else if* exp_i < sim_j:
        U_sim ← U_sim + 1.0
     *else*:
        U_exp ← U_exp + 0.5
        U_sim ← U_sim + 0.5

2. We determine if U_exp is statistically significant:

   for i in range(len(exp)):
     *if* len(exp) × len(sim) − min(U_exp, U_sim) ≤ U_critic:
        null hypothesis, H₀, is rejected
        U_model ← U_model + 1.0

Note

The U-test is the only fitness function that has known limits: For a perfect model, the U-test is zero. A complete wrong model will have a U_model equal to the number of Observables times the number of experimental time points. For instance, the example model we use to compare with BioNetFit has 2 Observables and 7 experimental time points, then a max U_model equal to 14.