qmutilde in pyhf

[WRenouf]

Hi all,

I'm just getting my feet using the pyhf software and ran into a couple of errors.

I took a simple hypothesis test model from the pyhf examples and attempted to run it for integer test values from 0 to 400 for 9 different signal files (just looped over below 9 times with a different spec input to the workspace).

Trying this gave the error Use the qmu_tilde test statistic (pyhf.infer.test_statistics.qmu_tilde) instead. qmu test statistic used for fit configuration with POI bounded at zero. even though I had specified that I wanted qmutilde to be the test statistic and would give it for each value of the poisson statistic for each signal (9 * 400 = 3600 times). The printed messages meant the code would run very slowly.

workspace = pyhf.Workspace(spec)
model = workspace.model(measurement_name="Measurement")
data = pyhf.tensorlib.astensor(background_data + model.config.auxdata)
poisson_vals = np.linspace(0, 400, 400)
init_parameters = model.config.suggested_init()
parameter_bounds = model.config.suggested_bounds()
fixed_parameters = model.config.suggested_fixed()
results = [
    pyhf.infer.hypotest(
        test_poi,
        data,
        model,
        test_stat=pyhf.infer.test_statistics.qmu_tilde(
            test_poi, data, model, init_parameters, parameter_bounds, fixed_parameters
        ),
        return_expected_set=True,
    )
    for test_poi in poisson_vals
]

I then updated from 0.5.4 to 0.6.0 to see if that may help and it could no longer deal with the previous code; I then changed the results array to the simpler one seen below and re-ran it.

results = [
    pyhf.infer.hypotest(
        test_poi, data, model, test_stat="qmu_tilde", return_expected_set=True
    )
    for test_poi in poisson_vals
]

Which gave the error pyhf.exceptions.InvalidTestStatistic. I suspect that the first way of doing it has been changed in the update but I cannot figure out how to get the qmu_tilde statistic to run now and was wondering if anyone had a solution.

Thanks in advance.

[alexander-held]

Hi, the test_stat is called "qtilde" instead, see the AsymptoticCalculator documentation for details (linked from the hypotest documentation but admittedly easy to overlook).

There is also a related issue about allowing additional names: #1246.

[matthewfeickert]

@WRenouf Please apply @alexander-held's correction

results = [
    pyhf.infer.hypotest(
        test_poi,
        data,
        model,
        init_par=init_parameters,
        par_bounds=parameter_bounds,
        fixed_params=fixed_parameters,
        test_stat="qtilde",
        return_expected_set=True,
    )
    for test_poi in poisson_vals
]

which will definitely resolve your first problem. If you are having issues after this please either link your full workspace (in a GitHub Gist) or attach it as a file here.

[matthewfeickert]

For posterity, I'll also note that the v0.5.4 pyhf.infer.calculators.AsymptoticCalculator API doesn't support a test_stat kwarg but a qtilde Bool instead. So the v0.5.4 API call would have been something like

results = [
    pyhf.infer.hypotest(
        test_poi,
        data,
        model,
        init_par=init_parameters,
        par_bounds=parameter_bounds,
        fixed_params=fixed_parameters,
        qtilde=True,
        return_expected_set=True,
    )
    for test_poi in poisson_vals
]

v0.6.0 was an API breaking change.

[WRenouf]

Thanks all, changing it to qtilde seems to have fixed the error. Silly error on my part.

[matthewfeickert]

No worries! I promise you that you won't be the last one to have this happen to them. Also thanks for being the inaugural GitHub Discussions Q&A poster! 😃 🚀

[WRenouf]

I feel honoured 😃