ValueError: The truth value of an array ... when using fixed_params kwarg for pyhf.infer.hypotest

[andrewfowlie]

Summary

I can't get the fixed_params kwarg of pyhf.infer.hypotest to work. Here is a an example from the docs without it that works for me

import pyhf

pyhf.set_backend("numpy")

model = pyhf.simplemodels.uncorrelated_background(
    signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
)
observations = [51, 48]
data = pyhf.tensorlib.astensor(observations + model.config.auxdata)
mu_test = 1.0

pyhf.infer.hypotest(
    mu_test, data, model, return_expected_set=True, test_stat="qtilde"
)

Now, let me try to fix the parameters.

ndim = len(model.config.par_names)
fixed_params = pyhf.tensorlib.astensor([True] * ndim)

pyhf.infer.hypotest(
    mu_test, data, model, fixed_params=fixed_params, return_expected_set=True, test_stat="qtilde"
)

I find

Traceback (most recent call last):
  File "/home/andrew/bug.py", line 20, in <module>
    pyhf.infer.hypotest(
  File "/home/andrew/bug/bug/lib/python3.11/site-packages/pyhf/infer/__init__.py", line 155, in hypotest
    fixed_params = fixed_params or pdf.config.suggested_fixed()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

If this is a bug, it you can probably fix this by doing

fixed_params = fixed_params if fixed_params is not None else pdf.config.suggested_fixed()

and it likely affects init_pars and par_bounds too. Though maybe I am passing the wrong kind of object to the method.

OS / Environment

PRETTY_NAME="Ubuntu 23.04"
NAME="Ubuntu"
VERSION_ID="23.04"
VERSION="23.04 (Lunar Lobster)"
VERSION_CODENAME=lunar
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=lunar
LOGO=ubuntu-logo

Steps to Reproduce

import pyhf

pyhf.set_backend("numpy")

model = pyhf.simplemodels.uncorrelated_background(
    signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
)
observations = [51, 48]
data = pyhf.tensorlib.astensor(observations + model.config.auxdata)
mu_test = 1.0

pyhf.infer.hypotest(
    mu_test, data, model, return_expected_set=True, test_stat="qtilde"
)

ndim = len(model.config.par_names)
fixed_params = pyhf.tensorlib.astensor([True] * ndim)

pyhf.infer.hypotest(
    mu_test, data, model, fixed_params=fixed_params, return_expected_set=True, test_stat="qtilde"
)

File Upload (optional)

No response

Expected Results

I expected to compute the CLs with some fixed parameters.

Actual Results

Traceback (most recent call last):
  File "/home/andrew/bug.py", line 20, in <module>
    pyhf.infer.hypotest(
  File "/home/andrew/bug/bug/lib/python3.11/site-packages/pyhf/infer/__init__.py", line 155, in hypotest
    fixed_params = fixed_params or pdf.config.suggested_fixed()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

pyhf Version

pyhf, version 0.7.5

Code of Conduct

• I agree to follow the Code of Conduct

[matthewfeickert]

Hi @andrewfowlie. This is a bug of documentation and another reason we need to get types in at some point. The v0.7.5 pyhf.infer.hypotest docs give fixed_params as

pyhf/src/pyhf/infer/__init__.py

Lines 64 to 65 in 5154cda

	fixed_params (:obj:`tensor` of :obj:`bool`): The flag to set a parameter constant to its starting
	value during minimization.

but it should be a tuple or list of bool.

Here's your example, but working:

import pyhf

pyhf.set_backend("numpy")

model = pyhf.simplemodels.uncorrelated_background(
    signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
)
observations = [51, 48]
data = pyhf.tensorlib.astensor(observations + model.config.auxdata)
mu_test = 1.0

fixed_params = model.config.suggested_fixed()
fixed_params[1:] = True, True  # The POI can't be fixed and still perform inference
print(f"Fixed parameters: {fixed_params}")

cls_obs, cls_exp_band = pyhf.infer.hypotest(
    mu_test,
    data,
    model,
    fixed_params=fixed_params,
    return_expected_set=True,
    test_stat="qtilde",
)

print(f"Observed CLs: {cls_obs}")
print(f"Expected CLs band: {cls_exp_band}")

# tuple is valid too
fixed_params = tuple(fixed_params)
pyhf.infer.hypotest(
    mu_test,
    data,
    model,
    fixed_params=fixed_params,
    return_expected_set=True,
    test_stat="qtilde",
)

Fixed parameters: [False, True, True]
Observed CLs: 0.02127891847130824
Expected CLs band: [array(0.00080112), array(0.00552974), array(0.03326288), array(0.15388567), array(0.45916615)]

[andrewfowlie]

Thanks Matthew, using a list or tuple works for me. Yes it's a small bug in the docs.

[matthewfeickert]

@andrewfowlie This is now corrected in the pyhf v0.7.6 docs.

[andrewfowlie]

Thanks Matthew. I think that this affects the init_pars and par_bounds docs too.