Breaking a statistical model into pieces

[jackaraz]

Hi all, I hope you are all well. I have a naive question; let's say I have a JSON file describing my statistical model, and I wanna plug this in pyhf and compute likelihoods etc. Would It be kosher to use only a subset of bins defined in a statistical model? If the answer is not strictly no, what should I look out for to ensure the validity of the new construction?

Thanks!

[alexander-held]

Hi @jackaraz, this is conceptually perfectly fine. See also #956 for a related request. Assuming that the model you are starting with has been thoroughly validated, the biggest thing to look out for might be whether the loss of information from the bins that are dropped could compromise the fit stability. This could happen for example due to parameters becoming highly correlated. In addition, your model might start behaving in undesirable ways when e.g. control regions are dropped that help constrain parameters to physically sensible values, and the related degrees of freedom are instead used to correct unrelated mis-modeling problems. That part is not really specific to using a subset of bins, the same considerations apply for any model.

[jackaraz]

Hi @alexander-held, thanks for the prompt reply! I believe the stability will be compromised within the default boundaries, correct? So, in theory, I should be able to compensate for this by changing the boundaries, or am I missing something?

[alexander-held]

The problem can be more general than just the bounds. Let's take the example of a single signal region bin that has contributions from process A, process B and signal. Both process A and B are floating freely in this imaginary example, controlled by different normfactor modifiers, and you want to measure signal. You will need at least two more bins to disentangle the normalizations of A and B from signal to not have a degenerate setup. Removing too many bins may cause these situations where individual parameters become highly correlated (or completely degenerate).

[jackaraz]

Ah, thanks for the clarification; that really helped!