Is padding in empty bins expected?

[zjkjsd]

Aloha experts,

In one of my practices, I fit normalizations of 6 samples, i.e. the only modifier of each sample is a normfactor. Each sample has ~2000 bins but a significant fraction of bins are empty (see template pic below). When I run fitting:

pyhf.infer.mle.fit(data, model, init_pars=init_parameters, fixed_params=fix_parameters, 
                   return_uncertainties=True, return_correlations=True)

Either error of Optimization failed. Estimated distance to minimum too large. or Inequality constraints incompatible always shows up. Reading through the existing discussions, I found that the problem could be avoided if empty bins in the model (template json file) are set to a small number rather than empty (e.g. assigned to 0.0001 rather than 0). (it turns out that the empty bins in data won't cause problems)

However, as the empty bins are supposed to have 0 counts and there are so many of them, I am worried that padding these bins would affect the fitting performance. Is this behavior expected or will be changed in the future?

Thanks!
Boyang

[alexander-held]

Hi @zjkjsd, I assume that you have some bins where your model predicts exactly zero events but there actually are some events in data? The underlying problem then would be that you do not have enough simulated events to populate this large amount of bins. While setting the bin yield to a small number technically solves the problem, it is not a good solution: you are avoiding the case where you observe data in a bin where your model can't predict any events (which is what presumably causes the failures), but you do not actually know how many events you really expect in such a bin. The only real solution would be to merge bins together or to simulate more samples such that you have a reliable estimate for all your templates in all your bins.

[zjkjsd]

Thank you @alexander-held !
This is very helpful, I will try to merge those empty bins. One more question: Actually, most of the empty bins are due to kinematic limitations and can never populate events, should I remove those bins? I was worried that manually removing empty bins causes biases.

[alexander-held]

I am not sure what kind of bias you have in mind. If the bin is kinematically inaccessible anyway then I see no reason to keep it.

[zjkjsd]

I wasn't sure how to calculate the exact shape of the kinematic edges, so I picked empty bins from a very large MC to define the edges. I feel this is not so rigorous but I can't have a better idea.

[alexander-held]

This now becomes very much a question about the physics you are trying to measure here, and I can't give much general advice. Depending on what you want to measure here you could use different observables to bin in to more easily define edges of accessible parameter space. Binning can also affect performance, so you may want to pick a binning to optimize that. There's no general way to go about finding a suitable binning, I would suggest trying out different approaches to learn how they compare.

[zjkjsd]

Thank you very much! very helpful!