[RF] Handling of the negative values in RooChebychev causes fit divergence

[Bilokin]

• [V ] Checked for duplicates

Describe the bug

The second-order RooChebyshev class produces negative values for certain parameter range around c1=-0.5 and c2=-0.55.
These negative values are then handled differently in different versions of ROOT

Expected behavior

A preferred solution would be to set negative values to 0 as in ROOT 6.20, because setting them to +inf is causing the fits to diverge.
Is it possible to avoid the negative values in RooChebychev in the first place?

To Reproduce

The code which produces the plots above:

import ROOT
canvas = ROOT.TCanvas("cv1", "cv1", 700, 500)
#ROOT.__version__ = '6.20/04'  # Earlier pyROOT implementations have no __version__
mass = ROOT.RooRealVar("KS_M", "m(#pi^{+}#pi^{-}) GeV/c^{2}", 0.47, 0.53)
frame = mass.frame(ROOT.RooFit.Bins(100), ROOT.RooFit.Title(ROOT.__version__))
c1 = ROOT.RooRealVar("c1", " 1st cheb parameter",-0.5, -1, 1)
c2 = ROOT.RooRealVar("c2", " 2nd cheb parameter", -0.55, -1, 1)
bkg = ROOT.RooChebychev("bkg", "Chebyshev Polynomial", mass, ROOT.RooArgList(c1, c2))
bkg.plotOn(frame)
canvas.cd()
frame.Draw()
canvas.Draw()
canvas.Update()
canvas.SaveAs(f'pdf/chebtest_{ROOT.__version__.replace("/","")}.png')
input()

Setup

I used 6.20 and 6.24 ROOT versions on Ubuntu 18.04.

Additional context

[Bilokin]

Hi @lmoneta @guitargeek,
Is there a workaround to set the negative values to 0 instead of +inf?

[ferdymercury]

Is it possible to avoid the negative values in RooChebychev in the first place?

Since the underlying Chebyshev function in that case is:
-1.1 x^2 - 0.5 x + 1.55

and it is being evaluated between -1 and +1, it will return negative values for any x > 0.98 (the analytical root of the equation).

Reproducer:

double coeffs[2] = {-0.5, -0.55};
double x_in = 0.53; double xMin = 0.47; double xMax = 0.53;
const double xPrime = (x_in - 0.5 * (xMax + xMin)) / (0.5 * (xMax - xMin));
cout << 1 + xPrime * coeffs[0] + (2 * xPrime * xPrime - 1)*coeffs[1] << endl;

So to ensure avoiding negative values in the whole range, one would need to set coeffs[0] + coeffs[1] > (-1)

For example, if you change c1 from -0.5 to -0.45, the negative values are gone.

Or alternatively, just create the plot between -1 and 0.98

[ferdymercury]

Closing since there was no reply, please reopen the issue if the explanation / workaround is not satisfactory.

[guitargeek]

Thanks @ferdymercury. And note that while clipping the values to zero seems to be useful for the plot, it is not useful when using it in a fit. If the pdf is evaluated where it is zero, the NLL will be NaN again anyway.

So indeed I agree that it would be a good solution to restrict the range of the plot.

However, this issue has a second part: @Bilokin claimed that there are fits that converged in ROOT 6.20, but not anymore with current ROOT. We can't verify if this is solved my now, because no reproducer of the fit was provided.

Let's give @Bilokin more time to follow up with a reproducer for the regression in fit convergence 🙂 That would be very interesting to see and fix. Could you provide that? Sorry for the late answer and thanks @ferdymercury for taking action here!

[ferdymercury]

@guitargeek Here is maybe a reproducer (I guess) of the fit 'divergence': https://root-forum.cern.ch/t/treatment-of-negative-pdf-in-low-statistics-fits/53248