[tree,hist] do not silently hide sqrt of negative value in T(Tree)Formula

[ferdymercury]

return nan instead of sqrt(abs(value))

Fixes #22755

Was like that since the early beginnigns https://github.com/root-project/root/blame/dd588ebc2c01182af573ea9ac7ce2ddcbc734b6e/treeplayer/src/TTreeFormula.cxx

[github-actions]

Test Results

    23 files      23 suites   3d 11h 59m 49s ⏱️
 3 872 tests  3 839 ✅ 0 💤 33 ❌
78 775 runs  78 742 ✅ 0 💤 33 ❌

For more details on these failures, see this check.

Results for commit 85307bd.

[guitargeek]

Thank you very much!

What about adding a unit test like this to tree/treeplayer/test/regression.cxx?

// https://github.com/root-project/root/issues/22755
// sqrt() of a negative argument used to evaluate to sqrt(abs(x)) instead of NaN
TEST(TTreeFormulaRegressions, SqrtOfNegative)
{
   TTree t("t", "t");
   int x = 0;
   t.Branch("x", &x, "x/I");
   t.Fill();
   t.GetEntry(0);

   TTreeFormula tf("tf", "sqrt(-4.0)", &t);
   EXPECT_TRUE(std::isnan(tf.EvalInstance()));
   TTreeFormula tf2("tf2", "sqrt(x - 4)", &t);
   EXPECT_TRUE(std::isnan(tf2.EvalInstance()));
   TTreeFormula tf3("tf3", "sqrt(x + 9)", &t);
   EXPECT_FLOAT_EQ(tf3.EvalInstance(), 3.);
}

Also, this is a behavioral change that needs to be explained in the release notes, with an entry like this for example:

#### Behavior change: `sqrt()` of negative arguments in TTreeFormula now returns NaN

Since its introduction in 1995, the formula engine used by `TTree::Draw()`, `TTree::Scan()` and `TTreeFormula`
silently evaluated `sqrt(x)` as `sqrt(abs(x))` for negative arguments (or as `0` in the optimized evaluation path
of the legacy `ROOT::v5::TFormula`). This could produce silently wrong results, e.g. in selections involving
`sqrt` of an expression that can become negative. `sqrt()` now returns NaN for negative arguments, consistent
with `TMath::Sqrt()`, the standard C `sqrt()`, and the modern `TFormula` used by `TF1`.
Note that in a selection, a NaN evaluates as `false`, so entries where the `sqrt` argument is negative now fail
the cut instead of being selected based on `sqrt(abs(x))`.