Fix predict difference

[ShvetsKS]

related issue: #6350

[trivialfis]

Could you please provide a brief explanation for what happened? Also a unittest. Lastly I see that you are adding an extra allocation, does it impact perf?

[trivialfis]

Ah, I saw your reply on the issue. Will continue there.

[ShvetsKS]

Could you please provide a brief explanation for what happened? Also a unittest. Lastly I see that you are adding an extra allocation, does it impact perf?

The reason of difference in output result is changed sequence of floating point operations:
Before a4ce0ea optimizations we accumulated trees responses for each sample in local variable psum with 0 initial value. And than increment out_pred[i].
But in a4ce0ea optimizations we increment out_pred[i] directly (initial value 0.5 usually), so the fp error was a little bit different.

There is already implemented cpp tests for cpu predictor (cpu_predictor->PredictBatch), there is no significant changes. Or you mentioned about ReduceResult?

I checked it on several benchmarks, seems there is no performance affection.

[ShvetsKS]

https://xgboost-ci.net/blue/organizations/jenkins/xgboost-win64/detail/PR-6384/3/pipeline#step-89-log-1987
Not sure that it's related to current changes as locally all tests from tests/python/test_with_sklearn.py are passed.

@trivialfis should we restart Jenkins Win64: Test ?

[trivialfis]

@ShvetsKS Let's pause on this with comment: #6350 (comment) . We @RAMitchell @hcho3 agreed that we should not establish the tradition of fixing floating change across versions, as you stated, that will severely limit our development. We haven't decided how to document or whether do we need to document this.

[trivialfis]

But to reply your question on the failing CI. Yeah, it happens all the time that error is only reproducible on CI ... Something I have been bumping into quite often. The error in this PR is new to me, so it might be specific to this PR instead of CI glitches. You can see on our issues list for all flaky tests we have found.

[ShvetsKS]

@trivialfis seems I should close this PR as #6350 was resolved? :)

[hcho3]

Let us continue our discussion in #6350.