RecoTracker/MkfitCore: Add fast-math and function attribute need to prevent segfault

[gartung]

Add fast-math flag and attribute needed to prevent segfault when fast-math is used

This PR adds back the options which enable the vectorized function from libmvec on EL8. This causes a differences in RECO DQM.

Followup PR to #37868

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38147/30290

• This PR adds an extra 20KB to repository

[cmsbuild]

A new Pull Request was created by @gartung (Patrick Gartung) for master.

It involves the following packages:

• RecoTracker/MkFitCore (reconstruction)

@jpata, @cmsbuild, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @ebrondol, @gpetruc, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[gartung]

enable profiling

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38147/30296

• This PR adds an extra 28KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38147/30296/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38147/30296/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38147/30297

• This PR adds an extra 28KB to repository

[cmsbuild]

Pull request #38147 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-77cf4a/25125/summary.html
COMMIT: 19b46ed
CMSSW: CMSSW_12_5_X_2022-05-31-1100/el8_amd64_gcc10
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38147/25125/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 10965 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3648315
• DQMHistoTests: Total failures: 14532
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3633760
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.004 KiB( 49 files compared)
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-77cf4a/25291/summary.html
COMMIT: d3576b6
CMSSW: CMSSW_12_5_X_2022-06-05-0000/el8_amd64_gcc10
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38147/25291/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 10969 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3651603
• DQMHistoTests: Total failures: 14536
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3637044
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.004 KiB( 49 files compared)
• DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[clacaputo]

type tracking

[clacaputo]

assign tracking-pog

The reco differences introduced by this PR seems minimal to me, but I'd like to have a crosscheck by tracking-pog

[cmsbuild]

New categories assigned: tracking-pog

@mmusich,@vmariani,@mtosi you have been requested to review this Pull request/Issue and eventually sign? Thanks

[slava77]

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-77cf4a/25291/summary.html

---

Additional Tests: PROFILING

Looking at the profile, the mkFit related calls got down from 15.26 ticks to 13.81. So, this is about 10% reduction.

[gartung]

The PR link is https://cmssdt.cern.ch/SDT/cgi-bin/igprof-navigator/CMSSW_12_5_X_2022-06-05-0000/el8_amd64_gcc10/profiling/11834.21/PR-77cf4a/25291/igprofCPU_step3.19/57

[slava77]

The PR link is https://cmssdt.cern.ch/SDT/cgi-bin/igprof-navigator/CMSSW_12_5_X_2022-06-05-0000/el8_amd64_gcc10/profiling/11834.21/PR-77cf4a/25291/igprofCPU_step3.19/57

thanks for reporting, I fixed it now in the original message

[clacaputo]

Hi @gartung @slava77 , do we want to backport it for 12_4_X?

[slava77]

Hi @gartung @slava77 , do we want to backport it for 12_4_X?

this PR is not purely technical and differences in physics are visually larger than what you'd usually see with small shuffle at the numerical precision level.

e.g. the number of tracks in detached triplet and pixelLess iterations went down a bit, as can be seen in DQM bin-to-bin for wf 136.874 https://tinyurl.com/2d3gpg9c

this is consistent with an observation from a ttbar MC sample that the fake rate in these iterations goes down @leonardogiannini please share a link

So, it may make more sense to have this formally validated first and then backported.

[leonardogiannini]

this is the validation on 10 muon sample (pT 0.2 to 1000)
http://uaf-10.t2.ucsd.edu/~legianni/Check-PR38147-fastmath-10mu-112/

ttbar with PU sample
http://uaf-10.t2.ucsd.edu/~legianni/Check-PR38147-fastmath-112/

red only adds the PR 38147 to the code wrt to blue.

[clacaputo]

+reconstruction

• add fast-math flag to MkfitCore
• 10% time reduction RecoTracker/MkfitCore: Add fast-math and function attribute need to prevent segfault #38147 (comment)
• PR not purely technical, some differences are visible (RecoTracker/MkfitCore: Add fast-math and function attribute need to prevent segfault #38147 (comment)), consistent with a bigger ttbar validation RecoTracker/MkfitCore: Add fast-math and function attribute need to prevent segfault #38147 (comment)

[mmusich]

+1

• based on RecoTracker/MkfitCore: Add fast-math and function attribute need to prevent segfault #38147 (comment)

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1

• As discussed at tody's ORP meeting, it needs to be better validated before thinking about possible backports