Update ONNXRuntime to a custom version #5259

hqucms · 2019-10-03T16:30:13Z

Based on the master branch of the official repo @ https://github.com/microsoft/onnxruntime/tree/9f633c5bd9ecccdb0d02e0088d516d67a9dab7cb.

Changes on top of that:

Remove the hard-coded thread pool in LSTM: hqucms/onnxruntime@04f3c76
Control MLAS runtime kernel selection w/ an environment variable MLAS_DYNAMIC_CPU_ARCH: hqucms/onnxruntime@7222aea. We set MLAS_DYNAMIC_CPU_ARCH=0 in the CMS build so it does not attempt to use AVX (or more advanced) instructions.
Propagate the cmake change from @smuzaffar to use pre-installed protobuf.

cmsbuild · 2019-10-03T16:30:42Z

A new Pull Request was created by @hqucms (Huilin Qu) for branch IB/CMSSW_11_0_X/master.

@cmsbuild, @smuzaffar, @mrodozov can you please review it and eventually sign? Thanks.
cms-bot commands are listed here

onnxruntime.spec

onnxruntime-toolfile.spec

cmsbuild · 2019-10-04T08:39:35Z

Pull request #5259 was updated.

smuzaffar · 2019-10-04T09:13:23Z

please test

cmsbuild · 2019-10-04T09:13:48Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2781/console Started: 2019/10/04 11:14

cmsbuild · 2019-10-04T11:33:48Z

+1
Tested at: bef6536
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fc3e51/2781/summary.html

cmsbuild · 2019-10-04T11:33:51Z

Comparison job queued.

cmsbuild · 2019-10-04T13:19:55Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#28112,cms-data/RecoBTag-Combined#25
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2786/console Started: 2019/10/04 15:21

cmsbuild · 2019-10-04T13:39:56Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fc3e51/2781/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2961049
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2960707
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

cmsbuild · 2019-10-04T15:18:43Z

-1

Tested at: bef6536

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd181f/2786/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests RelVals

Unit Tests:

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals:

When I ran the RelVals I found an error in the following workflows:
136.8311 step2

runTheMatrix-results/136.8311_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017/step2_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017.log

136.7611 step2

runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

135.4 step3

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step3_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

136.788 step3

runTheMatrix-results/136.788_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017/step3_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017.log

136.85 step3

runTheMatrix-results/136.85_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

1325.7 step2

runTheMatrix-results/1325.7_TTbar_13_94Xv2NanoAODINPUT+TTbar_13_94Xv2NanoAODINPUT+NANOEDMMC2017_94XMiniAODv2+HARVESTNANOAODMC2017_94XMiniAODv2/step2_TTbar_13_94Xv2NanoAODINPUT+TTbar_13_94Xv2NanoAODINPUT+NANOEDMMC2017_94XMiniAODv2+HARVESTNANOAODMC2017_94XMiniAODv2.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25.log

11634.0 step3

runTheMatrix-results/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU.log

12434.0 step3

runTheMatrix-results/12434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35.log

20434.0 step3

runTheMatrix-results/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44.log

22034.0 step3

runTheMatrix-results/22034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D46_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D46+RecoFullGlobal_2026D46+HARVESTFullGlobal_2026D46/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D46_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D46+RecoFullGlobal_2026D46+HARVESTFullGlobal_2026D46.log

250202.181 step4

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step4_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

cmsbuild · 2019-10-04T15:18:50Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

cmsbuild · 2019-10-08T14:24:40Z

Pull request #5259 was updated.

smuzaffar · 2019-10-08T14:26:24Z

@hqucms , I took your changes and included then in cms-externals/onnxruntime.
This allows us to update automatically when we need to move to new commit or tag. If you need to update it more then please make a PR to cms-externals/onnxruntime against cms/master/9f633c5b branch

hqucms · 2019-10-08T14:28:40Z

@smuzaffar That's great! Thank you very much!

cmsbuild · 2019-10-19T12:59:38Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#28112,cms-data/RecoBTag-Combined#25
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/3037/console Started: 2019/10/19 15:03

cmsbuild · 2019-10-19T16:11:56Z

+1
Tested at: 92e9dfc
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd181f/3037/summary.html

cmsbuild · 2019-10-19T16:12:05Z

Comparison job queued.

cmsbuild · 2019-10-19T18:38:05Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd181f/3037/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 3 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2961064
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2960722
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

slava77 · 2019-10-29T20:52:23Z

onnxruntime.spec

-%define tag 2a8b02097514ca90ac7471054460dcdc47b69e1b
-%define branch cms/v%{realversion}
+%define tag 2824909ae569932d9aee1462049ff0da1e766989
+%define branch cms/master/9f633c5b


should we advance to 0.5.1, which is going to be more recent than the master/9f633c5b?

slava77 · 2019-10-29T20:59:35Z

@smuzaffar
based on the run time tests, I think that this is good to go
It will be easier to have this in the IB when further iterating the review of cms-sw/cmssw#28112

smuzaffar · 2019-10-29T21:09:59Z

+externals

cmsbuild · 2019-10-29T21:10:20Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_11_0_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild added comparison-pending externals-pending orp-pending pending-signatures tests-pending labels Oct 3, 2019

hqucms mentioned this pull request Oct 3, 2019

ONNXRuntime-based implementation of DeepJet, DeepAK8 and DeepDoubleX cms-sw/cmssw#28112

Merged

smuzaffar reviewed Oct 4, 2019

View reviewed changes

onnxruntime.spec Outdated Show resolved Hide resolved

smuzaffar reviewed Oct 4, 2019

View reviewed changes

onnxruntime-toolfile.spec Show resolved Hide resolved

Update ONNXRuntime to a custom version.

bef6536

hqucms force-pushed the onnxruntime/IB/CMSSW_11_0_X/master branch from 6071874 to bef6536 Compare October 4, 2019 08:39

cmsbuild added tests-started and removed tests-pending labels Oct 4, 2019

cmsbuild added tests-approved and removed tests-started labels Oct 4, 2019

cmsbuild mentioned this pull request Oct 4, 2019

Add ONNX models of DeepJet and DeepBoostedJet cms-data/RecoBTag-Combined#25

Merged

cmsbuild added requires-external tests-started and removed tests-approved labels Oct 4, 2019

cmsbuild added comparison-available and removed comparison-pending labels Oct 4, 2019

use onnxruntime from cms-externals

92e9dfc

cmsbuild added comparison-pending tests-pending and removed comparison-available requires-external tests-approved labels Oct 8, 2019

cmsbuild added requires-external tests-started and removed tests-pending labels Oct 19, 2019

cmsbuild added tests-approved and removed tests-started labels Oct 19, 2019

cmsbuild added comparison-available and removed comparison-pending labels Oct 19, 2019

slava77 reviewed Oct 29, 2019

View reviewed changes

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Oct 29, 2019

smuzaffar merged commit 6327483 into cms-sw:IB/CMSSW_11_0_X/master Oct 29, 2019

hqucms mentioned this pull request Nov 12, 2019

Update ONNX Runtime to v1.0.0 #5346

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update ONNXRuntime to a custom version #5259

Update ONNXRuntime to a custom version #5259

hqucms commented Oct 3, 2019

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 4, 2019

smuzaffar commented Oct 4, 2019

cmsbuild commented Oct 4, 2019 •

edited

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019 •

edited

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 8, 2019

smuzaffar commented Oct 8, 2019

hqucms commented Oct 8, 2019

cmsbuild commented Oct 19, 2019 •

edited

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019

slava77 Oct 29, 2019 •

edited

slava77 commented Oct 29, 2019

smuzaffar commented Oct 29, 2019

cmsbuild commented Oct 29, 2019

Update ONNXRuntime to a custom version #5259

Update ONNXRuntime to a custom version #5259

Conversation

hqucms commented Oct 3, 2019

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 4, 2019

smuzaffar commented Oct 4, 2019

cmsbuild commented Oct 4, 2019 • edited

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019 • edited

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 8, 2019

smuzaffar commented Oct 8, 2019

hqucms commented Oct 8, 2019

cmsbuild commented Oct 19, 2019 • edited

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019

slava77 Oct 29, 2019 • edited

Choose a reason for hiding this comment

slava77 commented Oct 29, 2019

smuzaffar commented Oct 29, 2019

cmsbuild commented Oct 29, 2019

cmsbuild commented Oct 4, 2019 •

edited

cmsbuild commented Oct 4, 2019 •

edited

cmsbuild commented Oct 19, 2019 •

edited

slava77 Oct 29, 2019 •

edited