Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[10_6_X] Use ONNXRuntime for ParticleNet and update the training for UL #31037

Merged
merged 4 commits into from Aug 17, 2020

Conversation

hqucms
Copy link
Contributor

@hqucms hqucms commented Aug 3, 2020

PR description:

Backport of #30599 and #31036.

This PR updates the ParticleNet tagger to the new training [V01] developed for the UL re-MiniAOD. The training is derived on UL17+UL18 samples and using Puppi tune V14. The new training improves the performance for UL samples and the new Puppi tune. More information can be found in the JME talks [1, 2] and the BTV talk.

Requires:

PR validation:

The CMSSW implementation is compared to the training framework and consistent results are obtained.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2020

A new Pull Request was created by @hqucms (Huilin Qu) for CMSSW_10_6_X.

It involves the following packages:

PhysicsTools/ONNXRuntime
PhysicsTools/PatAlgos
RecoBTag/Configuration
RecoBTag/FeatureTools
RecoBTag/MXNet
RecoBTag/ONNXRuntime

@perrotta, @jpata, @cmsbuild, @santocch, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @hatakeyamak, @emilbols, @peruzzim, @seemasharmafnal, @mmarionncern, @ahinzmann, @smoortga, @jdolen, @ferencek, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @andrzejnovak, @clelange, @riga, @JyothsnaKomaragiri, @mverzett, @gpetruc, @mariadalfonso this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor

slava77 commented Aug 6, 2020

Requires:

* [cms-data/RecoBTag-Combined#34](https://github.com/cms-data/RecoBTag-Combined/pull/34)

* [cms-sw/cmsdist#6139](https://github.com/cms-sw/cmsdist/pull/6139)

apparently it was supposed to be cms-sw/cmsdist#6145 , which is already merged

@slava77
Copy link
Contributor

slava77 commented Aug 6, 2020

test parameters:

@slava77
Copy link
Contributor

slava77 commented Aug 6, 2020

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 6, 2020

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#6159

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2020

+1
Tested at: dc54bdf
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17ea8e/8643/summary.html
CMSSW: CMSSW_10_6_X_2020-08-06-2300
SCRAM_ARCH: slc7_amd64_gcc700

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2020

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2020

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17ea8e/8643/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 3214618
  • DQMHistoTests: Total failures: 2
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3214282
  • DQMHistoTests: Total skipped: 334
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 140 log files, 14 edm output root files, 34 DQM output files

@slava77
Copy link
Contributor

slava77 commented Aug 7, 2020

+1

for #31037 dc54bdf

@slava77
Copy link
Contributor

slava77 commented Aug 10, 2020

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 10, 2020

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#6159

@cmsbuild
Copy link
Contributor

+1
Tested at: dc54bdf
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17ea8e/8690/summary.html
CMSSW: CMSSW_10_6_X_2020-08-10-1100
SCRAM_ARCH: slc7_amd64_gcc700

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17ea8e/8690/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 74 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 3214618
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3214283
  • DQMHistoTests: Total skipped: 334
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 140 log files, 14 edm output root files, 34 DQM output files

@slava77
Copy link
Contributor

slava77 commented Aug 10, 2020

Reco comparison results: 74 differences found in the comparisons

as expected after #31072 , the UL remini backport now shows differences in the UL remini workflow 136.88811 as expected; the differences are in slimmedJetsAK8 jet tag discriminants.

@slava77
Copy link
Contributor

slava77 commented Aug 10, 2020

backport of #30599
backport of #31036

@slava77
Copy link
Contributor

slava77 commented Aug 17, 2020

@silviodonato @qliphy
this is missing just the analysis signature.
May I suggest to bypass it here.
Thank you.

@qliphy qliphy merged commit a43e2ab into cms-sw:CMSSW_10_6_X Aug 17, 2020
@qliphy
Copy link
Contributor

qliphy commented Aug 17, 2020

+1

@santocch
Copy link

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants