Updated json for DeepCSV Phase 2 model #26072

dseith · 2019-03-05T15:47:01Z

A pull request with the new json for DeepCSV Phase 2 has been made to: https://github.com/cms-btv-pog/RecoBTag-Combined

Accordingly, the name of the json in this file is updated to match the now available Phase 2 json.

added correct json for PhaseII to pfDeepCSVJetTags_cfi.py

Update pfDeepCSVJetTags_cfi.py

cmsbuild · 2019-03-05T15:47:28Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-03-05T15:53:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-26072/8630

This PR adds an extra 12KB to repository

cmsbuild · 2019-03-05T15:53:51Z

A new Pull Request was created by @dseith (David Seith) for master.

It involves the following packages:

RecoBTag/Combined

@cmsbuild, @perrotta, @slava77 can you please review it and eventually sign? Thanks.
@imarches, @smoortga, @acaudron, @HeinerTholen, @JyothsnaKomaragiri, @mverzett, @ferencek, @pvmulder this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

slava77 · 2019-03-05T15:59:48Z

A pull request with the new json for DeepCSV Phase 2 has been made to: https://github.com/cms-btv-pog/RecoBTag-Combined

we need a data PR in the cms-data/RecoBTag-Combined repository

dseith · 2019-03-05T16:03:59Z

The PR is already there: cms-btv-pog/RecoBTag-Combined#3

dseith · 2019-03-05T16:05:04Z

Ah my bad
Its in the cms-btv-pog repo
I will make a pull request to the cms-data repo

dseith · 2019-03-05T16:16:24Z

I've now made the PR to cms-data/RecoBTag-Combined: cms-data/RecoBTag-Combined#20

smuzaffar · 2019-03-05T22:23:55Z

please test with cms-sw/cmsdist#4752

cmsbuild · 2019-03-05T22:24:22Z

The tests are being triggered in jenkins.
Using externals from cms-sw/cmsdist#4752
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/33425/console

cmsbuild · 2019-03-06T02:56:43Z

+1
Tested at: 3be52ce
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26072/33425/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
3be52ce
f19c944
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26072/33425/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26072/33425/git-merge-result

cmsbuild · 2019-03-06T02:56:52Z

Comparison job queued.

cmsbuild · 2019-03-06T04:15:21Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26072/33425/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 82 differences found in the comparisons
DQMHistoTests: Total files compared: 32
DQMHistoTests: Total histograms compared: 3114826
DQMHistoTests: Total failures: 102
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3114527
DQMHistoTests: Total skipped: 197
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
Checked 133 log files, 14 edm output root files, 32 DQM output files

cmsbuild · 2019-05-09T15:45:54Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-data/RecoBTag-Combined#23
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/128/console Started: 2019/05/09 17:48

cmsbuild · 2019-05-10T06:50:41Z

-1

Tested at: 3be52ce

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
3be52ce
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26072/128/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26072/128/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cf893c/128/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

Unit Tests:

I found errors in the following unit tests:

---> test testConditionDatabase_2 had ERRORS

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
3be52ce
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26072/128/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26072/128/git-merge-result

cmsbuild · 2019-05-10T06:50:47Z

Comparison job queued.

cmsbuild · 2019-05-10T11:06:16Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cf893c/128/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 108 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3211964
DQMHistoTests: Total failures: 129
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3211631
DQMHistoTests: Total skipped: 204
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
Checked 137 log files, 14 edm output root files, 33 DQM output files

slava77 · 2019-05-10T12:28:25Z

@cmsbuild please test with cms-data/RecoBTag-Combined#23

@smuzaffar I see in the compile log
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cf893c/128/build.log
that essentially a full build is done.
It is rather wasteful at least in this context.
Perhaps we can add some control words to ask for a light build.

slava77 · 2019-05-11T16:02:42Z

Compared to the previous #26072 (comment), it seems like we have some improvements.

Here are comparisons relative to the previous version of the training:

Both are showing some slight improvement.

Compared to the current baseline,
we are even better in the BvsAll b.v.l ROC

and just about the same (not slightly worse as in the previous version) in the CvsL c.v.l ROC

slava77 · 2019-05-11T16:12:03Z

+1

for #26072 3be52ce combined with cms-data/RecoBTag-Combined#23 966f953

code changes are in line with the PR description and the follow up review
jenkins tests pass and comparisons with the baseline show differences only phase-2 workflows in Btag plots with deepCSV discriminants
local tests with 1K events without PU show some improvement in performance Updated json for DeepCSV Phase 2 model #26072 (comment)

fabiocos · 2019-05-11T20:56:45Z

@kpedro88 the ball is in your field now...

kpedro88 · 2019-05-11T20:58:34Z

+upgrade

cmsbuild · 2019-05-11T20:59:07Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-05-11T21:05:26Z

+1

fabiocos · 2019-05-11T21:06:01Z

merge

dseith added 2 commits March 5, 2019 16:41

Update pfDeepCSVJetTags_cfi.py

3411d3f

added correct json for PhaseII to pfDeepCSVJetTags_cfi.py

Merge pull request #1 from dseith/reco_btag_combined_phase2_deepcsv

3be52ce

Update pfDeepCSVJetTags_cfi.py

cmsbuild added this to the CMSSW_10_6_X milestone Mar 5, 2019

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Mar 5, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 5, 2019

dseith mentioned this pull request Mar 5, 2019

Phase II models for DeepCSV and DeepJet cms-data/RecoBTag-Combined#20

Merged

cmsbuild mentioned this pull request Mar 5, 2019

[10.6.X] Updated RecoBTag-Combined: Phase II models for DeepCSV and DeepJet cms-sw/cmsdist#4752

Closed

cmsbuild added requires-external tests-started and removed tests-pending labels Mar 5, 2019

cmsbuild added tests-approved and removed tests-started labels Mar 6, 2019

cmsbuild added comparison-pending tests-pending and removed tests-approved labels May 9, 2019

cmsbuild added tests-started and removed tests-pending labels May 9, 2019

cmsbuild added tests-rejected and removed tests-started labels May 10, 2019

cmsbuild added comparison-available and removed comparison-pending labels May 10, 2019

cmsbuild added reconstruction-approved and removed reconstruction-pending labels May 11, 2019

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels May 11, 2019

cmsbuild added orp-approved and removed orp-pending labels May 11, 2019

cmsbuild merged commit 9cc73ff into cms-sw:master May 11, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated json for DeepCSV Phase 2 model #26072

Updated json for DeepCSV Phase 2 model #26072

dseith commented Mar 5, 2019

cmsbuild commented Mar 5, 2019

cmsbuild commented Mar 5, 2019

cmsbuild commented Mar 5, 2019

slava77 commented Mar 5, 2019

dseith commented Mar 5, 2019

dseith commented Mar 5, 2019

dseith commented Mar 5, 2019

smuzaffar commented Mar 5, 2019

cmsbuild commented Mar 5, 2019 •

edited

cmsbuild commented Mar 6, 2019

cmsbuild commented Mar 6, 2019

cmsbuild commented Mar 6, 2019

cmsbuild commented May 9, 2019 •

edited

cmsbuild commented May 10, 2019

cmsbuild commented May 10, 2019

cmsbuild commented May 10, 2019

slava77 commented May 10, 2019

slava77 commented May 11, 2019

slava77 commented May 11, 2019

fabiocos commented May 11, 2019

kpedro88 commented May 11, 2019

cmsbuild commented May 11, 2019

fabiocos commented May 11, 2019

fabiocos commented May 11, 2019

Updated json for DeepCSV Phase 2 model #26072

Updated json for DeepCSV Phase 2 model #26072

Conversation

dseith commented Mar 5, 2019

cmsbuild commented Mar 5, 2019

cmsbuild commented Mar 5, 2019

cmsbuild commented Mar 5, 2019

slava77 commented Mar 5, 2019

dseith commented Mar 5, 2019

dseith commented Mar 5, 2019

dseith commented Mar 5, 2019

smuzaffar commented Mar 5, 2019

cmsbuild commented Mar 5, 2019 • edited

cmsbuild commented Mar 6, 2019

cmsbuild commented Mar 6, 2019

cmsbuild commented Mar 6, 2019

cmsbuild commented May 9, 2019 • edited

cmsbuild commented May 10, 2019

cmsbuild commented May 10, 2019

cmsbuild commented May 10, 2019

slava77 commented May 10, 2019

slava77 commented May 11, 2019

slava77 commented May 11, 2019

fabiocos commented May 11, 2019

kpedro88 commented May 11, 2019

cmsbuild commented May 11, 2019

fabiocos commented May 11, 2019

fabiocos commented May 11, 2019

cmsbuild commented Mar 5, 2019 •

edited

cmsbuild commented May 9, 2019 •

edited