Backport to 9_4_X of TensorFlow-based DeepDoubleB tagger #23582

jmduarte · 2018-06-13T17:23:16Z

Backport of #23033 so the DeepDoubleB tagger may be used in 9_4_X. The tagger is disabled in MINIAOD by default.

Rebased and locally tested over CMSSW_9_4_X_2018-06-10-0000

…ig,tau2_flightDistance2dSig

…ats/BTauReco/interface/TaggingVariable.h

…guards

slava77 · 2018-06-15T16:33:03Z

RecoBTag/TensorFlow/BuildFile.xml

+<use name="PhysicsTools/TensorFlow"/>
+<use name="TrackingTools/IPTools"/>
+<use name="DataFormats/ParticleFlowCandidate"/>
+<use name="DataFormats/PatCandidates"/>


These DataFormats were not needed in the master version of this file.
Was it really needed here?

Yes, I'm not entirely sure what changed, but when I remove the two DataFormats lines, I get compilation errors like:

>> Building shared library tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/libRecoBTagTensorFlow.so tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/TrackInfoBuilder.o: In function `btagbtvdeep::TrackInfoBuilder::buildTrackInfo(reco::Candidate const*, ROOT::Math::DisplacementVector3D<ROOT::Math::Cartesian3D<double>, ROOT::Math::DefaultCoordinateSystemTag> const&, Vector3DBase<float, GlobalTag>, reco::Vertex const&)': TrackInfoBuilder.cc:(.text+0x14e): undefined reference to `typeinfo for pat::PackedCandidate' TrackInfoBuilder.cc:(.text+0x162): undefined reference to `typeinfo for reco::PFCandidate' TrackInfoBuilder.cc:(.text+0x1ca): undefined reference to `reco::PFCandidate::trackRef() const' TrackInfoBuilder.cc:(.text+0x1eb): undefined reference to `reco::PFCandidate::trackRef() const' TrackInfoBuilder.cc:(.text+0x20e): undefined reference to `reco::PFCandidate::trackRef() const' TrackInfoBuilder.cc:(.text+0x8dc): undefined reference to `reco::PFCandidate::gsfTrackRef() const' TrackInfoBuilder.cc:(.text+0x8fd): undefined reference to `reco::PFCandidate::gsfTrackRef() const' TrackInfoBuilder.cc:(.text+0x920): undefined reference to `reco::PFCandidate::gsfTrackRef() const' TrackInfoBuilder.cc:(.text+0x1792): undefined reference to `pat::PackedCandidate::unpackTrk() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/TrackInfoBuilder.o: In function `pat::PackedCandidate::pseudoTrack() const': TrackInfoBuilder.cc:(.text._ZNK3pat15PackedCandidate11pseudoTrackEv[_ZNK3pat15PackedCandidate11pseudoTrackEv]+0x19): undefined reference to `pat::PackedCandidate::unpackTrk() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/TrackInfoBuilder.o: In function `reco::PFCandidate::bestTrack() const': TrackInfoBuilder.cc:(.text._ZNK4reco11PFCandidate9bestTrackEv[_ZNK4reco11PFCandidate9bestTrackEv]+0x39): undefined reference to `reco::PFCandidate::trackRef() const' TrackInfoBuilder.cc:(.text._ZNK4reco11PFCandidate9bestTrackEv[_ZNK4reco11PFCandidate9bestTrackEv]+0x54): undefined reference to `reco::PFCandidate::trackRef() const' TrackInfoBuilder.cc:(.text._ZNK4reco11PFCandidate9bestTrackEv[_ZNK4reco11PFCandidate9bestTrackEv]+0x6f): undefined reference to `reco::PFCandidate::trackRef() const' TrackInfoBuilder.cc:(.text._ZNK4reco11PFCandidate9bestTrackEv[_ZNK4reco11PFCandidate9bestTrackEv]+0x169): undefined reference to `reco::PFCandidate::gsfTrackRef() const' TrackInfoBuilder.cc:(.text._ZNK4reco11PFCandidate9bestTrackEv[_ZNK4reco11PFCandidate9bestTrackEv]+0x184): undefined reference to `reco::PFCandidate::gsfTrackRef() const' TrackInfoBuilder.cc:(.text._ZNK4reco11PFCandidate9bestTrackEv[_ZNK4reco11PFCandidate9bestTrackEv]+0x1a4): undefined reference to `reco::PFCandidate::gsfTrackRef() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ShallowTagInfoConverter.o: In function `btagbtvdeep::bTagToFeatures(reco::TaggingVariableList const&, btagbtvdeep::ShallowTagInfoFeatures&)': ShallowTagInfoConverter.cc:(.text+0x1a): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' ShallowTagInfoConverter.cc:(.text+0x33): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' ShallowTagInfoConverter.cc:(.text+0x4d): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' ShallowTagInfoConverter.cc:(.text+0x67): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' ShallowTagInfoConverter.cc:(.text+0x81): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ShallowTagInfoConverter.o:ShallowTagInfoConverter.cc:(.text+0x9b): more undefined references to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' follow tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ShallowTagInfoConverter.o: In function `btagbtvdeep::bTagToFeatures(reco::TaggingVariableList const&, btagbtvdeep::ShallowTagInfoFeatures&)': ShallowTagInfoConverter.cc:(.text+0xe6): undefined reference to `reco::TaggingVariableList::getList(reco::btau::TaggingVariableName, bool) const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/BoostedDoubleSVTagInfoConverter.o: In function `btagbtvdeep::doubleBTagToFeatures(reco::TaggingVariableList const&, btagbtvdeep::BoostedDoubleSVTagInfoFeatures&)': BoostedDoubleSVTagInfoConverter.cc:(.text+0x1a): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' BoostedDoubleSVTagInfoConverter.cc:(.text+0x33): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' BoostedDoubleSVTagInfoConverter.cc:(.text+0x4d): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' BoostedDoubleSVTagInfoConverter.cc:(.text+0x67): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' BoostedDoubleSVTagInfoConverter.cc:(.text+0x81): undefined reference to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/BoostedDoubleSVTagInfoConverter.o:BoostedDoubleSVTagInfoConverter.cc:(.text+0x9b): more undefined references to `reco::TaggingVariableList::get(reco::btau::TaggingVariableName, float) const' follow tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/NeutralCandidateConverter.o: In function `btagbtvdeep::packedCandidateToFeatures(pat::PackedCandidate const*, pat::Jet const&, float, float, btagbtvdeep::NeutralCandidateFeatures&)': NeutralCandidateConverter.cc:(.text+0x1c8): undefined reference to `pat::PackedCandidate::puppiWeight() const' NeutralCandidateConverter.cc:(.text+0x241): undefined reference to `pat::PackedCandidate::unpack() const' NeutralCandidateConverter.cc:(.text+0x254): undefined reference to `pat::PackedCandidate::unpack() const' NeutralCandidateConverter.cc:(.text+0x270): undefined reference to `pat::PackedCandidate::unpack() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/NeutralCandidateConverter.o: In function `pat::PackedCandidate::pt() const': NeutralCandidateConverter.cc:(.text._ZNK3pat15PackedCandidate2ptEv[_ZNK3pat15PackedCandidate2ptEv]+0x21): undefined reference to `pat::PackedCandidate::unpack() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/NeutralCandidateConverter.o: In function `pat::PackedCandidate::phi() const': NeutralCandidateConverter.cc:(.text._ZNK3pat15PackedCandidate3phiEv[_ZNK3pat15PackedCandidate3phiEv]+0x21): undefined reference to `pat::PackedCandidate::unpack() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/NeutralCandidateConverter.o:NeutralCandidateConverter.cc:(.text._ZNK3pat15PackedCandidate3etaEv[_ZNK3pat15PackedCandidate3etaEv]+0x21): more undefined references to `pat::PackedCandidate::unpack() const' follow tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ChargedCandidateConverter.o: In function `btagbtvdeep::packedCandidateToFeatures(pat::PackedCandidate const*, pat::Jet const&, btagbtvdeep::TrackInfoBuilder const&, float, float, btagbtvdeep::ChargedCandidateFeatures&)': ChargedCandidateConverter.cc:(.text+0x42): undefined reference to `pat::PackedCandidate::puppiWeight() const' ChargedCandidateConverter.cc:(.text+0x15c): undefined reference to `pat::PackedCandidate::unpackTrk() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ChargedCandidateConverter.o: In function `btagbtvdeep::recoCandidateToFeatures(reco::PFCandidate const*, reco::Jet const&, btagbtvdeep::TrackInfoBuilder const&, float, float, float, int, edm::Ref<std::vector<reco::Vertex, std::allocator<reco::Vertex> >, reco::Vertex, edm::refhelper::FindUsingAdvance<std::vector<reco::Vertex, std::allocator<reco::Vertex> >, reco::Vertex> > const&, btagbtvdeep::ChargedCandidateFeatures&)': ChargedCandidateConverter.cc:(.text+0x1dd): undefined reference to `reco::PFCandidate::trackRef() const' ChargedCandidateConverter.cc:(.text+0x1f8): undefined reference to `reco::PFCandidate::trackRef() const' ChargedCandidateConverter.cc:(.text+0x38b): undefined reference to `reco::PFCandidate::trackRef() const' ChargedCandidateConverter.cc:(.text+0x3a9): undefined reference to `reco::PFCandidate::trackRef() const' ChargedCandidateConverter.cc:(.text+0x3c4): undefined reference to `reco::PFCandidate::trackRef() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ChargedCandidateConverter.o:ChargedCandidateConverter.cc:(.text+0x42c): more undefined references to `reco::PFCandidate::trackRef() const' follow tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ChargedCandidateConverter.o: In function `btagbtvdeep::recoCandidateToFeatures(reco::PFCandidate const*, reco::Jet const&, btagbtvdeep::TrackInfoBuilder const&, float, float, float, int, edm::Ref<std::vector<reco::Vertex, std::allocator<reco::Vertex> >, reco::Vertex, edm::refhelper::FindUsingAdvance<std::vector<reco::Vertex, std::allocator<reco::Vertex> >, reco::Vertex> > const&, btagbtvdeep::ChargedCandidateFeatures&)': ChargedCandidateConverter.cc:(.text+0x7a9): undefined reference to `reco::PFCandidate::gsfTrackRef() const' ChargedCandidateConverter.cc:(.text+0x7c4): undefined reference to `reco::PFCandidate::gsfTrackRef() const' ChargedCandidateConverter.cc:(.text+0x7df): undefined reference to `reco::PFCandidate::gsfTrackRef() const' ChargedCandidateConverter.cc:(.text+0x899): undefined reference to `reco::PFCandidate::gsfTrackRef() const' ChargedCandidateConverter.cc:(.text+0x8b7): undefined reference to `reco::PFCandidate::gsfTrackRef() const' tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ChargedCandidateConverter.o:ChargedCandidateConverter.cc:(.text+0x8d2): more undefined references to `reco::PFCandidate::gsfTrackRef() const' follow tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/ChargedCandidateConverter.o: In function `void btagbtvdeep::commonCandidateToFeatures<pat::PackedCandidate>(pat::PackedCandidate const*, reco::Jet const&, btagbtvdeep::TrackInfoBuilder const&, float const&, float const&, btagbtvdeep::ChargedCandidateFeatures&)': ChargedCandidateConverter.cc:(.text._ZN11btagbtvdeep25commonCandidateToFeaturesIN3pat15PackedCandidateEEEvPKT_RKN4reco3JetERKNS_16TrackInfoBuilderERKfSE_RNS_24ChargedCandidateFeaturesE[_ZN11btagbtvdeep25commonCandidateToFeaturesIN3pat15PackedCandidateEEEvPKT_RKN4reco3JetERKNS_16TrackInfoBuilderERKfSE_RNS_24ChargedCandidateFeaturesE]+0x2a6): undefined reference to `pat::PackedCandidate::unpack() const' collect2: error: ld returned 1 exit status config/SCRAM/GMake/Makefile.rules:2068: recipe for target 'tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/libRecoBTagTensorFlow.so' failed gmake: *** [tmp/slc6_amd64_gcc630/src/RecoBTag/TensorFlow/src/RecoBTagTensorFlow/libRecoBTagTensorFlow.so] Error 1 gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

ok
nothing to do for this file then.

slava77 · 2018-06-21T09:22:42Z

ping
there are two unresolved comments in this PR review.

cmsbuild · 2018-06-21T12:16:24Z

Pull request #23582 was updated. @perrotta, @monttj, @cmsbuild, @slava77, @gpetruc, @arizzi can you please check and sign again.

slava77 · 2018-06-21T12:30:02Z

@cmsbuild please test

cmsbuild · 2018-06-21T12:30:24Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28799/console Started: 2018/06/21 14:31
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28800/console Started: 2018/06/21 14:49

cmsbuild · 2018-06-21T14:26:48Z

+1
Tested at: eb073bb
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23582/28800/summary.html

cmsbuild · 2018-06-21T14:26:59Z

Comparison job queued.

cmsbuild · 2018-06-21T15:07:48Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23582/28800/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 27
DQMHistoTests: Total histograms compared: 2721493
DQMHistoTests: Total failures: 104
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2721227
DQMHistoTests: Total skipped: 162
DQMHistoTests: Total Missing objects: 0

slava77 · 2018-06-26T07:15:03Z

backport of #23033

slava77 · 2018-06-26T07:20:51Z

+1

for #23582 eb073bb

backport from the master version is correct, apart from disabling the double-b tagger in PhysicsTools/PatAlgos/python/slimming/applyDeepBtagging_cff.py in order to preserve compatibility with existing workflows in 94X. The double-b tagger can be enabled at analysis level if desired.
jenkins tests pass and comparisons with the baseline show no differences as expected

fabiocos · 2018-07-03T16:19:07Z

+1

fabiocos · 2018-07-04T08:49:42Z

merge

Javier and others added 30 commits June 10, 2018 22:19

add double-b features

ab2442c

rearrange files

daac8fa

update src and interface

331fca7

define templates in .h, instantiate in .cc

7a97867

update tensor shape; fix tensor filler

cb6d367

fix double-b inputs, except for slight diff in tau1_flightDistance2dS…

060bd56

…ig,tau2_flightDistance2dSig

fix packed candidate loop for AK8 jets (double loop)

b8ee263

add ptratio to charged candidate

be36349

make jet radius configurable for drmin

86dbd2c

fix producer for RECO/AOD

56c846b

testing from MINIAOD

ca5e3f4

remove vertex_associator from deepdoubleb in jetTools

5505b34

rm printouts and vertex associator

4ba69b5

testing configs

053eebe

update default model file with pt cut and mass sculpting penalty

46b9363

enable DeepDoubleB tagger in MINIAOD

6c730c6

add class versions

d66c827

add comment that these variables should match the content of DataForm…

0db5231

…ats/BTauReco/interface/TaggingVariable.h

factorize deep b-tagging on top of MINIAOD into applyDeepBtagging_cff.py

12c8d45

remove unnecessary tags in BuildFile.xml; fix indenting

46c742a

fix naming conventions for converter classes, functions, and include …

e938b24

…guards

double jetR -> float jetR

f6bb68d

add comment on unused variable

0442ad1

remove commented out code

29b8997

generalize range for different jetR

cdb2d90

add newline for readability

047f27a

moved unnecessary includes for interface from .h to .cc

854153d

fix indentation in deep_helpers

666a1c9

clang-format DeepDoubleBTagInfoProducer

aea698c

mv RecoBTag/DeepFlavour -> RecoBTag/TensorFlow

e909dbb

slava77 reviewed Jun 15, 2018

View reviewed changes

add boolean (false by default) to control whether to apply deep double-b

eb073bb

cmsbuild added comparison-pending tests-pending and removed comparison-available requires-external tests-approved labels Jun 21, 2018

cmsbuild added tests-started and removed tests-pending labels Jun 21, 2018

cmsbuild added tests-approved and removed tests-started labels Jun 21, 2018

cmsbuild added comparison-available and removed comparison-pending labels Jun 21, 2018

cmsbuild added the backport-ok label Jun 26, 2018

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Jun 26, 2018

cmsbuild added orp-approved and removed orp-pending labels Jul 3, 2018

cmsbuild merged commit 795d4ef into cms-sw:CMSSW_9_4_X Jul 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport to 9_4_X of TensorFlow-based DeepDoubleB tagger #23582

Backport to 9_4_X of TensorFlow-based DeepDoubleB tagger #23582

jmduarte commented Jun 13, 2018

slava77 Jun 15, 2018

jmduarte Jun 21, 2018

slava77 Jun 21, 2018

slava77 commented Jun 21, 2018

cmsbuild commented Jun 21, 2018

slava77 commented Jun 21, 2018

cmsbuild commented Jun 21, 2018 •

edited

cmsbuild commented Jun 21, 2018

cmsbuild commented Jun 21, 2018

cmsbuild commented Jun 21, 2018

slava77 commented Jun 26, 2018

slava77 commented Jun 26, 2018

fabiocos commented Jul 3, 2018

fabiocos commented Jul 4, 2018

Backport to 9_4_X of TensorFlow-based DeepDoubleB tagger #23582

Backport to 9_4_X of TensorFlow-based DeepDoubleB tagger #23582

Conversation

jmduarte commented Jun 13, 2018

slava77 Jun 15, 2018

Choose a reason for hiding this comment

jmduarte Jun 21, 2018

Choose a reason for hiding this comment

slava77 Jun 21, 2018

Choose a reason for hiding this comment

slava77 commented Jun 21, 2018

cmsbuild commented Jun 21, 2018

slava77 commented Jun 21, 2018

cmsbuild commented Jun 21, 2018 • edited

cmsbuild commented Jun 21, 2018

cmsbuild commented Jun 21, 2018

cmsbuild commented Jun 21, 2018

slava77 commented Jun 26, 2018

slava77 commented Jun 26, 2018

fabiocos commented Jul 3, 2018

fabiocos commented Jul 4, 2018

cmsbuild commented Jun 21, 2018 •

edited