Update of the Beam Halo Filter (both CSC and Calo algos) #13926

lathomas · 2016-04-04T21:23:13Z

Update of the Beam Halo Filter. This PR is an update of:
#13849

The strategy to catch beam halo events is described in the following presentation:
https://indico.cern.ch/event/514418/contribution/17/attachments/1250430/1843419/laurent_v2.pdf
The implementation has however changed according to the comments made during the meeting: the information related to calorimeters is now saved in EcalHaloData and HcalHaloData (DataFormats/METReco) rather than in CSCHaloData. Methods using both calo and CSC information are saved in GlobalHaloData.
These classes are produced by CSC/Ecal/Hcal/Global HaloProducer (in RecoMET/METProducers). The actual code filling these classes is in CSC/Ecal/Hcal/Global HaloAlgo (in RecoMET/METAlgorithms).
A version of the CSCTightHalo filter similar to 2015 is still present (and still the default, for now). It contains the various changes:
-Methods using halo trigger bits are set to trivial values.
-The matching of segments in opposite endcaps is slightly optimized.
Two new filters are also added:
GlobalTightHaloFilter2016 and GlobalSuperTightHaloFilter2016
The flags are also saved in MINIAOD.
These filters use methods from GlobalHaloData, (which is saved in AOD).
The Ecal/Hcal HaloData classes now contain a list to all the ECAL/HCAL halo cluster candidates, which themselves contain all the rechits with et>1 (ecal) or 2 (hcal).
These classes are saved in RECO but not AOD so far. I got the following size (using edmEventSize):

SinglePhoton Run2015D:
recoCSCHaloData_CSCHaloData__reRECO. 128.03 29.67
recoEcalHaloData_EcalHaloData__reRECO. 489.93 204.79
recoHcalHaloData_HcalHaloData__reRECO. 178.04 70.63
recoGlobalHaloData_GlobalHaloData__reRECO. 186.59 35.8
recoBeamHaloSummary_BeamHaloSummary__reRECO. 131.45 28.75

TTbar:
recoCSCHaloData_CSCHaloData__RECO. 127.53 26.64
recoEcalHaloData_EcalHaloData__RECO. 380.77 164.5
recoHcalHaloData_HcalHaloData__RECO. 214.7 95.28
recoGlobalHaloData_GlobalHaloData__RECO. 185.69 34.96
recoBeamHaloSummary_BeamHaloSummary__RECO. 131.09 28.14

The producer CaloRecHitsBeamHaloCleaned (in RecoMET/METProducers) returns a list of clean rechits. It's planned to use it for MET HLT triggers to clean beam halos.

cmsbuild · 2016-04-04T21:23:41Z

A new Pull Request was created by @lathomas for CMSSW_8_1_X.

It involves the following packages:

DataFormats/METReco
PhysicsTools/PatAlgos
RecoMET/METAlgorithms
RecoMET/METFilters
RecoMET/METProducers

@cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@TaiSakuma, @imarches, @ahinzmann, @acaudron, @mmarionncern, @rappoccio, @jdolen, @nhanvtran, @schoef, @ferencek, @gpetruc, @mariadalfonso, @pvmulder this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

slava77 · 2016-04-04T21:24:07Z

@cmsbuild please test

cmsbuild · 2016-04-04T21:24:26Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12166/console

cmsbuild · 2016-04-05T01:44:11Z

Comparison not run due to Build errors (RelVals and Igprof tests were also skipped)

cmsbuild · 2016-04-05T01:44:49Z

-1
Tested at: 536fb20
I found an error when building:

>> Compiling  /build/cmsbuild/jenkins-workarea/workspace/ib-any-integration/CMSSW_8_1_X_2016-04-04-1100/src/L1Trigger/CSCTrackFinder/src/CSCTFTrackBuilder.cc 
Copying tmp/slc6_amd64_gcc530/src/HLTrigger/Timer/src/HLTriggerTimer/libHLTriggerTimer.so to productstore area:
In file included from /build/cmsbuild/jenkins-workarea/workspace/ib-any-integration/CMSSW_8_1_X_2016-04-04-1100/src/L1Trigger/CSCTrackFinder/interface/CSCTFSPCoreLogic.h:10:0,
                 from /build/cmsbuild/jenkins-workarea/workspace/ib-any-integration/CMSSW_8_1_X_2016-04-04-1100/src/L1Trigger/CSCTrackFinder/interface/CSCTFSectorProcessor.h:20,
                 from /build/cmsbuild/jenkins-workarea/workspace/ib-any-integration/CMSSW_8_1_X_2016-04-04-1100/src/L1Trigger/CSCTrackFinder/src/CSCTFSectorProcessor.cc:1:
/build/cmsbuild/jenkins-workarea/workspace/ib-any-integration/CMSSW_8_1_X_2016-04-04-1100/poison/L1Trigger/CSCTrackFinder/src/core_2010_01_22/vpp_generated.h:1:2: error: #error THIS FILE HAS BEEN REMOVED FROM THE PACKAGE.
 #error THIS FILE HAS BEEN REMOVED FROM THE PACKAGE.
  ^
In file included from /build/cmsbuild/jenkins-workarea/workspace/ib-any-integration/CMSSW_8_1_X_2016-04-04-1100/src/L1Trigger/CSCTrackFinder/interface/CSCTFSPCoreLogic.h:11:0,
                 from /build/cmsbuild/jenkins-workarea/workspace/ib-any-integration/CMSSW_8_1_X_2016-04-04-1100/src/L1Trigger/CSCTrackFinder/interface/CSCTFSectorProcessor.h:20,
                 from /build/cmsbuild/jenkins-workarea/workspace/ib-any-integration/CMSSW_8_1_X_2016-04-04-1100/src/L1Trigger/CSCTrackFinder/src/CSCTFSectorProcessor.cc:1:

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13926/12166/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
a1b87ee
a1a7545
8e18df4
11c7a67
2c103d5
40cddb8
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13926/12166/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13926/12166/git-merge-result

slava77 · 2016-04-05T02:45:11Z

@lathomas
looks like you rebased all the way to the head of 81X (instead of the last working IB as I suggested) and jenkins couldn't build it due to externals updates.
We'll need to wait for the next IB to show up (CMSSW_8_1_X_2016-04-04-2300 or whatever works after it)

lathomas · 2016-04-05T12:49:16Z

@slava77 OK, sorry for the mistake. Do I have to do anything or can you just restart the tests one the next IB is available?

slava77 · 2016-04-05T13:06:21Z

@lathomas
I will restart some time later.
The release is out, but cms-bot/jenkins hasn't recognized it yet

lathomas · 2016-04-05T13:08:27Z

@slava77 Thank you ! Hope it will go smoothly this time !

slava77 · 2016-04-05T13:53:02Z

@cmsbuild please test
new IB and baseline jenkins results are out. This should be good to test now.

cmsbuild · 2016-04-05T13:53:24Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12174/console

cmsbuild · 2016-04-19T15:26:52Z

Pull request #13926 was updated. @cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please check and sign again.

slava77 · 2016-04-19T16:14:03Z

@cmsbuild please test

@lathomas thank you for the quick fix.

cmsbuild · 2016-04-19T16:14:22Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12495/console

cmsbuild · 2016-04-19T22:03:42Z

+1
Tested at: e856ab1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13926/12495/summary.html

slava77 · 2016-04-20T03:27:29Z

@cmsbuild please test
maybe comparisons complete this time

cmsbuild · 2016-04-20T03:28:40Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12504/console

slava77 · 2016-04-20T04:41:05Z

+1

for #13926 8ed4b41

code changes are in line with description, presentation in a reco meeting and in follow up to the code review
jenkins tests pass and comparisons with baseline (comparisons rerun manually using outputs from jenkins) show differences only in halo-related distributions; new variables are not monitored yet. There are no new products saved, only changes to existing ones.
- some validation/DQM plots for new halo data products is needed, eventually
technical performance checks were done earlier Update of the Beam Halo Filter (both CSC and Calo algos) #13926 (comment), latest updates shouldn't affect the results adversely.

cmsbuild · 2016-04-20T06:12:56Z

-1

Tested at: e856ab1

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13926/12504/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following worklfows:
140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

134.911 step3

runTheMatrix-results/134.911_RunSinglePh2015D+RunSinglePh2015D+HLTDR2_25ns+RECODR2_25nsreHLT+HARVESTDR2_25nsreHLT/step3_RunSinglePh2015D+RunSinglePh2015D+HLTDR2_25ns+RECODR2_25nsreHLT+HARVESTDR2_25nsreHLT.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4.log

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 20 07:47:28 2016-date Wed Apr 20 07:41:07 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=HeavyIons -n 10 --conditions auto:run2_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 20 07:49:37 2016-date Wed Apr 20 07:41:10 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 20 07:47:32 2016-date Wed Apr 20 07:42:24 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 20 07:53:01 2016-date Wed Apr 20 07:50:00 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 20 07:56:58 2016-date Wed Apr 20 07:50:04 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_DATA.root --fileout file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 20 07:57:34 2016-date Wed Apr 20 07:53:08 2016 s - exit: 16640

slava77 · 2016-04-20T06:22:42Z

failures in tests are present in the IB as well due to JetCorrectionsRecord availability in the GT

cmsbuild · 2016-04-20T10:27:22Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13926/12495/summary.html

Update of the Beam Halo Filter (both CSC and Calo algos)

536fb20

cmsbuild added this to the Next CMSSW_8_1_X milestone Apr 4, 2016

cmsbuild added reconstruction-pending analysis-pending pending-signatures tests-pending orp-pending comparison-pending labels Apr 4, 2016

cmsbuild added tests-started and removed tests-pending labels Apr 4, 2016

cmsbuild added comparison-notrun and removed comparison-pending labels Apr 5, 2016

cmsbuild added tests-rejected comparison-pending and removed comparison-notrun tests-started labels Apr 5, 2016

cmsbuild added tests-pending and removed tests-rejected labels Apr 5, 2016

cmsbuild removed the tests-pending label Apr 5, 2016

Update

e856ab1

cmsbuild added tests-pending comparison-pending and removed comparison-available tests-approved labels Apr 19, 2016

cmsbuild added tests-started and removed tests-pending labels Apr 19, 2016

cmsbuild added tests-approved and removed tests-started labels Apr 19, 2016

cmsbuild added tests-pending and removed tests-approved labels Apr 20, 2016

cmsbuild added tests-started and removed tests-pending labels Apr 20, 2016

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Apr 20, 2016

cmsbuild added tests-rejected and removed tests-started labels Apr 20, 2016

davidlange6 merged commit f119195 into cms-sw:CMSSW_8_1_X Apr 20, 2016

cmsbuild added comparison-available and removed comparison-pending labels Apr 20, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update of the Beam Halo Filter (both CSC and Calo algos) #13926

Update of the Beam Halo Filter (both CSC and Calo algos) #13926

lathomas commented Apr 4, 2016

cmsbuild commented Apr 4, 2016

slava77 commented Apr 4, 2016

cmsbuild commented Apr 4, 2016

cmsbuild commented Apr 5, 2016

cmsbuild commented Apr 5, 2016

slava77 commented Apr 5, 2016

lathomas commented Apr 5, 2016

slava77 commented Apr 5, 2016

lathomas commented Apr 5, 2016

slava77 commented Apr 5, 2016

cmsbuild commented Apr 5, 2016

cmsbuild commented Apr 19, 2016

slava77 commented Apr 19, 2016

cmsbuild commented Apr 19, 2016

cmsbuild commented Apr 19, 2016

slava77 commented Apr 20, 2016

cmsbuild commented Apr 20, 2016

slava77 commented Apr 20, 2016

cmsbuild commented Apr 20, 2016

slava77 commented Apr 20, 2016

cmsbuild commented Apr 20, 2016

Update of the Beam Halo Filter (both CSC and Calo algos) #13926

Update of the Beam Halo Filter (both CSC and Calo algos) #13926

Conversation

lathomas commented Apr 4, 2016

cmsbuild commented Apr 4, 2016

slava77 commented Apr 4, 2016

cmsbuild commented Apr 4, 2016

cmsbuild commented Apr 5, 2016

cmsbuild commented Apr 5, 2016

slava77 commented Apr 5, 2016

lathomas commented Apr 5, 2016

slava77 commented Apr 5, 2016

lathomas commented Apr 5, 2016

slava77 commented Apr 5, 2016

cmsbuild commented Apr 5, 2016

cmsbuild commented Apr 19, 2016

slava77 commented Apr 19, 2016

cmsbuild commented Apr 19, 2016

cmsbuild commented Apr 19, 2016

slava77 commented Apr 20, 2016

cmsbuild commented Apr 20, 2016

slava77 commented Apr 20, 2016

cmsbuild commented Apr 20, 2016

slava77 commented Apr 20, 2016

cmsbuild commented Apr 20, 2016