pr90x L1T GT packer of ExternalConditions #16794

rekovic · 2016-11-29T15:31:17Z

90X PR: Needed for HI MC production.

Put back packing of ExternalConditions in packer GTSetup.
Configure default to use simGtExtFakeStage2Digis which is part of SimL1Emulator sequence by default.

Also fix L1REPACK:uGT configuration to use the same default ExtInputTag=simGtExtFakeStage2Digis.

…e default to use simGtExtFakeStage2Digis. Fix L1REPACK:uGT configuration to use the same default.

rekovic · 2016-11-29T15:31:35Z

please test

cmsbuild · 2016-11-29T15:31:43Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/16654/console

cmsbuild · 2016-11-29T15:31:44Z

A new Pull Request was created by @rekovic for CMSSW_9_0_X.

It involves the following packages:

Configuration/StandardSequences
EventFilter/L1TRawToDigi

@cmsbuild, @rekovic, @franzoni, @mulhearn, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @kreczko, @makortel, @felicepantaleo, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @dgulhan this is something you requested to watch as well.
@slava77, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

rekovic · 2016-11-29T16:03:51Z

+1

cmsbuild · 2016-11-29T17:12:46Z

+1
Tested at: 8a0fcd7
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16794/16654/summary.html

cmsbuild · 2016-11-29T17:12:48Z

Comparison job queued.

cmsbuild · 2016-11-29T20:04:48Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16794/16654/summary.html

The workflows 1003.0, 1001.0, 1000.0, 140.53, 136.731, 4.22 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

davidlange6 · 2016-11-30T12:41:14Z

Configuration/StandardSequences/python/SimL1EmulatorRepack_uGT_cff.py

@@ -46,7 +46,7 @@
    packGtStage2.JetInputTag    = cms.InputTag("unpackCaloStage2","Jet")
    packGtStage2.EtSumInputTag  = cms.InputTag("unpackCaloStage2","EtSum")
    packGtStage2.GtInputTag     = cms.InputTag("simGtStage2Digis") # as in default
-    packGtStage2.ExtInputTag    = cms.InputTag("simGtStage2Digis") # as in default
+    packGtStage2.ExtInputTag    = cms.InputTag("simGtExtFakeStage2Digis") # as in default


@rekovic - since I'm not sure about what monitoring we actually have in place - what is the effect of this change on pp MC (eg, the Moriond configuration)?

@davidlange6
I did some investigation, and I started a github issue cms-l1t-offline#440. I expect @fwyzard / @Martin-Grunewald will comment.

… for either RAW DATA or RAW MC.

cmsbuild · 2016-12-09T13:19:26Z

Pull request #16794 was updated. @cmsbuild, @rekovic, @franzoni, @mulhearn, @davidlange6 can you please check and sign again.

rekovic · 2016-12-09T13:56:12Z

+1

rekovic · 2016-12-09T13:56:19Z

please test

cmsbuild · 2016-12-09T13:56:27Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/16898/console Started: 2016/12/09 14:57

cmsbuild · 2016-12-09T15:08:04Z

+1
Tested at: 8b92951
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16794/16898/summary.html

cmsbuild · 2016-12-09T15:08:10Z

Comparison job queued.

cmsbuild · 2016-12-09T19:21:25Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16794/16898/summary.html

rekovic · 2016-12-12T08:55:00Z

@davidlange6 This is all understood now. Can merge.
HLT for p-p does not use BPTX (Externals) and Moriond 2017 MC Production was validated as OK
without the ExternalConditions packed.

davidlange6 · 2016-12-12T09:11:56Z

i'm not sure I follow the thread - unless you are doing something with the random number seed (or running threaded), you expect 1 to 1 agreement unless something changes even if random numbers are used. can you clarify?

rekovic · 2016-12-12T12:54:00Z

@davidlange6 double checking. will report again.

fwyzard · 2016-12-13T08:53:25Z

@davidlange6 I don't think the (lack of) reproducibility is tied to these changes; it is already there in CMSSW 8.0.24-patch1 .
Running the DIGI+L1 step with

cmsDriver.py test \
    -s DIGI:pdigi,L1 \
    --no_output \
    --python_filename=test.py \
    --processName HLTX \
    --conditions 80X_mcRun2_asymptotic_2016_TrancheIV_v6_Tr4GT_v6 \
    --era Run2_2016 \
    --filein /store/relval/CMSSW_8_0_21/RelValTTbar_13/GEN-SIM-DIGI-RAW-HLTDEBUG/PU25ns_80X_mcRun2_asymptotic_2016_TrancheIV_v6_Tr4GT_v6-v1/10000/049B4664-8A98-E611-8F0C-0025905B8562.root \
    -n 1000 \
    --customise=HLTrigger/Configuration/CustomConfigs.L1THLT

multiple times with 4 threads leads to different L1 results, but running only the L1 step with

cmsDriver.py test \
    -s L1 \
    --no_output \
    --python_filename=test.py \
    --processName HLTX \
    --conditions 80X_mcRun2_asymptotic_2016_TrancheIV_v6_Tr4GT_v6 \
    --era Run2_2016 \
    --filein /store/relval/CMSSW_8_0_21/RelValTTbar_13/GEN-SIM-DIGI-RAW-HLTDEBUG/PU25ns_80X_mcRun2_asymptotic_2016_TrancheIV_v6_Tr4GT_v6-v1/10000/049B4664-8A98-E611-8F0C-0025905B8562.root \
    -n 1000 \
    --customise=HLTrigger/Configuration/CustomConfigs.L1THLT

does not.

davidlange6 · 2016-12-13T09:48:58Z

so threading plus random numbers will mean things are not reproducible (I had asked @rekovic about this and thought I understood his tests were not threaded) - so if you go to one thread do you see the same issue?

…

On Dec 13, 2016, at 10:53 AM, Andrea Bocci ***@***.***> wrote: @davidlange6 I don't think the (lack of) reproducibility is tied to these changes; it is already there in CMSSW 8.0.24-patch1 . Running the DIGI+L1 step with cmsDriver.py test \ -s DIGI:pdigi,L1 \ --no_output \ --python_filename=test.py \ --processName HLTX \ --conditions 80X_mcRun2_asymptotic_2016_TrancheIV_v6_Tr4GT_v6 \ --era Run2_2016 \ --filein /store/relval/CMSSW_8_0_21/RelValTTbar_13/GEN-SIM-DIGI-RAW-HLTDEBUG/PU25ns_80X_mcRun2_asymptotic_2016_TrancheIV_v6_Tr4GT_v6-v1/10000/049B4664-8A98-E611-8F0C-0025905B8562.root \ -n 1000 \ --customise=HLTrigger/Configuration/CustomConfigs.L1THLT multiple times with 4 threads leads to different L1 results, but running only the L1 step with cmsDriver.py test \ -s L1 \ --no_output \ --python_filename=test.py \ --processName HLTX \ --conditions 80X_mcRun2_asymptotic_2016_TrancheIV_v6_Tr4GT_v6 \ --era Run2_2016 \ --filein /store/relval/CMSSW_8_0_21/RelValTTbar_13/GEN-SIM-DIGI-RAW-HLTDEBUG/PU25ns_80X_mcRun2_asymptotic_2016_TrancheIV_v6_Tr4GT_v6-v1/10000/049B4664-8A98-E611-8F0C-0025905B8562.root \ -n 1000 \ --customise=HLTrigger/Configuration/CustomConfigs.L1THLT does not. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

fwyzard · 2016-12-13T10:41:48Z

yes, if I run the DIGI+L1 job multiple times with a single thread I do get identical results.

davidlange6 · 2016-12-13T10:43:33Z

ok, so maybe I misunderstood @rekovic yesterday..seems all is working as expected.

…

On Dec 13, 2016, at 12:41 PM, Andrea Bocci ***@***.***> wrote: yes, if I run the DIGI+L1 job multiple times with a single thread I do get identical results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

fwyzard · 2016-12-13T10:44:54Z

Well, if the random number generator is per-module but global (as opposed to per-stream) it makes sense that running with multiple threads/streams can give different results across different jobs, since the order of execution of the events is not guaranteed, right ?
I guess one could have per-stream, per-module random number generators if reproducibility is a concern.

fwyzard · 2016-12-13T10:45:47Z

Scratch that - even if the generator were per-stream, the order of events across the streams is not guaranteed.

Martin-Grunewald · 2016-12-13T10:48:22Z

Reproducibility with MT is important.
Yes, I can switch off MT, but maybe MT has/reveals other bugs (eg, modules not really MT safe)

fwyzard · 2016-12-13T11:04:41Z

Reproducibility with MT is important.
Yes, I can switch off MT, but maybe MT has/reveals other bugs (eg, modules not really MT safe)

Unfortunately I think in this case reproducibility cannot be guaranteed - it's a similar issue to what we have with the HLT prescales.

At least, I do not have any bright ideas about this...

rekovic · 2016-12-13T11:31:51Z

@davidlange6
As mentioned to you on skype, irreproducibility has nothing to do with packing of the ExternalConditions per se (i.e. this PR). The irreproducibility was observed in some tests triggered by the code review comments of this PR. It is now concluded that the issue is the multithreading (MT).

Results of current unit tests of L1T (DIGI,L1T, DIGI,L1T,DIGI2RAW, DIGI,L1T,DIGI2RAW,RAW2DIGI are reproducible as they are not MT. At least MT is not turned on anywhere explicitely. Example unit-test:

test  -s DIGI:pdigi,L1,DIGI2RAW \
--conditions auto:run2_mc \
--datatier DIGI-RAW-HLTDEBUG \
 -n 1000 \
--era Run2_2016 \
--eventcontent RAW \
--filein /store/group/dpg_trigger/comm_trigger/TriggerStudiesGroup/STORM/GEN-SIM/CMSSW_8/06F2C3AC-8957-E611-9DDF-0025905B85D8.root \
--customise=L1Trigger/Configuration/customiseUtils.L1TGlobalDigisSummary \
--customise=L1Trigger/Configuration/customiseUtils.L1TAddInfoOutput

However, if L1THLT customisation is added, as in case of @fwyzard example above,

  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT

the irrecproducibility is observed. Apparently, HLT configuration turns on the MT.

davidlange6 · 2016-12-14T15:06:03Z

Hi @rekovic - that seems to be correct, yes. (hopefully anyone using this customize in batch/grid is aware and changes their config accordingly)

Put back packing of ExternalConditions in packer GTSetup and configur…

8a0fcd7

…e default to use simGtExtFakeStage2Digis. Fix L1REPACK:uGT configuration to use the same default.

cmsbuild added this to the Next CMSSW_9_0_X milestone Nov 29, 2016

cmsbuild added comparison-pending l1-pending operations-pending orp-pending pending-signatures tests-pending labels Nov 29, 2016

cmsbuild added tests-started and removed tests-pending labels Nov 29, 2016

rekovic mentioned this pull request Nov 29, 2016

pr80x L1T GT packer of ExternalConditions #16796

Merged

cmsbuild added l1-approved and removed l1-pending labels Nov 29, 2016

cmsbuild added tests-approved and removed tests-started labels Nov 29, 2016

cmsbuild added comparison-available and removed comparison-pending labels Nov 29, 2016

davidlange6 reviewed Nov 30, 2016

View reviewed changes

rekovic mentioned this pull request Dec 2, 2016

Packing of ExternalConditions in GT - and HLT cms-l1t-offline/cmssw#440

Open

L1REPACK:uGT use only the unpacked uGT inputs to re-emulate uGT. Good…

8b92951

… for either RAW DATA or RAW MC.

cmsbuild removed comparison-available l1-approved tests-approved labels Dec 9, 2016

cmsbuild added the tests-pending label Dec 9, 2016

cmsbuild added l1-approved and removed l1-pending labels Dec 9, 2016

cmsbuild added tests-started and removed tests-pending labels Dec 9, 2016

cmsbuild added tests-approved and removed tests-started labels Dec 9, 2016

cmsbuild added comparison-available and removed comparison-pending labels Dec 9, 2016

davidlange6 merged commit cddbf19 into cms-sw:CMSSW_9_0_X Dec 14, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pr90x L1T GT packer of ExternalConditions #16794

pr90x L1T GT packer of ExternalConditions #16794

rekovic commented Nov 29, 2016 •

edited

rekovic commented Nov 29, 2016

cmsbuild commented Nov 29, 2016 •

edited

cmsbuild commented Nov 29, 2016

rekovic commented Nov 29, 2016

cmsbuild commented Nov 29, 2016

cmsbuild commented Nov 29, 2016

cmsbuild commented Nov 29, 2016

davidlange6 Nov 30, 2016

rekovic Dec 2, 2016

cmsbuild commented Dec 9, 2016

rekovic commented Dec 9, 2016

rekovic commented Dec 9, 2016

cmsbuild commented Dec 9, 2016 •

edited

cmsbuild commented Dec 9, 2016

cmsbuild commented Dec 9, 2016

cmsbuild commented Dec 9, 2016

rekovic commented Dec 12, 2016

davidlange6 commented Dec 12, 2016

rekovic commented Dec 12, 2016

fwyzard commented Dec 13, 2016

davidlange6 commented Dec 13, 2016 via email

fwyzard commented Dec 13, 2016

davidlange6 commented Dec 13, 2016 via email

fwyzard commented Dec 13, 2016

fwyzard commented Dec 13, 2016

Martin-Grunewald commented Dec 13, 2016

fwyzard commented Dec 13, 2016

rekovic commented Dec 13, 2016

davidlange6 commented Dec 14, 2016

pr90x L1T GT packer of ExternalConditions #16794

pr90x L1T GT packer of ExternalConditions #16794

Conversation

rekovic commented Nov 29, 2016 • edited

rekovic commented Nov 29, 2016

cmsbuild commented Nov 29, 2016 • edited

cmsbuild commented Nov 29, 2016

rekovic commented Nov 29, 2016

cmsbuild commented Nov 29, 2016

cmsbuild commented Nov 29, 2016

cmsbuild commented Nov 29, 2016

davidlange6 Nov 30, 2016

Choose a reason for hiding this comment

rekovic Dec 2, 2016

Choose a reason for hiding this comment

cmsbuild commented Dec 9, 2016

rekovic commented Dec 9, 2016

rekovic commented Dec 9, 2016

cmsbuild commented Dec 9, 2016 • edited

cmsbuild commented Dec 9, 2016

cmsbuild commented Dec 9, 2016

cmsbuild commented Dec 9, 2016

rekovic commented Dec 12, 2016

davidlange6 commented Dec 12, 2016

rekovic commented Dec 12, 2016

fwyzard commented Dec 13, 2016

davidlange6 commented Dec 13, 2016 via email

fwyzard commented Dec 13, 2016

davidlange6 commented Dec 13, 2016 via email

fwyzard commented Dec 13, 2016

fwyzard commented Dec 13, 2016

Martin-Grunewald commented Dec 13, 2016

fwyzard commented Dec 13, 2016

rekovic commented Dec 13, 2016

davidlange6 commented Dec 14, 2016

rekovic commented Nov 29, 2016 •

edited

cmsbuild commented Nov 29, 2016 •

edited

cmsbuild commented Dec 9, 2016 •

edited