New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
EMTF 2017 PR #4 - 2017 CondDB and O2O #19928
Conversation
A new Pull Request was created by @abrinke1 for master. It involves the following packages: CondCore/L1TPlugins @ghellwig, @arunhep, @cerminar, @cmsbuild, @rekovic, @franzoni, @ggovi, @davidlange6, @mulhearn, @lpernie can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
please test |
The tests are being triggered in jenkins. |
-1 Tested at: c7ca155 The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: You can see the results of the tests here: I found follow errors while testing this PR Failed tests: RelVals AddOn
When I ran the RelVals I found an error in the following worklfows: runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step2_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log135.4 step1 runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log136.7611 step2 runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016+HARVESTDR2_REMINIAOD_data2016/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016+HARVESTDR2_REMINIAOD_data2016.log1306.0 step2 runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step2_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log1330.0 step2 runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step2_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log10042.0 step2 runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step2_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log25202.0 step2 runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step2_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log10024.0 step2 runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step2_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log10824.0 step2 runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+ALCAFull_2018+HARVESTFull_2018/step2_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+ALCAFull_2018+HARVESTFull_2018.log20034.0 step2 runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17/step2_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17.log20434.0 step2 runTheMatrix-results/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19/step2_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19.log
I found errors in the following addon tests: cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_Fake2 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016 --fileout file:RelVal_Raw_Fake2_DATA.root --filein /store/data/Run2015D/MuonEG/RAW/v1/000/256/677/00000/80950A90-745D-E511-92FD-02163E011C5D.root : FAILED - time: date Thu Jul 27 00:49:31 2017-date Thu Jul 27 00:48:54 2017 s - exit: 18688 The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: |
Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped) |
@arunhep @davidlange6 @kkotov @rekovic |
@abrinke1 , The error you see here: is generated in this line: and results from the first literal in "FirmwareVersion" being capitalized despite it is lower-case in the config: |
@kkotov Thanks! I hope this commit will fix it - let me know if you see anything fishy. |
…to EMTF_PR_2017_93X_part4
Just pushed a little more clean-up ... @arunhep @davidlange6 can you trigger the cmsbuild tests again? Thanks, |
@cmsbuild please test |
The tests are being triggered in jenkins. |
-1 Tested at: 34dc38a The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: You can see the results of the tests here: I found follow errors while testing this PR Failed tests: RelVals AddOn
When I ran the RelVals I found an error in the following worklfows: runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step2_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log136.731 step2 runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step2_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log4.22 step2 runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log140.53 step2 runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log136.7611 step2 runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016+HARVESTDR2_REMINIAOD_data2016/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016+HARVESTDR2_REMINIAOD_data2016.log1000.0 step2 runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log1001.0 step2 runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log1003.0 step2 runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log10024.0 step1 runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step1_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log10042.0 step1 runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step1_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log10824.0 step1 runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+ALCAFull_2018+HARVESTFull_2018/step1_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+ALCAFull_2018+HARVESTFull_2018.log20034.0 step1 runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17/step1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17.log20434.0 step1 runTheMatrix-results/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19/step1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19.log
I found errors in the following addon tests: cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_Fake2 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016 --fileout file:RelVal_Raw_Fake2_DATA.root --filein /store/data/Run2015D/MuonEG/RAW/v1/000/256/677/00000/80950A90-745D-E511-92FD-02163E011C5D.root : FAILED - time: date Thu Jul 27 18:15:20 2017-date Thu Jul 27 18:14:45 2017 s - exit: 16640 The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: |
Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped) |
Hi all, Turns out the previous bug @kkotov was not the only one - or maybe not even the bug causing the crash in the first place. @rekovic @arunhep @davidlange6 @ALL : is there a way I can get more detailed error messages, or re-run the failing sequence myself? Thanks much, |
@abrinke1 To run the workflow 136.731 locally, execut |
@abrinke1 |
@abrinke1 , I am pretty ignorant for what concerns the "plugin-poisoned" concept (although, I could guess the idea behind). So far I was simply removing these files from the lib/ directory in my working area and launching the jobs again. Doing so I never observed any problems, but I could not see why these files are appearing in a first place in a rather trivial contexts (not to mention that I have a strong feeling this was not always a reproducible behavior as setting up a new working directory helped me to get around these errors sometimes). While fixing this problem in a local environment is easy, I have no idea on what to do during the unit tests. I'd give it another try hoping that it won't show up, but I hope someone more knowledgeable can comment here. |
@rekovic , btw. I'd bye you a beer if you show me the hint how to run gdb in lxplus environment. I once spent non-negligible amount of time to see how to make gdb to resolve shared object dependencies. It happens automatically everywhere else, but not on lxplus. |
@rekovic @arunhep @davidlange6 @ALL , The failing sequence that @kkotov was mentioning is below. Like he says, I have no idea why a poisoned plugin would appear or not appear - the code compiles fine. Does anyone have suggestions on what's going on, or how to fix it? Thanks much, runTheMatrix.py -l 136.731 cd 136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2 cmsDriver.py step2 --datatier FEVTDEBUGHLT --conditions auto:run2_hlt_relval -s L1REPACK:Full,HLT:@relval2016 --process reHLT --data --era Run2_2016 --eventcontent FEVTDEBUGHLT -n 100 --filein filelist:step1_dasquery.log --lumiToProcess step1_lumiRanges.log --fileout file:step2.root > step2_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log 2 Step: L1REPACK Spec: ['Full'] |
are the condformats that this PR removes part of the global tag?
… On Jul 31, 2017, at 11:28 AM, abrinke1 ***@***.***> wrote:
@rekovic @arunhep @davidlange6 @ALL ,
The failing sequence that @kkotov was mentioning is below. Like he says, I have no idea why a poisoned plugin would appear or not appear - the code compiles fine. Does anyone have suggestions on what's going on, or how to fix it?
Thanks much,
Andrew
runTheMatrix.py -l 136.731
ERROR executing [description of process]
cd 136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2
cmsDriver.py step2 --datatier FEVTDEBUGHLT --conditions auto:run2_hlt_relval -s ***@***.*** --process reHLT --data --era Run2_2016 --eventcontent FEVTDEBUGHLT -n 100 --filein filelist:step1_dasquery.log --lumiToProcess step1_lumiRanges.log --fileout file:step2.root > step2_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log 2
Step: L1REPACK Spec: ['Full']
L1T INFO: L1REPACK:Full (intended for 2016 data) will unpack all L1T inputs, re-L1T INFO: L1REPACK:Full (intended for 2016 data) will unpack all L1T inputs, re-emulated (Stage-2), and pack uGT, uGMT, and Calo Stage-2 output.
----- Begin Fatal Exception 31-Jul-2017 11:20:13 CEST-----------------------
An exception of category 'PluginLibraryLoadError' occurred while
[0] Constructing the EventProcessor
[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'
Exception Message:
unable to load /afs/cern.ch/user/a/abrinke1/CentralPR/CMSSW_9_3_X_2017-07-25-2300/lib/slc6_amd64_gcc630/poisoned/plugin-poisoned-CondCoreL1TPlugins.so because /afs/cern.ch/user/a/abrinke1/CentralPR/CMSSW_9_3_X_2017-07-25-2300/lib/slc6_amd64_gcc630/poisoned/plugin-poisoned-CondCoreL1TPlugins.so: cannot open shared object file: No such file or directory
----- End Fatal Exception -------------------------------------------------
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
Hmm ... good question. @kkotov would have to answer that one - but he's traveling at the moment. However, I re-added the original CondFormats/DataRecord/src/L1TMuonEndcapParamsRcd.cc and CondFormats/DataRecord/interface/L1TMuonEndcapParamsRcd.h files, re-compiled, and re-ran - but that didn't change anything. So it doesn't seem to depend on the fact that those files are were missing. |
given the plugin declariations that are removed (nice - "C" goes to "c" 👎 - that will have to be changed unfortunately) - that is not going to change anything.. |
For now, moving on to PR #20006 for DQM, which is not dependent on any O2O / ConfFormats payloads. Will come back to this, probably with a new PR. |
Can this PR be closed if not relevant now? |
Sorry, thought I had already closed it - now closed. |
Part 4 of set of PRs following PR #19743
(Skipped part originally labeled as number 3)
Updated CondDB and O2O functionality from @kkotov to support 2017 pT assignment and primitive conversion in emulator.