GTupdates_for_910pre3_L1O2OTags_HCAL2018Geo_PCL_TkAl_AlignableThresholds #18269

arunhep · 2017-04-07T16:45:12Z

Summary of changes in Global Tags

RunI data

RunI HLT processing : 91X_dataRun2_HLT_frozen_v3 as 91X_dataRun2_HLT_frozen_v1 with the following changes:
- addition of L1 O2O caloParam and Muon Param tags https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2878.html
RunI Offline processing : 91X_dataRun2_v3 as 91X_dataRun2_v1 with the following changes:
- addition of L1 O2O caloParam and Muon Param tags https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2878.html
- addition of new record with PCL alignable configurable thresholds for tracker alignment

RunII data

RunII HLTHI processing : 91X_dataRun2_HLTHI_frozen_v3 as 91X_dataRun2_HLTHI_frozen_v1 with the following changes:
- addition of L1 O2O caloParam and Muon Param tags https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2878.html
RunII HLT processing : 91X_dataRun2_HLT_frozen_v3 as 91X_dataRun2_HLT_frozen_v1 with the following changes:
- addition of L1 O2O caloParam and Muon Param tags https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2878.html
RunII HLT relval processing : 91X_dataRun2_HLT_relval_v3 as 91X_dataRun2_HLT_relval_v1 with the following changes:
- addition of L1 O2O caloParam and Muon Param tags https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2878.html
RunII Offline processing : 91X_dataRun2_v3 as 91X_dataRun2_v1 with the following changes:
- addition of L1 O2O caloParam and Muon Param tags https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2878.html
- addition of new record with PCL alignable configurable thresholds for tracker alignment
RunII Offline relval processing : 91X_dataRun2_relval_v3 as 91X_dataRun2_relval_v1 with the following changes:
- addition of L1 O2O caloParam and Muon Param tags https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2878.html
- addition of new record with PCL alignable configurable thresholds for tracker alignment
RunII Prompt processing : 91X_dataRun2_PromptLike_v3 as 91X_dataRun2_PromptLike_v1 with the following changes:
- addition of L1 O2O caloParam and Muon Param tags https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2878.html
- addition of new record with PCL alignable configurable thresholds for tracker alignment

Upgrade

PhaseI 2017 cosmics peak scenario : 91X_upgrade2017cosmics_realistic_peak_v3 as 91X_upgrade2017cosmics_realistic_peak_v2 with the following changes:
- updates of Pixel local reco conditions : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2885.html
PhaseI 2017 cosmics scenario : 91X_upgrade2017cosmics_realistic_deco_v3 as 91X_upgrade2017cosmics_realistic_deco_v2 with the following changes:
- updates of Pixel local reco conditions : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2885.html
PhaseI 2017 design scenario : 91X_upgrade2017_design_IdealBS_v3 as 91X_upgrade2017_design_IdealBS_v2 with the following changes:
- updates of Pixel local reco conditions : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2885.html
PhaseI 2017 realistic scenario : 91X_upgrade2017_realistic_v3 as 91X_upgrade2017_realistic_v2 with the following changes:
- updates of Pixel local reco conditions : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2885.html
PhaseI 2018 cosmics scenario : 91X_upgrade2018cosmics_realistic_deco_v3 as 91X_upgrade2018cosmics_realistic_deco_v2 with the following changes:
- updates of Pixel local reco conditions : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2885.html
- Update of HCAL geometry : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2859.html
PhaseI 2018 design scenario : 91X_upgrade2018_design_IdealBS_v3 as 91X_upgrade2018_design_IdealBS_v2 with the following changes:
- Update of HCAL geometry : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2859.html
- updates of Pixel local reco conditions : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2885.html
PhaseI 2018 realistic scenario : 91X_upgrade2018_realistic_v3 as 91X_upgrade2018_realistic_v2 with the following changes:
- updates of Pixel local reco conditions : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2885.html
- Update of HCAL geometry : https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2859.html

cmsbuild · 2017-04-07T16:45:31Z

A new Pull Request was created by @arunhep (Arun Kumar) for master.

It involves the following packages:

Configuration/AlCa

@ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @mmusich, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @ghellwig, @tocheng this is something you requested to watch as well.
@Muzaffar, @davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

arunhep · 2017-04-07T16:46:28Z

please test

cmsbuild · 2017-04-07T16:46:48Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19015/console Started: 2017/04/07 18:47

arunhep · 2017-04-07T16:49:33Z

@veszpv @mmusich @kkotov @ianna @abdoulline FYI

mmusich · 2017-04-07T17:06:47Z

@arunhep @franzoni this will not work.

I am expecting failure of wf 1001.0, due to a duplication of ESSource for the AlignPCLThresholdsRcd record.
You'll need first to remove (as I explicitly mentioned in private message and as well here ) the additional ESSource I added in two files in order for the tests to pass:

Alignment/CommonAlignmentProducer/python/ALCARECOPromptCalibProdSiPixelAli_cff.py
Alignment/CommonAlignmentProducer/python/AlcaSiPixelAliHarvester_cff.py

You are welcome to cherry-pick this commit : mmusich@d87f994

…lignPCLThresholdsRcd

cmsbuild · 2017-04-07T17:45:53Z

Pull request #18269 was updated. @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @mmusich, @davidlange6 can you please check and sign again.

arunhep · 2017-04-07T17:49:33Z

@cmsbuild please abort

mmusich · 2017-04-07T17:50:50Z

@arunhep thanks!

arunhep · 2017-04-07T17:59:14Z

@cmsbuild please abort

cmsbuild · 2017-04-07T18:14:52Z

-1

Tested at: ead3bfe

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18269/19015/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following worklfows:
4.22 step2

runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py RelVal -s L1REPACK:GCTGT --data --scenario=pp -n 10 --conditions auto:run2_hlt_Fake1 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --fileout file:RelVal_Raw_Fake1_DATA.root --filein /store/data/Run2015D/MuonEG/RAW/v1/000/256/677/00000/80950A90-745D-E511-92FD-02163E011C5D.root : FAILED - time: date Fri Apr 7 20:04:31 2017-date Fri Apr 7 20:01:33 2017 s - exit: 16640

cmsbuild · 2017-04-07T18:14:54Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

arunhep · 2017-04-07T20:41:31Z

please test

cmsbuild · 2017-04-07T20:41:48Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19017/console Started: 2017/04/07 22:42

cmsbuild · 2017-04-07T22:25:47Z

-1

Tested at: fe5238d

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18269/19017/summary.html

I found follow errors while testing this PR

Failed tests: AddOn

AddOn:

I found errors in the following addon tests:

cmsDriver.py RelVal -s L1REPACK:GCTGT --data --scenario=pp -n 10 --conditions auto:run2_hlt_Fake1 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --fileout file:RelVal_Raw_Fake1_DATA.root --filein /store/data/Run2015D/MuonEG/RAW/v1/000/256/677/00000/80950A90-745D-E511-92FD-02163E011C5D.root : FAILED - time: date Sat Apr 8 00:09:12 2017-date Sat Apr 8 00:05:43 2017 s - exit: 16640

cmsbuild · 2017-04-07T22:25:49Z

Comparison job queued.

cmsbuild · 2017-04-10T14:27:35Z

Pull request #18269 was updated. @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @mmusich, @davidlange6 can you please check and sign again.

arunhep · 2017-04-10T14:35:07Z

please test

cmsbuild · 2017-04-10T14:35:24Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19052/console Started: 2017/04/10 16:35

cmsbuild · 2017-04-10T16:19:10Z

+1
Tested at: 43b2401
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18269/19052/summary.html

cmsbuild · 2017-04-10T16:19:12Z

Comparison job queued.

cmsbuild · 2017-04-10T17:51:27Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18269/19052/summary.html

Comparison Summary:

You potentially added 45 lines to the logs
Reco comparison results: 3458 differences found in the comparisons
DQMHistoTests: Total files compared: 23
DQMHistoTests: Total histograms compared: 1917243
DQMHistoTests: Total failures: 9558
DQMHistoTests: Total nulls: 6
DQMHistoTests: Total successes: 1907506
DQMHistoTests: Total skipped: 173
DQMHistoTests: Total Missing objects: 0
Checked 94 log files, 14 edm output root files, 23 DQM output files

arunhep · 2017-04-10T20:03:15Z

@mmusich @dimattia @ianna @bsunanda
can you please share your thoughts to the comment by @franzoni #18269 (comment)
we want to report it tomorrow in orp.

mmusich · 2017-04-11T06:35:20Z

@arunhep
in relation to your enquiry in comments #18269 (comment) and #18269 (comment) (i.e. the tiny change in the SiStrip DIGIs spectra)
from quick look to the DQM code, those histograms shall be filled here.

The configuration of the SiStripMonitorDigi module is defined here, where I don't see any explicit dependence on Pixel conditions; @fioriNTU to confirm.
Incidentally the module level plots should be switched-off by default in this view, so this is the lowest hierarchy level histogram for DIGIs.

Also the DIGI collection validation plots, seem to point to a genuine change in the Strip DIGIs.

Now, at contrast with the description you provided, for the PhaseI 2017 realistic scenario Global Tag there is at least one change involving the digitization step (namely via SiPixelLorentzAngleSimRcd that is consumed in the pixel digitizer) see GT diff. I am wondering if we might be witnessing irreproducibilities inherent from random number generation processes stemming from different conditions input. I am adding @boudoul as ad-interim Strip simulation contact.

franzoni · 2017-04-11T06:57:03Z

Thanks @mmusich,
we navigated the code yesterday and reached the same conclusions on SiStripMonitorDigi.cc .

from random number generation processes stemming from different conditions input

How far would the irreproducibility triggered by a change in digi conditions propagate (I have a limited understanding of the pseudo-random number generation ) ? Your hypothesis is supported by small perturbations to the ECAL digi occupancy , though no analogous changes are found in the HcalDigiTask.

abdoulline · 2017-04-11T06:58:28Z

From HCAL side with the update of the HCALParameters_Geometry_91YV3 tag
we just expect to fix the crashes in L1 emulator for 2018 workflows, as Yana has correctly described
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2859/1/1.html

mmusich · 2017-04-11T09:09:21Z

we navigated the code yesterday and reached the same conclusions on SiStripMonitorDigi.cc .

Nice!

How far would the irreproducibility triggered by a change in digi conditions propagate (I have a limited understanding of the pseudo-random number generation ) ?

Not being a digitization expert I cannot judge and defer to those more savvy than I am in this respect.
I can claim - though - that re-running exactly the same setup for wf 10024 but reverting the change in SiPixelLorentzAngleSimRcd in step3 via:

process.GlobalTag = GlobalTag(process.GlobalTag, '91X_upgrade2017_realistic_v3', '')
process.GlobalTag.toGet.extend(
    cms.VPSet(cms.PSet(connect= cms.string("frontier://FrontierProd/CMS_CONDITIONS"),
                       record = cms.string("SiPixelLorentzAngleSimRcd"),
                       tag = cms.string("SiPixelLorentzAngleSim_phase1_mc_v1")
                       )
              )
    )

I am able to restore a bitwise identity with the previous spectra.

franzoni · 2017-04-11T09:39:03Z

hello @mmusich
thanks, we have josbs running exactly what test, running with 91X_upgrade2017_realistic_v3_PIXEL_RECO_UP_ONLY.

We've also found that the DIGI step of the workflow has a single instance of the RandomNumberGeneratorService , which supports your original conjecture.


> runTheMatrix.py -l 10024 --command=' --dump_python '
> grep  RandomNumberGeneratorService TTbar_13TeV_TuneCUETP8M1_cfi_GEN_SIM.py
process.RandomNumberGeneratorService = cms.Service("RandomNumberGeneratorService",

Thanks for the help, this PR is good to go for AlCa.

franzoni · 2017-04-11T09:41:25Z

+1

cmsbuild · 2017-04-11T09:41:46Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @Muzaffar, @davidlange6, @smuzaffar

davidlange6 · 2017-04-11T19:49:20Z

+1

Upgrade 2017/2018 and data GTs updated

ead3bfe

cmsbuild added this to the Next CMSSW_9_1_X milestone Apr 7, 2017

cmsbuild added alca-pending comparison-pending orp-pending pending-signatures tests-pending labels Apr 7, 2017

cmsbuild added tests-started and removed tests-pending labels Apr 7, 2017

Remove Un-needed ESSources in Alignment/CommonAlignmentProducer for A…

fe5238d

…lignPCLThresholdsRcd

cmsbuild added tests-pending and removed tests-started labels Apr 7, 2017

cmsbuild added tests-started and removed tests-pending labels Apr 7, 2017

cmsbuild removed the tests-started label Apr 7, 2017

cmsbuild added comparison-pending tests-pending labels Apr 10, 2017

cmsbuild added tests-started and removed tests-pending labels Apr 10, 2017

cmsbuild added tests-approved and removed tests-started labels Apr 10, 2017

cmsbuild added comparison-available and removed comparison-pending labels Apr 10, 2017

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Apr 11, 2017

cmsbuild added orp-approved and removed orp-pending labels Apr 11, 2017

cmsbuild merged commit e820ed5 into cms-sw:master Apr 11, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GTupdates_for_910pre3_L1O2OTags_HCAL2018Geo_PCL_TkAl_AlignableThresholds #18269

GTupdates_for_910pre3_L1O2OTags_HCAL2018Geo_PCL_TkAl_AlignableThresholds #18269

arunhep commented Apr 7, 2017 •

edited

cmsbuild commented Apr 7, 2017

arunhep commented Apr 7, 2017

cmsbuild commented Apr 7, 2017 •

edited

arunhep commented Apr 7, 2017

mmusich commented Apr 7, 2017 •

edited

cmsbuild commented Apr 7, 2017

arunhep commented Apr 7, 2017 •

edited

mmusich commented Apr 7, 2017

arunhep commented Apr 7, 2017

cmsbuild commented Apr 7, 2017

cmsbuild commented Apr 7, 2017

arunhep commented Apr 7, 2017

cmsbuild commented Apr 7, 2017 •

edited

cmsbuild commented Apr 7, 2017

cmsbuild commented Apr 7, 2017

cmsbuild commented Apr 10, 2017

arunhep commented Apr 10, 2017

cmsbuild commented Apr 10, 2017 •

edited

cmsbuild commented Apr 10, 2017

cmsbuild commented Apr 10, 2017

cmsbuild commented Apr 10, 2017

arunhep commented Apr 10, 2017

mmusich commented Apr 11, 2017

franzoni commented Apr 11, 2017

abdoulline commented Apr 11, 2017

mmusich commented Apr 11, 2017

franzoni commented Apr 11, 2017

franzoni commented Apr 11, 2017

cmsbuild commented Apr 11, 2017

davidlange6 commented Apr 11, 2017

GTupdates_for_910pre3_L1O2OTags_HCAL2018Geo_PCL_TkAl_AlignableThresholds #18269

GTupdates_for_910pre3_L1O2OTags_HCAL2018Geo_PCL_TkAl_AlignableThresholds #18269

Conversation

arunhep commented Apr 7, 2017 • edited

Summary of changes in Global Tags

RunI data

RunII data

Upgrade

cmsbuild commented Apr 7, 2017

arunhep commented Apr 7, 2017

cmsbuild commented Apr 7, 2017 • edited

arunhep commented Apr 7, 2017

mmusich commented Apr 7, 2017 • edited

cmsbuild commented Apr 7, 2017

arunhep commented Apr 7, 2017 • edited

mmusich commented Apr 7, 2017

arunhep commented Apr 7, 2017

cmsbuild commented Apr 7, 2017

cmsbuild commented Apr 7, 2017

arunhep commented Apr 7, 2017

cmsbuild commented Apr 7, 2017 • edited

cmsbuild commented Apr 7, 2017

cmsbuild commented Apr 7, 2017

cmsbuild commented Apr 10, 2017

arunhep commented Apr 10, 2017

cmsbuild commented Apr 10, 2017 • edited

cmsbuild commented Apr 10, 2017

cmsbuild commented Apr 10, 2017

cmsbuild commented Apr 10, 2017

arunhep commented Apr 10, 2017

mmusich commented Apr 11, 2017

franzoni commented Apr 11, 2017

abdoulline commented Apr 11, 2017

mmusich commented Apr 11, 2017

franzoni commented Apr 11, 2017

franzoni commented Apr 11, 2017

cmsbuild commented Apr 11, 2017

davidlange6 commented Apr 11, 2017

arunhep commented Apr 7, 2017 •

edited

cmsbuild commented Apr 7, 2017 •

edited

mmusich commented Apr 7, 2017 •

edited

arunhep commented Apr 7, 2017 •

edited

cmsbuild commented Apr 7, 2017 •

edited

cmsbuild commented Apr 10, 2017 •

edited