New Run3 pileup scenario: 10h fill, 2h leveling

[knollejo]

New pileup scenario for Run 3. Details about the expected LHC conditions and the effects that have been considered have been presented in the PPD meeting here. To build the scenario, we have simulated an LHC fill of 10h with an initial 2h leveling period. This has to be understood as an optimistic average of all 2022 fills which are planned to converge to 6h leveling periods at the end of 2022. The fraction of events with a given average pileup is weighted by the instantaneous luminosity.

We tested running cmsDriver with --pileup 2022_LHC_Simulation_10h_2h by hand and found no problems.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34460/23906

• This PR adds an extra 20KB to repository

[cmsbuild]

A new Pull Request was created by @knollejo (Joscha Knolle) for master.

It involves the following packages:

• Configuration/StandardSequences (operations)
• SimGeneral/MixingModule (simulation)

@civanch, @silviodonato, @mdhildreth, @cmsbuild, @qliphy, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@fabiocos, @makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @Martin-Grunewald, @lecriste, @mtosi, @ebrondol, @mmusich, @dgulhan, @slomeo this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

[mmusich]

@knollejo I was under the impression the target leveling for 2022 is more 6h:

Also are there plans to make this the default RelVal scenario ?
@rappoccio (I don't have Andreas gitHub tag)

[rappoccio]

Hi, @mmusich , all. Let's discuss at the PPD Coordination meeting tomorrow. We'll present the proposal from LUM and we can discuss if it is sufficient. We intend to make it default for relvals and Run 3 MC production.

[davidlange6]

This graphic is the end of the 2022 running goal for leveling not what is expected for the overall 2022 run. LPC suggests the average PU is 35 in 2022 (averaged over what is not totally clear)

…

On Jul 13, 2021, at 5:21 PM, Marco Musich ***@***.***> wrote: @knollejo I was under the impression the target leveling for 2022 is more 6h: Also are there plans to make this the default RelVal scenario ? @rappoccio (I don't have Andreas gitHub tag) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[cschwick]

The graphics shown by @mmusich is the best possible scenario towards the end of 2022 calculated by the LHC experts. It for sure does not represent how the fills in 2022 look like ON AVERAGE. The pileup distribution derived from our proposed fill has larger contributions at lower pileup which makes reweighing the MonteCarlo possible in a larger pileup range (as I have understood). I still think it is an optimistic scenario. If I am wrong we will have a fantastic high integrated lumi year in 2022 :-) Of course it is possible to adapt the distribution in any way if you wish.

[mmusich]

This graphic is the end of the 2022 running goal for leveling not what is expected for the overall 2022 run. LPC suggests the average PU is 35 in 2022 (averaged over what is not totally clear)

Thanks, this makes sense. It would be perhaps convenient to post the target PU profile plot directly in the gitHub thread for future reference.

[mmusich]

It for sure does not represent how the fills in 2022 look like ON AVERAGE.

I think the average makes sense for large MC productions, while for relval perhaps an optimistic luminosity (PU) (pessimistic in terms of the detector performance) scenario would be useful to test algorithm robustness in the "worst case scenario". I am not proposing that, though, as this is already vastly better than the current totally irrealistic relval profile.

[cschwick]

HI @davidlange6 , thanks for the LPC guess of the average pileup. The average pileup of the distribution we propose is by the way 36. So a pretty compatible guess from that perspective.

[cschwick]

And here comes the pileup distribution we propose:

The cut text of the legend should be:
...varying from 12.5% to 25% in 8h

The distribution has been generated with the tool on our webpage
https://cmslumipog.web.cern.ch/cgi-bin/pileupDistribution.py
It contains the simulation results of the LHC colleagues who provided us with the underlying data of their plots.

[VinInn]

why the plot above does not corresponds to this (slide 14)

and is instead very similar to the 16h fill? (slide 16 of https://indico.cern.ch/event/1045566/contributions/4392196/attachments/2261950/3839554/PileupGuessing.pdf)

[knollejo]

why the plot above does not corresponds to this (slide 14)
and is instead very similar to the 16h fill? (slide 16 of https://indico.cern.ch/event/1045566/contributions/4392196/attachments/2261950/3839554/PileupGuessing.pdf)

For the plots in the presentation, the events are weighted by luminosity. The plots in this thread, which correspond to the numbers in the pull request, just gives the number of events. That's why higher-pileup peaks are larger in the presentation than in this thread.

[VinInn]

Thanks, understood. I overlooked the scale on the left.
Still, what matters (for signal) is lumi: how this is managed?
(I mean if One looks to PU distribution for ttbar I would expect something close to the plot in the presentation)

[knollejo]

This is the proposed scenario but weighted with the luminosity. It is very similar to the distribution on slide 14 of that presentation.

[VinInn]

Ok. So this is the pileup distribution we should expect for signal events, isn't it?

[knollejo]

One should note that this (the scenario and the plots so far) does not include Poisson statistics for the number of pileup events (because that is taken care of by CMSSW). The lumi-weighted pileup distribution with Poisson statistics folded in, which is what we expect for signal events, is this:

[cschwick]

Not sure what you mean with "signal events", Vincenzo. This is the pileup distribution for all bunch crossings in CMS (before triggering or anything). And without Poisson as Joscha mentioned (and lumi weighted).

[VinInn]

That when CMSSW had to sample PU to add it to a "signal event" the distribution (before poisson?) should be lumi-weighed.

[VinInn]

@cmsbuild , please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34460/23973

• This PR adds an extra 24KB to repository

[cmsbuild]

Pull request #34460 was updated. @civanch, @silviodonato, @mdhildreth, @cmsbuild, @qliphy, @fabiocos, @davidlange6 can you please check and sign again.

[knollejo]

After continuing the discussion here on other channels to clear some confusion about what exactly is needed as an input, we have updated the scenario to have the fraction of events weighted by the instantaneous luminosity, which was not the case originally.

[civanch]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-129862/16989/summary.html
COMMIT: a970aa1
CMSSW: CMSSW_12_0_X_2021-07-19-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34460/16989/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-129862/11634.912_TTbar_14TeV+2021_DD4hepDB+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA

Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 2996268
• DQMHistoTests: Total failures: 7
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2996239
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: no differences found

[civanch]

+1

[qliphy]

+operations

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy, @perrotta (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1