Handling Run-Dependent MC

[srimanob]

Impact of the new feature
WMAgent to handle Run-Dependent MC using a (to be) agreed method.

Is your feature request related to a problem? Please describe.
Nope

Describe the solution you'd like
Workflow injected from PPD will contain
runs_weights = [(315257, 11.11), (316082, 11.11), (316720, 11.11), (317527, 11.11), (320917, 11.11), (321414, 11.11), (321973, 11.11), (322492, 11.11), (324245, 11.12)]

when total LS is calculated from WMAgents, the firstLuminosityBlockForEachRun will then be assigned properly.

For example from above runs_weights, and workflows with total 2000 LS, we should get
315257: 0.1111 * 2000 = 222 => Assign run 315257 from 1 to 222
316082: 0.1111 * 2000 = 222 => Assign run 316082 from 223 to 444
316720: 0.1111 * 2000 = 222 => Assign run 316720 from 445 to 666
317527: 0.1111 * 2000 = 222 => Assign run 317527 from 667 to 888
320917: 0.1111 * 2000 = 222 => Assign run 320917 from 889 to 1110
321414: 0.1111 * 2000 = 222 => Assign run 321414 from 1111 to 1332
321973: 0.1111 * 2000 = 222 => Assign run 321973 from 1333 to 1554
322492: 0.1111 * 2000 = 222 => Assign run 322492 from 1555 to 1776
324245: 0.1112 * 2000 = 222 => Assign run 324245 from 1777 to 2000
(some rounding may need)

Then PoolSource needs firstLuminosityBlockForEachRun to be properly set, e.g.
process.source.firstLuminosityBlockForEachRun = cms.untracked.VLuminosityBlockID(*[cms.LuminosityBlockID(x,y) for x,y in ((315257, 1), (316082, 223), (316720, 445), (317527, 667), (320917, 889), (321414, 1111), (321973, 1333), (322492, 1555), (324245, 1777))]),

Describe alternatives you've considered
No at the moment

Additional context
It's unclear that in the workflow injected from PPD side, do we need to insert dummy of process.source.firstLuminosityBlockForEachRun as customization or not. Or WMA can add it automatically when it's trigger by some parameter, i.e. when runs_weights is enabled, or some comment of the workflow.

We may need to agree if runs_weights should be in percentage or luminosity as in Run-1, https://github.com/cms-sw/cmssw/blob/master/SimGeneral/Configuration/python/RunsAndWeights_Run2012_AB_C_D_oneRunPerEra.py

[srimanob]

Following the way of Run I, I've introduce weights file in
cms-sw/cmssw#31107

[amaltaro]

@srimanob just to confirm. This is the issue you were asking about in the CompOps meeting, right?

[srimanob]

Hi @amaltaro
right, thx.

[christopheralanwest]

What is the scheme to ensure that each MC sample has the same weighting of runs? In the currently proposed scheme which determines the run number based on the lumiblock, the IOVs will only be weighted evenly if the number of lumiblocks in the same is evenly divisible by the number of IOVs. Will this constraint be enforced somehow? Or are we expecting that the user manually perform the weighting themselves for each sample?

Is it certain that the weights file in cms-sw/cmssw#31107 is the appropriate place to configure the lumi-weighting? I'm not an expert, but it appears that supplying this weight file triggers the Run 1 mechanism. @Dr15Jones should be able to confirm or deny my suspicion.

From the perspective of AlCa, I would like to be able that request subsystem condition experts prepare tags containing run-dependent conditions. But before I can make this request, I need to know what IOV structure will be assumed. Can you guarantee that the IOV structure chosen by ECAL will be supported in the end?

[srimanob]

Hi @christopheralanwest
The PR I made is the guideline in how we can weight, I just want to make a starting point by following what we have in RunI. If we would like to change, we can also. This is mostly depended on PPD and Computing to agree on how to pass these numbers and interpret in computing.

For the IOV, I think we should start from ECAL proposed IOVs and ask others DPG's opinion. Could this discussion be organized in the Alca meeting?

[vkuznet]

@amaltaro , could you please provide technical details where these changes in DMWM stack should go. So far I do not see anything called firstLuminosityBlockForEachRun in WMCore codebase, and PoolSource only appears in configuration files.

[amaltaro]

@vkuznet this issue hasn't been considered for this quarter, so I suggest to not even invest time looking into it right now.

However, given that you ping'ed on this and that there has not been any activity on it for the last 2 years...
@srimanob Phat, has there been any changes on this topic? any changes to the requirements and/or anything relevant to the WM system? Thanks

[vkuznet]

@amaltaro , thanks for clarification, I was following this document looking issues with High priority. Since I was not present on meeting which decided which priorities will go into Q1 I asked my question directly on this ticket. But of course, if it is not listed for this quarter I'll not spend time on it.