Migrate Digi step to Tasks

[makortel]

PR description:

This PR migrates the Digi step and also the DataMixerPreMix_cff (as it is closely related to Digi step) to use Tasks in the configuration. This change should increase the available concurrency in the jobs running the Digi step.

PR validation:

Limited matrix runs.

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-26539/9426

• This PR adds an extra 28KB to repository

[cmsbuild]

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

Configuration/StandardSequences
RecoJets/Configuration
RecoMET/Configuration
SimCalorimetry/Configuration
SimMuon/Configuration
SimMuon/GEMDigitizer

@perrotta, @civanch, @mdhildreth, @cmsbuild, @franzoni, @kpedro88, @slava77, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@TaiSakuma, @gouskos, @felicepantaleo, @abbiendi, @rappoccio, @Martin-Grunewald, @seemasharmafnal, @mmarionncern, @battibass, @ahinzmann, @jhgoh, @jdolen, @HuguesBrun, @drkovalskyi, @dildick, @trocino, @GiacomoSguazzoni, @rovere, @VinInn, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @dgulhan, @clelange, @calderona, @mariadalfonso, @folguera this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[makortel]

@cmsbuild, please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/34349/console Started: 2019/04/25 21:07

[cmsbuild]

+1
Tested at: da984a0
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26539/34349/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26539/34349/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 33
• DQMHistoTests: Total histograms compared: 3211964
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3211759
• DQMHistoTests: Total skipped: 204
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
• Checked 137 log files, 14 edm output root files, 33 DQM output files

[kpedro88]

@makortel will this have an effect without changing the actual digi workflows to use unscheduled mode?

[makortel]

@kpedro88 Yes, it has effect. With "unscheduled mode" you probably refer to --runUnscheduled option to cmsDriver.py, right? The only effect of that option is an addition of convertToUnscneduled() customize function, which moves all EDProducers from Paths to Tasks (and associates those Tasks with the original Paths). Internally the framework has been unscheduled-only for a long time already.

The issue #26517 gives further pointers why --runUnscheduled can't be universally applied to jobs with DIGI step.

[kpedro88]

@makortel thanks for the clarification

[kpedro88]

+upgrade

[civanch]

+1

[perrotta]

+1

• Technical: migration from Sequences to Tasks is implemented as proposed in the PR description
• Jet and MET touched for reco
• Jenkins tests pass and show no differences

[fabiocos]

+operations

the update of the StandardSequences is coherent with the purpose of the PR

[fabiocos]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.