Fastsim New CFG structure (preparations for premixing)

[lveldere]

This PR superseeds #8414

The FastSim cfg structure was altered,
to better resemble the standard cfg structure.
This will increase maintainability on long term,
and on short term it is a necessary step in the preparation for premixing.

Changes

• cmsDriver: the FASTSIM, deprecated since a long time, is removed from ConfigBuilder
• cmsDriver & runTheMatrix: FastSim has now the following steps in cmsDriver GEN,SIM,RECOBEFMIX,DIGI,L1,L1Reco,RECO,EI,HLT:@frozen25ns,VALIDATION all FastSim runTheMatrix workflows are migrated accordingly
• HLT: L1 and L1Reco step are moved outside the HLT step and are now separate steps in FastSim
• HLT: clones of collections are replaced with EDAliases
• HLT DQM / Validation: to avoid crahses in runTheMatrix workflows the inputlabel for SimVertices consumed by HLTriggerOffline/Btag is made configurable
• L1/DIGI: packing and unpacking of calorimeter digis is removed and in order not to change any of the default InputTags, EDAliases are used to mimic the presense of post pack/unpack digi collections
• FASTSIM: unnecessary dependence of FamosProducer on BeamSpot is removed
• unit tests: outdated fastsim unit tests are removed from addOnTest.py
• Mixing: reco track accumulator includes MVA values in mixing
• Mixing: firstStepPrimaryVertices including PU tracks is emulated by reconstructing it from all
  generalTracks since after mixing no initialStepTrack collection is available.
• MIxing: DataMixingModule can be configured to use RecoTrackAccumulator to mix tracker information

TODO after this PR

• HLT cfg needs a more permanent solution which does not require to import the menu inside a wrapper in FastSimulation/Configuration (see e.g. FastSimulation/Configuration/python/HLT_GRun_Famos_cff.py)
  discussion initiated in Fastsim newcfg premix march19 #8414
• HLT rare segfualt in L2muons
• tracking Perhaps a cleaner solution is needed on long term to delete the modules from full iteratitve tracking from the Reconstrion cfg
• FASTSIM Remove all FastSim cfg and other code that has become obsolete

Tests

• runTheMatrix.py -l 5.1,135.1,135.8,25400
• wf 135.1 with 1000 events, sanity checks for distributions of main physics objects
• detailed comparison of old dumped cfg vs new dumped cfg

[lveldere]

please test

[cmsbuild]

The tests are being triggered in jenkins.

[cmsbuild]

A new Pull Request was created by @lveldere for CMSSW_7_5_X.

Fastsim New CFG structure (preparations for premixing)

It involves the following packages:

Configuration/Applications
Configuration/PyReleaseValidation
FastSimulation/CaloRecHitsProducer
FastSimulation/Configuration
FastSimulation/Event
FastSimulation/EventProducer
FastSimulation/HighLevelTrigger
FastSimulation/Tracking
HLTriggerOffline/Btag
SLHCUpgradeSimulations/Configuration
SimGeneral/DataMixingModule
Utilities/ReleaseScripts

@civanch, @Dr15Jones, @lveldere, @danduggan, @boudoul, @ssekmen, @mdhildreth, @cmsbuild, @srimanob, @nclopezo, @deguio, @franzoni, @ktf, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @imarches, @acaudron, @wmtan, @wddgit, @Martin-Grunewald, @ferencek, @matt-komm, @pvmulder this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@nclopezo you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

[franzoni]

+1
I am ok with the logic of the chances to relval_steps
pushing this forward, upon request of Lukas @lveldere .
Have had not time to dig deep into the changes to Configuration/Applications/python/ConfigBuilder.py . Note: it's possible that my +1 will be received as a signature for operations, too.

@lveldere : since there's now a digi step, would it make sense adding the DQM (besides VALIDATION) ? Can you give that thought / try (in perspective, not for this PR)

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_5_X IBs unless changes or unless it breaks tests. This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @nclopezo, @ktf, @smuzaffar

[lveldere]

@franzoni

thanks a lot

adding the DQM step is one of the things I have in mind for the future.

[cmsbuild]

+1
Tested at: ca801ba
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8455/4144/summary.html