Develop a Working Recipe for Stage 2 (and Stage 1) #101
Comments
|
Phew... full chain, into L1TNtuple. Needs cleanup, but working. |
|
@apana want to take it from here? |
mulhearn
changed the title
|
Current legacy inputs to the 2016 Muon trigger Emulator: Barrel:
End Cap:
Overlap:
|
|
The Overlap has more inputs:
|
|
Thanks! |
|
Noting some questions for later:
|
|
Completed a successful cleanup of SimL1Emulation. Now it properly uses Eras to smoothly transition between the Legacy/2015/2016 trigger configurations. Each major system (Muon/Calo/Global) has a simDigi_cff.py config which handles the entire subsystem. Conditions are loaded via hackConditions.py for each major system, to load any needed conditions not yet in GT for that era. Here's some sample debug output showing the various configurations of Emulators:
|
|
Cleanup of RawToDigi and DigiToRaw, nearly complete. Using standard sequences for Stage-1, here is output of emulation compared to packed-then-unpacked version. from emulation directly: packed-unpacked: |
|
Using standard sequences, new GMT muon unpacker seems to be working as well. from emulator: after packing/unpacking: |
|
However (1) Stage-2 Calo unpacker (or packer) (or both) not yet working in standard sequence. Also, something is odd with the rawDataCollector, so for now I am specifying the records directly, e.g. but clearly this needs a fix too. UPDATE: This one fixed thanks to Jim noticing missing rawDataCollector config changes during migration of our customizations to local L1TDigiToRaw_cff. |
|
Error message when unpacking Stage-2: This one is likely OK, as this FED is not filled by packer (only available from data). We will have a mode to quiet these errors but ignoring for now is fine. Jim investigating the empty caloStage2Digis collections after unpacking. |
|
Noting for later fix that the simDigis names I picked are inconsistent with the unpacker/packer digis names. I'll make them consistent later... Fixed, our digis are now called: |
|
Missing calo objects in L1Trigger/L1TCommon/test/runStandardSequences.py were due to out of date raw format definition in calo stage 2 digi2raw. I have pushed a fix for this to l1t-dev-recipe-CMSSW_7_6_0 |
|
Super! |
|
Almost there... L1Emulation, RawToDigi, DigiToRaw are now all working in the standard sequences. It is all Era driven, for Legacy/Stage1/Stage2 (though only Gmt and Layer2 are included in DigiToRaw for Stage 2 at the moment). I've added a config for ReEmulation of L1T from RAW, but it requires more debugging. Once working, its probably best to recast it as a customization, as expected for use in HLT. |
|
For Calo sequence, L1T re-emulation from RAW is working: Looking into muons, and need to quiet some errors. |
|
Muons are fine too: |
|
I just confirmed that there have been updates to Stage-2 calo emulator since #100. I simply need to update the integration branches and the dev-recipe branch to include these. Will do this later tonight... |
mulhearn
changed the title
|
Leaving this one open until we see some decent results from Mario and Len using the recipe... |
|
Some more issues to consider (expanding scope of issue to include Stage-1):
|
mulhearn
changed the title
|
@gennai the example code has been updated to show how to access all physics quantities from ntuple. Apologies to all that a late night commit error left recipe out of sync with the branch... now fixed. |
|
I've added a fictitious muon quality adjuster to the chain which puts the quality flags from each TF on a consistent setting (for now only 0=bad 1=good) so these may now be selected consistently in ntuple. Obviously more turning will be needed but this already knocks rate down to something comparable with stage1 muon rate. |
|
@jimbrooke et all are investigating incorrect phi distribution for Taus, as well as comparison of sums from latest recipe to what was presented at TSG. Remaining calo objects all look in reasonable agreement with what was shown. |
|
Latests update to the recipe branch includes fixes for Tau phi distribution and new electron isolation LUT. |
|
Latest update includes new Tau LUTs. |
|
It turns out adding v1 of HLT seed filter dramatically increases compile time, so while that task is complete, it is in a separate branch: |
|
thomas notifies me of an additional commit for new stage 2 EG LUTs: |
|
Hi Mike, Best, Il giorno 01 feb 2016, alle ore 15:40, mulhearn <notifications@github.commailto:notifications@github.com> ha scritto: thomas notifies me of an additional commit for new stage 2 EG LUTs: — |
|
@gennai it is a serious hassle to keep two versions going... (Reason: technically the answer to your question is "no", the commits all need to be rebased to the different version, and then there are conflicts that have to be manually resolved... not to mention human error due to keeping track of what has been ported where...) So unless there is a compelling reason not to, its best to advance together... note also you'll never have 1x1 TPs in 76x... |
|
Sounds good to me. Best, |
|
That commit is only four hours old! I'll merge it now... |
|
@jimbrooke: I've confirmed myself that only 1x1 HF related commits are missing from the branch, but when you get a chance, can you check the list of explicit recent commits listed in the summary (on github topic) and confirm that I am not missing anything from layer-2 Calo team? |
|
@gennai: its in now. Pending confirmation from jim it should be fully up to date for Calo. I'm in the process of updating with latest muon changes now. |
|
Now fully up to date with muon development branch (at commit 9ec8653). |
|
@mulhearn I am pretty confident all changes from 76X branches are in here |
|
I was just about to push a bunch of updates to dev recipes (even marked as done already above) but they are stuck in my local repo until I can manage to log back into lxplus... |
|
A lot of people have bumped into the same problem with dependent packages... As a general rule of thumb (for all your CMS work!) the first thing you should think of when you get a crash you don't understand is: This will check for additional packages in CMSSW that depends on your local file changes, and check them out locally (because of -a), so there libraries will be built properly including your changes. (Mixing libraries which access the same object with a different definition often leads to crashes!) Note this problem can creep up even if you didn't use the "--unsafe" option during cms-merge-topic, (which ignores dependent packages for a faster build) if you introduce such a change yourself or via e.g. "git pull". Its fine to use "--unsafe" and ignore this, but when you see a crash, remember to try this first! |
|
An updates from Thomas needs a cherry-pick: |
|
And from Aaron: |
|
Mike, now that 800pre6 is out, could you rebase the recipe to it ? Edit: sorry, ignore that... I had not realised that cms-sw#13166 did not make it into pre6 . |
|
about 1fc3e0e , is it something used only to make L1 ntuples and studies, or also in standard CMS workflows ? |
|
1fc3e0e is not used in any production workflows, just ntuples for L1 trigger studies... |
|
Hi guys, Conditions read from CMS_CONDITIONS via FrontierProd----- Begin Fatal Exception 09-Feb-2016 10:03:58 CET----------------------- Could anybody have a look at it? Best, 1fc3e0ehttps://github.com/cms-l1t-offline/cmssw/commit/1fc3e0eddab0028014194fa03a8bf9ae0adae878 is not used in any production workflows, just ntuples for L1 trigger studies... — |
|
Hi Simone, This got me crazy also already. For me doing a cmsenv used to recover it. Not sure why it's necessary though. Cheers, |
|
it seems not to work for me … Best, Hi Simone, This got me crazy also already. For me doing a cmsenv used to recover it. Not sure why it's necessary though. Cheers, — |
|
Hi Simone, Is L1Trigger/Configuration/python/L1TReEmulateFromRAW_cff.py in your -Len On Tue, Feb 9, 2016 at 10:52 AM, simone gennai notifications@github.com
|
|
I'll look into it.... but note with pre6 out, this dev-recipe branch has a short half-life... |
|
Note also that if you call 'python ' instead of 'cmsRun ' you'll get better debugging output... |
|
I am starting again from scratch (just to make sure my area was not fucked up …). |
|
Should be few hours... essentially as soon as I get a chance. |
|
Sounds good, Sorry for the false alarm. Best, Should be few hours... essentially as soon as I get a chance. — |
|
We have a new integration branch. Follow up here: #147 |
This issue has been retired. Please follow up at:
#147
Developing a recipe for running Stage-2 emulation for menu studies and HLT updates.
The topic branch for this task is here:
https://github.com/cms-l1t-offline/cmssw/tree/l1t-dev-recipe-CMSSW_8_0_0_pre5
The most recent control plots from a small 10k run over zero bias are here:
http://mulhearn.web.cern.ch/mulhearn/plots/recipe_latest/
To install and run do this:
To generate some basic rate and kinematic plots do:
runL1T runs the L1T emulators on digis, and writes L1TNtuple.
runStandardSequences runs the L1T emulators, the DigiToRaw, and RawToDigi standard sequences.
reEmul.py unpacks the inputs to L1T, re-emulates with Stage-1 or Stage-2 (depending on which era you select on the command line) and fills the L1TNtuple. See the file for other examples and limitations.
The macros demonstrate basic rate calculations and kinematic plots. It was written in most cases to use the hardware values, but you should now be able to use the corresponding physical values with no scale factor instead (update underway).
To update your working area, do
But take care that recent updates have introduced dependencies on external packages (see detailed note below) so it may be necessary (if you find your code is crashing) to do:
and recompile.
To check on recent commits to the recipe branch, look here:
https://github.com/cms-l1t-offline/cmssw/commits/l1t-dev-recipe-CMSSW_8_0_0_pre5
The current muon development branch:
https://github.com/cms-l1t-offline/cmssw/tree/l1t-muon-pass2-CMSSW_8_0_0_pre5
is synced for:
The calo stage-2 development branch is now in 80x to make use of 1x1 TPs currently in 80x only. Therefore, only a subset of commits in this (obsolete) branch:
https://github.com/cms-l1t-offline/cmssw/tree/l1t-calostage2-pass2-CMSSW_8_0_0_pre4
are actually be included in the recipe, and they are listed here:
Calo Stage 2 development work is now continuing on
https://github.com/cms-l1t-offline/cmssw/tree/l1t-calostage2-CMSSW_8_0_0_pre5
From https://github.com/cms-l1t-offline/cmssw/tree/l1t-calostage2-CMSSW_7_6_0:
The updates to L1TGlobal in:
https://github.com/cms-l1t-offline/cmssw/tree/l1t-global-CMSSW_8_0_X_2016-01-29-2300 (critical updates)
https://github.com/cms-l1t-offline/cmssw/tree/l1t-global-CMSSW_8_0_0_pre5 (development branch)
rely on a new UTM library and will not be integrated with dev-recipes until pre6.
The text was updated successfully, but these errors were encountered: