Production of MC samples for pixel efficiency studies at High PU #1
Comments
|
Files at |
|
For the record here are plots obtained with the following bash script#!/bin/bash
# Save current working dir so img can be outputted there later
W_DIR=$(pwd);
source /afs/cern.ch/cms/cmsset_default.sh;
eval `scram run -sh`;
# Go back to original working directory
cd $W_DIR;
if [ -d $W_DIR/plots_DynIneff ]; then
rm -fr $W_DIR/plots_DynIneff
fi
mkdir -p $W_DIR/plots_DynIneff
tags=(SiPixelDynamicInefficiency_L1_col100 SiPixelDynamicInefficiency_L1_col80 SiPixelDynamicInefficiency_L1_col85 SiPixelDynamicInefficiency_L1_col90 SiPixelDynamicInefficiency_L1_col95 SiPixelDynamicIn
efficiency_L2_col100 SiPixelDynamicInefficiency_L2_col80 SiPixelDynamicInefficiency_L2_col85 SiPixelDynamicInefficiency_L2_col90 SiPixelDynamicInefficiency_L2_col95)
plots=(Geom ColGeom ChipGeom)
for i in "${tags[@]}"
do
for j in "${plots[@]}"
do
getPayloadData.py \
--plugin pluginSiPixelDynamicInefficiency_PayloadInspector \
--plot plot_SiPixelDynamicInefficiency${j}FactorMap \
--tag ${i} \
--time_type Run \
--iovs '{"start_iov": "1", "end_iov": "1"}' \
--db sqlite_file:/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/${i}.db \
--test;
mv *.png $W_DIR/plots_DynIneff/${j}_${i}.png
done
getPayloadData.py \
--plugin pluginSiPixelDynamicInefficiency_PayloadInspector \
--plot plot_SiPixelDynamicInefficiencyPUPixelMaps \
--tag ${i} \
--time_type Run \
--iovs '{"start_iov": "1", "end_iov": "1"}' \
--db sqlite_file:/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/${i}.db \
--test;
mv *.png $W_DIR/plots_DynIneff/PU_${i}.png
donethat validate the internal content of the payloads posted at #1 (comment) |
|
Sample production request:- |
|
The location of the PixelHistoMaker output files on the CERN EOS: Data (runs 362433 to 362657 from MC: The above files are being copied to the CERN EOS. |
|
First pass of results
|
|
I requested transfer of the produced RelVals to T2_HU_Budapest to protect them from being automatically deleted based on the standard 3-month lifetime for RelVals. Below is the list of Rucio rule IDs: |
|
Requested GT creation with the preliminary measurement |
|
For the record PI plot of the PU factors for the new tag: |
|
Relvals requested here:- |
|
I had a very quick look to the
|
|
I wanted to check, what to expect from the previous version of the .db and I've realized the following mistake... (In short: the dyn.ineff. factors are missing the x^4 fit parameter factor) I've been using pol4 for the fitting the dcol eff vs inst. lumi. plot, and I've fixed the parameter 0 to 1 (to have 100% eff at 0 inst. lumi). And when I was printing out the efficiency factors, I was using the number GetNumberFreeParameters() to loop on all parameters. But since I've fixed 1 parameters, this was missing the last parameter... So this validation is useless, since the function with only 4 parameters looks like this |
this explains why the new efficiency is better than the old one. I posted earlier in the thread a plot from the PI about the PU factors (see #1 (comment)). From there you can see there are only 4 parameters. Why wasn't it noticed before? |
|
I've checked your plot, but I've just didn't realize there is 1 factor missing. I'm sorry. |
|
@bartokm @ferencek @mroguljic
For tomorrow, we discuss the results for module dependency. |
|
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/validation/SiPixelDynamicInefficiency_PhaseI_2023_v1_fix.db |
and here as an appetizer is the plot for this payload:
(working on the plot for the polynomial representation) |
|
Hi @sroychow @bartokm, I mean the current 12_6_X GEN-SIM-RAW should be "ok" PS. |
they do have it, but it's the "naive" linear extrapolation from the Run-2 pre-kink data, which should be underestimating the real inefficiency at high PU. |
|
@sroychow the new GT has been created. |
still a bit rough at the edges (need to map the packed DetId to a human readable location in the detector), but here are the validation plots:
|
|
New payload
|
|
New GT: |
I checked the polynomial representation:
barring some mistakes on my side, layer2 (i.e. region3 in the plot) looks fishy? |
|
Indeed it looks bad... Let me double check |
I guess it only proves the payloads have always been bugged :D |
I had a quick chat with Tanja and she reminded me that the Phase 1 geometry was initially buggy so I re-run SiPixelDets.py in CMSSW_12_4_0 using the which is in line with the pixel hadrography plot and different from Danek's file (the timestamp of Danek's file is from 2016 so very likely corresponds to this initial buggy geometry). This means that the latest dynamic inefficiency payload is indeed buggy and we will need to re-run the RelVal production. @mmusich, I guess this also means that the PayloadInspector will need to be updated. A fixed dynamic inefficiency payload is available here So I guess what's left to be understood is the ladder numbering in the DQM plots. |
I've re-run the analyzer in a fresh Run-3 setup and now I get completely opposite conclusions: http://musich.web.cern.ch/musich/display/SiPixelPhase1Dets.txt my printouts: don't match at all. |
according to this printout, odd ladders (e.g. 1) are outer while even ladders (e.g. 2) are inner. inline bool isBPixOuterLadder(const DetId& detid, const TrackerTopology& tTopo, bool isPhase0)
/*--------------------------------------------------------------------*/
{
bool isOuter = false;
int layer = tTopo.pxbLayer(detid.rawId());
bool odd_ladder = tTopo.pxbLadder(detid.rawId()) % 2;
if (isPhase0) {
if (layer == 2)
isOuter = !odd_ladder;
else
isOuter = odd_ladder;
} else {
if (layer == 4)
isOuter = odd_ladder;
else
isOuter = !odd_ladder;
}
return isOuter;
}this code was copied from Janos' one here in SiPixelCoordinates.cc#L155-L180 // Using TrackerTopology
// Ladders have a staggered structure
// Non-flipped ladders are on the outer radius
// Phase 0: Outer ladders are odd for layer 1,3 and even for layer 2
// Phase 1: Outer ladders are odd for layer 4 and even for layer 1,2,3
int SiPixelCoordinates::outer(const DetId& detid) {
if (outer_.count(detid.rawId()))
return outer_[detid.rawId()];
if (!isBPix_(detid))
return outer_[detid.rawId()] = -9999;
int outer = -9999;
int layer = tTopo_->pxbLayer(detid.rawId());
bool odd_ladder = tTopo_->pxbLadder(detid.rawId()) % 2;
if (phase_ == 0) {
if (layer == 2)
outer = !odd_ladder;
else
outer = odd_ladder;
} else if (phase_ == 1) {
if (layer == 4)
outer = odd_ladder;
else
outer = !odd_ladder;
}
return outer_[detid.rawId()] = outer;
}which is then equally wrong.
Since On the bright side, this other piece of code here SiPixelPhase1TrackClusters.cc#L209-L212 bool iAmOuter = ((tkTpl->pxbLadder(id) % 2 == 1) && tkTpl->pxbLayer(id) != 4) ||
((tkTpl->pxbLadder(id) % 2 != 1) && tkTpl->pxbLayer(id) == 4);should be right.
I am speculating if the DQM code was originally written using as a guideline the buggy phase-1 geometry prior to its fixing. |
From the history on GitHub I see that this file was created in January of 2017 at which point the problem with the geometry was possibly not yet known. Nevertheless, it looks like the code was never fixed.
Indeed, there could be several places in DQM where things are not handled correctly due to buggy
It looks like this was fixed in July of 2017 before the file was moved to its current location.
This is very much possible and, unfortunately, some parts of the DQM code possibly never got fixed later on. This made me wonder if handling of the flipped and unflipped modules could have also been affected and from a quick search at least one place in the alignment code is working with the wrong geometry. |
looks like it's time to fix it. I am wondering if the same bug is present in other critical pieces pixel offline s/w not in cmssw. We might be only scratching the surface here.
as far as I understand the code for assigning the signed ladder is again in int SiPixelCoordinates::signed_ladder(const DetId& detid) {
if (signed_ladder_.count(detid.rawId()))
return signed_ladder_[detid.rawId()];
if (!isBPix_(detid))
return signed_ladder_[detid.rawId()] = -9999;
int signed_ladder = PixelBarrelName(detid, tTopo_, phase_).ladderName();
if (quadrant(detid) % 2)
signed_ladder *= -1;
return signed_ladder_[detid.rawId()] = signed_ladder;
}you might want to cross-check if it conforms with your expectations.
fortunately enough, the line you point resides in a not really critical piece of ancillary s/w inside a functionality that was never (afaik) used. On the other hand there are other points in cmssw that exploit the radius <-> flipping relationship.
please confirm. |
This is correct. |
From the code itself this looks fine but I also ran some tests and it works as expected. |
|
@bartokm @Tiziano @ferencek @mroguljic For the reference, the ALCARECO has quite a sizeable stat. Can you start on the analysis already? |
|
@bartokm it looks like the target dataset is also ready. Could you submit jobs for it as well? |
|
Following the discussion at yesterday's Pixel Offline meeting, I've prepared a new version of the payload: updating L1 and L2 factors, using the new hit efficiency vs double column efficiency function. |
|
Payload has been uploaded as |
there's something wrong again (either in the plot or in the payload): BPix1 / outer has lower efficiency than BPix1 / inner. |
indeed, when generating the plot in the PI , I see a different picture: https://cern.ch/go/WN8n |
|
@mmusich this link doesn't work for me |
|
@sroychow there is a problem with the short URL in the service, try now. |
|
anyhow @sroychow it begs the question how did you generate the wrong plot |
|
Looks like it's due to CMSSW version of PI: (https://cern.ch/go/Lq8d) |
I selected 13_1_0_pre1 , and got this https://cern.ch/go/Lq8d |
this link doens't work, anyhow you should always choose the latest version available (in the case at hand it seems you are using a version predating the integration of cms-sw/cmssw#40989 |
you all keep making the same mistake I did (please don't use the short URL, until it's fixed) |
Indeed it seems a bug in the shortening service. We are looking into this (I made CMSDBBROWS-138 to keep track). |
right understood, thanks.. sorry for the confusion |
|
here is for the record a new version of the plot with some improvement (inner / outer fully spelled out and fixed y-scale as recently requested privately):
|
To study pixel efficiency at high pu, Zmumu samples are needed. The idea is the following:-
SiPixelDynamicInefficiency124X_mcRun3_2022_realistic_postEE_Queue. The idea is to re-use GEN-SIM from 124X production and re-run the DIGI with PU mixing & RECO steps. @ SanjanaThe text was updated successfully, but these errors were encountered: