Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Particle net monitoring dqm 124 x #39146

Merged

Conversation

rgerosa
Copy link
Contributor

@rgerosa rgerosa commented Aug 23, 2022

This PR is a backport of the monitoring workflow in the offline DQM for the AK4 and AK8 ParticleNet based HLT triggers. The workflow runs over the HLTMonitor dataset only for pp collision data, it is switched off for HI as we don't need it.

The original PRs already merged in master are #39051 and #38748

We need a back port in order to include the monitoring in the next CMSSW 12-4-X release that will be used during pp data-taking.

This PR was tested on Run3 data from the HLTMonitor dataset as follows:

cmsDriver step2_data -s RAW2DIGI,L1Reco,RECO,DQM --eventcontent DQM --datatier DQMIO --conditions 124X_dataRun3_Express_v4 --era Run3 --geometry DB:Extended --process reRECO --filein file:/eos/cms/store/express/Run2022C/HLTMonitor/FEVTHLTALL/Express-v1/000/357/080/00000/001b6704-a4de-4adf-9d17-bd6721ad02ad.root -n -1 --data
cmsDriver step3_data -s HARVESTING:dqmHarvesting --harvesting AtRunEnd --conditions auto:run3_data --era Run3 --geometry DB:Extended --scenario pp --hltProcess reHLT --filein file:step2_RAW2DIGI_L1Reco_RECO_DQM.root --filetype DQM -n -1 --data --no_exec

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 23, 2022

A new Pull Request was created by @rgerosa (raffaele gerosa) for CMSSW_12_4_X.

It involves the following packages:

  • DQMOffline/Trigger (dqm)
  • RecoBTag/Configuration (reconstruction)
  • RecoBTag/ONNXRuntime (reconstruction)

@mandrenguyen, @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @clacaputo, @jpata, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@AlexDeMoor, @emilbols, @HuguesBrun, @missirol, @mtosi, @jhgoh, @JyothsnaKomaragiri, @sscruz, @Fedespring, @andrzejnovak, @demuller, @trocino, @cericeci, @rociovilar this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@emanueleusai
Copy link
Member

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7b784a/27029/summary.html
COMMIT: 1a27c61
CMSSW: CMSSW_12_4_X_2022-08-23-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39146/27029/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 8 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677242
  • DQMHistoTests: Total failures: 14
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3677206
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@emanueleusai
Copy link
Member

+1

@jpata
Copy link
Contributor

jpata commented Aug 29, 2022

Just to note: strictly speaking, this changes the physics content (by adding a new tagger) of a closed release. Usually this is against our no-change policy (IIUC), as now the same campaing can contain potentially different configurations.

I guess this can only be fine with the explicit confirmation of @cms-sw/pdmv-l2 if sample production has not yet actually started.

@kskovpen
Copy link
Contributor

+pdmv

The MC production has not yet started. Let's put it in.

@jpata
Copy link
Contributor

jpata commented Aug 29, 2022

+reconstruction

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_6_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

float minDR = 1000;
int match_index = -1;
for (size_t jhlt = 0; jhlt < ntrigobjecttomatch_; jhlt++) {
if (jhlt >= jetPNETScoreSortedIndicesHLT.size())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was not there in #38748

// match reco and hlt objects considering only the first ntrigobjecttomatch jets for both reco and HLT. Each of them must be matched
std::vector<int> matched_obj;
for (size_t jreco = 0; jreco < ntrigobjecttomatch_; jreco++) {
if (jreco >= jetPNETScoreSortedIndices.size())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was not there in #38748

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@perrotta indeed I added that check about vector size in order to be sure that there is no access to an invalid position. I will also propagate it back to the master branch tomorrow. Since this monitoring is important for pp collision we want to have it online as soon as the data-taking restarts. Hence, I hope this won't stall the approval of this PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rgerosa thank you for checking.
Yes, please prepare first the PR for the master, and also the backport for 12_5, in both cases by adding also the fix here below
Once they are merged, we will move forward also this backport PR for 12_4_X

continue;
float dR = reco::deltaR(selectedJets[jetPNETScoreSortedIndices.at(jreco)].p4(),
jetHLTRefs.at(jetPNETScoreSortedIndicesHLT.at(jhlt))->p4());
if (dR < hltRecoDeltaRmax_ and dR < minDR)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect that here you should have also updated minDR...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch .. indeed this has to be fixed. Updating the branch now

min_jet_pt = 25,
max_jet_eta = 2.5,
)

pfParticleNetAK4JetTags = boostedJetONNXJetTagsProducer.clone(
src = 'pfParticleNetAK4TagInfos',
preprocess_json = 'RecoBTag/Combined/data/ParticleNetAK4/CHS/V00/preprocess.json',
model_path = 'RecoBTag/Combined/data/ParticleNetAK4/CHS/V00/particle-net.onnx',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is different in master?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it didn't occur to me to check a line that was not changed in master or in the backport... :)
git blame tells me that #37964 changed the path, it was merged for CMSSW_12_5_0_pre3, so it's understandable it's different in 12_4_X.
Let me know if that didn't explain it.

@cmsbuild
Copy link
Contributor

Pull request #39146 was updated. @mandrenguyen, @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @clacaputo, @jpata, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

@emanueleusai
Copy link
Member

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7b784a/27175/summary.html
COMMIT: a453299
CMSSW: CMSSW_12_4_X_2022-08-29-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39146/27175/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 6 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677280
  • DQMHistoTests: Total failures: 14
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3677244
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@jpata
Copy link
Contributor

jpata commented Aug 30, 2022

+reconstruction

@emanueleusai
Copy link
Member

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_6_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

perrotta commented Aug 31, 2022

backport of #39051
...and #38748 and #39272

@perrotta
Copy link
Contributor

perrotta commented Sep 8, 2022

+1

@cmsbuild cmsbuild merged commit 3413b63 into cms-sw:CMSSW_12_4_X Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants