Muon Tomography with the reconstructed files

[indra-ehep]

PR description:

This adds features

1. To process the reconstructed files (step3.root) for Muon Tomography studies.

2. Checks added for detid validity. This has been found to spot the origin of the problem observed during DQM study. See related presentations at,
   a) See "2022-10-10_HGCal_in_CMSSW.pdf" by Indranil at https://indico.cern.ch/event/1208397/
   b) See Sunada's presentation at https://indico.cern.ch/event/1210093/

3. Added feature to mark the first three cell indices(added by Pruthvi).

Together with PR #39774 this completes the full picture of Muon Tomography both at the generation and reconstruction level.

PR validation:

runTheMatrix test is not applicable since it produces a separate tomography related output file.
Please perform standard test like coding convention and related tests.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport not required merging with 12-6-X is sufficient.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39795/32664

• This PR adds an extra 20KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39795/32664/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39795/32664/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39795/32665

• This PR adds an extra 20KB to repository

[cmsbuild]

A new Pull Request was created by @indra-ehep for master.

It involves the following packages:

• Validation/HGCalValidation (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @syuvivida, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@youyingli, @vandreev11, @sethzenz, @missirol, @rovere, @lgray, @cseez, @apsallid, @bsunanda, @pfs, @lecriste, @hatakeyamak, @ebrondol this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[indra-ehep]

@bsunanda , @emanueleusai , @smuzaffar , Can any you approve the start of the rest of the remaining tests (I meant the magic words of "please test") ?

[bsunanda]

@cmsbuild Please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68432d/28410/summary.html
COMMIT: 2cce1d5
CMSSW: CMSSW_12_6_X_2022-10-20-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39795/28410/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• Reco comparison had 6 failed jobs
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3391158
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3391136
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 201 log files, 48 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

@indra-ehep no, it did not work. You just added one more commit.
What I would do (importing from my private notes, but there should be some documentation somewhere)

git log
git reset --soft [commit]  #(where [commit] is the last commit in your branch before f7a5de78abbb1de7dc40777c93b7eadd256bb080
git commit -m "Adding code to process the reconstruction files for Muon Tomography"
git push -f my-cmssw muontomo-reco

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39795/32754

• This PR adds an extra 20KB to repository

[cmsbuild]

Pull request #39795 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @syuvivida, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

[indra-ehep]

@indra-ehep no, it did not work. You just added one more commit. What I would do (importing from my private notes, but there should be some documentation somewhere)

git log
git reset --soft [commit]  #(where [commit] is the last commit in your branch before f7a5de78abbb1de7dc40777c93b7eadd256bb080
git commit -m "Adding code to process the reconstruction files for Muon Tomography"
git push -f my-cmssw muontomo-reco

@perrotta Thanks, now it works. Please note that f7a5de7 was the first commit of muontomo-reco branch, so I have used,
[idas@ui2 src]$ git reset --soft f7a5de7
and the other commands as mentioned. This makes two similar looking commits as can be checked at https://github.com/indra-ehep/cmssw/commits/muontomo-reco/Validation/HGCalValidation , one dated 20th Oct and another today. Is that fine ?

[perrotta]

@perrotta Thanks, now it works. Please note that f7a5de7 was the first commit of muontomo-reco branch, so I have used,
[idas@ui2 src]$ git reset --soft f7a5de7
and the other commands as mentioned. This makes two similar looking commits as can be checked at https://github.com/indra-ehep/cmssw/commits/muontomo-reco/Validation/HGCalValidation , one dated 20th Oct and another today. Is that fine ?

Ok, good. The two commits ending up now in this PR, the first one being your previous first one, and the second one including all subsequent updates and modifications. Would you have git reset to the commit before your first one (i.e. the one before f7a5de7) then you would have ended up with just one single commit for the whole PR.

However, even two commits are cleaner and easier to review during possible future maintenances than 16.

[perrotta]

please test

[indra-ehep]

@perrotta Thanks, now it works. Please note that f7a5de7 was the first commit of muontomo-reco branch, so I have used,
[idas@ui2 src]$ git reset --soft f7a5de7
and the other commands as mentioned. This makes two similar looking commits as can be checked at https://github.com/indra-ehep/cmssw/commits/muontomo-reco/Validation/HGCalValidation , one dated 20th Oct and another today. Is that fine ?

Ok, good. The two commits ending up now in this PR, the first one being your previous first one, and the second one including all subsequent updates and modifications. Would you have git reset to the commit before your first one (i.e. the one before f7a5de7) then you would have ended up with just one single commit for the whole PR.

However, even two commits are cleaner and easier to review during possible future maintenances than 16.

The initial "git reset" I will follow next time onwards.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68432d/28530/summary.html
COMMIT: 2c51003
CMSSW: CMSSW_12_6_X_2022-10-26-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39795/28530/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 13 differences found in the comparisons
• Reco comparison had 6 failed jobs
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3384029
• DQMHistoTests: Total failures: 18
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3383989
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 201 log files, 48 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[indra-ehep]

@cmsdqm : Please let us know your feedback.

[indra-ehep]

@cmsdqm : Any decision ?

[emanueleusai]

+1

• resign

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1