Displaced tracking validation for Run 3 and Phase 2

[jalimena]

PR description:

This PR adds a collection to the tracking validation to validate displaced tracks. In particular, the usual tip and lip cuts on the tracking particles are relaxed. The collection is turned on only by use of a procModifier as shown below.

PR validation:

I've run the RECO, validation, and DQM steps (standard and trackingOnly) for ttbar and DisplacedSUSY samples, with Run 3 and Phase 2 conditions.

For example:
cmsDriver.py step3 --conditions auto:phase1_2021_realistic -s RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@ExtraHLT+@miniAODDQM --datatier GEN-SIM-RECO,MINIAODSIM,DQMIO -n 200 --geometry DB:Extended --era Run3 --eventcontent RECOSIM,MINIAODSIM,DQM --procModifiers displacedTrackValidation --no_exec
run over the displaced SUSY sample in Run 3 conditions (0 PU), and then harvesting the DQM, produces these plots as output:
https://jalimena.web.cern.ch/jalimena/plots/track_validation_plots_dispSUSY_2021PU0_11_3_0_pre2_ProcModifier/

and
cmsDriver.py step3_trackingOnly --conditions auto:phase2_realistic_T15 -s RAW2DIGI,RECO:reconstruction_trackingOnly,VALIDATION:@trackingOnlyValidation,DQM:@trackingOnlyDQM --datatier GEN-SIM-RECO,DQMIO -n 200 --geometry Extended2026D49 --era Phase2C9 --eventcontent RECOSIM,DQM --procModifiers displacedTrackValidation --no_exec
run over a ttbar sample in phase 2 conditions with the trackingOnly sequence (0 PU), and then harvesting the DQM, produces these plots as output:
https://jalimena.web.cern.ch/jalimena/plots/track_validation_plots_ttbar_phase2PU0_trackingOnly_11_3_0_pre2_ProcModifier/

if this PR is a backport please specify the original PR and why you need to backport that PR:

not a backport

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32157/19851

• This PR adds an extra 36KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Validation/RecoTrack/python/PostProcessorTracker_cfi.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel track reconstruction (10/N) #31722
  • File Validation/RecoTrack/python/TrackValidation_cff.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel track reconstruction (10/N) #31722
  • File Validation/RecoTrack/python/plotting/html.py modified in PR(s): DisplacedGeneralStep in tracking #32036, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel track reconstruction (10/N) #31722
  • File Validation/RecoTrack/python/plotting/trackingPlots.py modified in PR(s): DisplacedGeneralStep in tracking #32036, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel track reconstruction (10/N) #31722
  • File Validation/RecoTrack/test/trackingPerformanceValidation.py modified in PR(s): DisplacedGeneralStep in tracking #32036

[cmsbuild]

A new Pull Request was created by @jalimena (Juliette Alimena) for master.

It involves the following packages:

Validation/RecoTrack

@andrius-k, @kmaeshima, @ErnestaP, @cmsbuild, @jfernan2, @fioriNTU can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @wmtford, @ebrondol, @mtosi, @dgulhan this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[jfernan2]

please test

[cmsbuild]

The tests are being triggered in jenkins.

• CMSSW_11_2_X_2020-11-16-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10798/console Started: 2020/11/17 11:11

[cmsbuild]

+1
Tested at: 99db796
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-57113f/10798/summary.html
CMSSW: CMSSW_11_2_X_2020-11-16-2300
SCRAM_ARCH: slc7_amd64_gcc820

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-57113f/10798/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 35
• DQMHistoTests: Total histograms compared: 2529296
• DQMHistoTests: Total failures: 3091
• DQMHistoTests: Total nulls: 4
• DQMHistoTests: Total successes: 2526179
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 246524.994 KiB( 34 files compared)
• DQMHistoSizes: changed ( 10024.0,... ): 15600.468 KiB Tracking/TrackDisplaced
• DQMHistoSizes: changed ( 10024.0,... ): 0.004 KiB MessageLogger/Errors
• DQMHistoSizes: changed ( 10024.0,... ): 0.004 KiB MessageLogger/Warnings
• DQMHistoSizes: changed ( 1306.0,... ): 12137.562 KiB Tracking/TrackDisplaced
• DQMHistoSizes: changed ( 135.4,... ): 12088.062 KiB Tracking/TrackDisplaced
• DQMHistoSizes: changed ( 23234.0,... ): 12285.731 KiB Tracking/TrackDisplaced
• Checked 148 log files, 22 edm output root files, 35 DQM output files

[jfernan2]

@jalimena this PR is adding around 20k new MEs, I wonder if such a huge number of plots and possibilities is actually needed for Validation, I can imagine what a handicap can be to check all of them with releases and samples.

On the other hand the configuration is creating the new plots for every MC workflow, could you please reduce it to just Phase2 and Run3 samples?

Thanks
All details about the changes introduced in the workflows and samples here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_11_2_X_2020-11-16-2300+57113f/39929/dqm-histo-comparison-summary.html

[jalimena]

@jalimena this PR is adding around 20k new MEs, I wonder if such a huge number of plots and possibilities is actually needed for Validation, I can imagine what a handicap can be to check all of them with releases and samples.

On the other hand the configuration is creating the new plots for every MC workflow, could you please reduce it to just Phase2 and Run3 samples?

Thanks
All details about the changes introduced in the workflows and samples here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_11_2_X_2020-11-16-2300+57113f/39929/dqm-histo-comparison-summary.html

Hi @jfernan2 indeed I don't want to add (or look at) 20k new plots! One thing I see I can do is to remove them for fastsim, but maybe that's not a great idea, given that fastsim is beginning to be used for long-lived particle signals? I also see in the code how I can add the collection just for phase 2 (i.e. with things like phase2_tracker.toModify() in Validation/RecoTrack/python/TrackValidation_cff.py), but I'm unsure how to do it for just Run 3 here - maybe with phase1Pixel.toModify() as I see elsewhere in the package? @mtosi or other tracking experts, can you help me? What do you recommend I do, in practice, to redo the number of plots?

[mmusich]

@jalimena

, but I'm unsure how to do it for just Run 3 here - maybe with phase1Pixel.toModify() as I see elsewhere in the package?

there is a run3 specific modifier:

https://github.com/cms-sw/cmssw/blob/master/Configuration/Eras/python/Modifier_run3_common_cff.py

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-57113f/12553/summary.html
COMMIT: 0141ad3
CMSSW: CMSSW_11_3_X_2021-01-26-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32157/12553/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

• No significant changes to the logs found
• Reco comparison results: 1096 differences found in the comparisons
• DQMHistoTests: Total files compared: 37
• DQMHistoTests: Total histograms compared: 2716596
• DQMHistoTests: Total failures: 3611
• DQMHistoTests: Total nulls: 19
• DQMHistoTests: Total successes: 2712944
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -43.909 KiB( 36 files compared)
• DQMHistoSizes: changed ( 1000.0,... ): 0.078 KiB Tracking/MessageLog
• DQMHistoSizes: changed ( 140.53 ): -44.531 KiB Hcal/DigiRunHarvesting
• DQMHistoSizes: changed ( 140.53 ): -1.172 KiB RPC/DCSInfo
• Checked 156 log files, 37 edm output root files, 37 DQM output files

[jfernan2]

There are a lot of changes in 140.53 unrelated to this PR.... has somebody seen it before?
Thanks

[makortel]

There are a lot of changes in 140.53 unrelated to this PR.... has somebody seen it before?
Thanks

Note

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary.

in the comparison summary message.

[jfernan2]

please test with #32036

[jfernan2]

There are a lot of changes in 140.53 unrelated to this PR.... has somebody seen it before?
Thanks

Note

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary.

in the comparison summary message.

Thanks @makortel I hadn't noticed it

[makortel]

There is some more information in #32755

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-57113f/12568/summary.html
COMMIT: 0141ad3
CMSSW: CMSSW_11_3_X_2021-01-26-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32157/12568/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 37
• DQMHistoTests: Total histograms compared: 2716596
• DQMHistoTests: Total failures: 7
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2716567
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 1.794 KiB( 36 files compared)
• DQMHistoSizes: changed ( 1000.0,... ): 0.078 KiB Tracking/MessageLog
• Checked 156 log files, 37 edm output root files, 37 DQM output files

[jfernan2]

Still to do:

• modify Configuration/PyReleaseValidation appropriately to turn this validation sequence on for the processes we are interested in (I have asked for help by email)
• confirm with PPD/PDMV and DQM conveners that this validation plan is reasonable (awaiting a response to Mia's email)

@jalimena do you plan to include those in this PR or in an independent one?

[jalimena]

Still to do:

• modify Configuration/PyReleaseValidation appropriately to turn this validation sequence on for the processes we are interested in (I have asked for help by email)
• confirm with PPD/PDMV and DQM conveners that this validation plan is reasonable (awaiting a response to Mia's email)

@jalimena do you plan to include those in this PR or in an independent one?

@jfernan2 If I get some advice from anyone soon (perhaps @mtosi ?) about how to modify Configuration/PyReleaseValidation, I will happily include it promptly in this PR. However, I guess it is not strictly necessary and I can open a new one for it. If you all are ready to sign and merge it like ~now, then I can for sure open a new PR for the small additions needed.

[jfernan2]

Ok, so let's decouple the plots from their actual enabling. Thanks

[jfernan2]

+1

[jalimena]

thanks to you!

[silviodonato]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.