Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update RecoParticleFlow eventcontent format and remove drop #29470

Merged
merged 4 commits into from Apr 28, 2020

Conversation

jeongeun
Copy link
Contributor

PR description:

Update event content definitions to explicitly have RECO to be a superset of AOD and for FEVT to be a superset of RECO. Clean up 'drop' not really used in RecoParticleFlow. We use 7 'keep commands' with specific label names(L28-34) instead of 'drop recoPFCandidates_particleFlowTmp__*',.

In RecoParticleFlowAOD(L38-47), there is a 'drop *_pfElectronTranslator_*_*', and some 'keep reco*_pfElectronTranslator_*_*',`` 'keep reco*_pfPhotonTranslator_*_*', that need check from experts more clearly.
For current runTheMatrix outputs don't have pf*Translator in any (AOD/RECO/FEVT) event content.
So please check this pf*Translator really use or need in AOD event content.

(The previous tasks are PR#29025, PR#29158, PR#29225, and PR#29299.)

PR validation:

Event Content comparison check was also done and there is no change with these updates.
Tested in CMSSW_11_1_X, the basic test all passed in the CMSSW PR instructions.

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29470/14673

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @jeongeun (JeongEun Lee) for master.

It involves the following packages:

RecoParticleFlow/Configuration

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@mmarionncern, @cbernet, @lgray, @seemasharmafnal, @hatakeyamak this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 14, 2020

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5688/console Started: 2020/04/14 11:15

@cmsbuild
Copy link
Contributor

+1
Tested at: 38d7810
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5583d9/5688/summary.html
CMSSW: CMSSW_11_1_X_2020-04-13-2300
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

Comparison job queued.

'keep *_chargedHadronPFTrackIsolation_*_*'
)
)

# AOD content
RecoParticleFlowAOD = cms.PSet(
outputCommands = cms.untracked.vstring(
'drop CaloTowersSorted_towerMakerPF_*_*',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this drop really needed? Even before any keep statement for it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it isn't really used so it can be deleted. There is no difference after remove this drop line.
(The towerMakerPF doesn't appear to event contents.)

'keep recoCaloClusters_particleFlowEGamma_*_*',
'keep recoSuperClusters_particleFlowEGamma_*_*',
'keep recoCaloClusters_particleFlowSuperClusterECAL_*_*',
'keep recoSuperClusters_particleFlowSuperClusterECAL_*_*',
'keep recoConversions_particleFlowEGamma_*_*',
'keep recoPFCandidates_particleFlow_*_*',
'keep recoPFCandidates_particleFlowTmp_*_*',
'drop recoPFCandidates_particleFlowTmp__*',
'keep recoPFCandidates_particleFlowTmp_AddedMuonsAndHadrons_*',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hatakeyamak @bendavid please confirm that this list is exhaustive of what previously defined with

    'keep recoPFCandidates_particleFlowTmp_*_*',
    'drop recoPFCandidates_particleFlowTmp__*',

)

def _modifyPFEventContentForHGCalRECO( obj ):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, these two def's do not seem to be used anywere

'keep *_particleFlow_electrons_*',
'keep *_particleFlow_photons_*',
'keep *_particleFlow_muons_*',
'drop *_pfElectronTranslator_*_*', #please check pf*Translator really use in AOD
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably not needed, as all the corresponding keep statements are only after it

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5583d9/5688/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 2695166
  • DQMHistoTests: Total failures: 38
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2694809
  • DQMHistoTests: Total skipped: 319
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 147 log files, 16 edm output root files, 34 DQM output files

@silviodonato
Copy link
Contributor

@jeongeun @hatakeyamak @bendavid please have a look at @perrotta's comments above

@hatakeyamak
Copy link
Contributor

I have to do a better job to notice when my name is mentioned in PR conversations.

As far as I know, towerMakerPF is indeed not used any longer anywhere. As @abdoulline wrote #27767, it looks like it was used as inputs to PF clusters, before PF rechits started to be used? I have to admit, I don't know all the history, but it seem it's pretty safe to drop it as already done in this PR.

Maybe @jeongeun is kind enough to also remove
RecoParticleFlow/PFClusterProducer/python/towerMakerPF_cfi.py
? If this needs to be decoupled, we can do it in a follow-up PR.

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29470/14866

  • This PR adds an extra 16KB to repository

@cmsbuild
Copy link
Contributor

Pull request #29470 was updated. @perrotta, @cmsbuild, @slava77 can you please check and sign again.

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 27, 2020

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5868/console Started: 2020/04/27 18:05

@cmsbuild
Copy link
Contributor

+1
Tested at: b133226
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5583d9/5868/summary.html
CMSSW: CMSSW_11_1_X_2020-04-27-1100
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5583d9/5868/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 2696435
  • DQMHistoTests: Total failures: 24
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2696092
  • DQMHistoTests: Total skipped: 319
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 147 log files, 16 edm output root files, 34 DQM output files

@perrotta
Copy link
Contributor

+1

  • Reco event contents re-organized according to the plan
  • Saved reco outputs are not modified
  • Jenkins tests pass

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

@silviodonato
Copy link
Contributor

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants