[HGCAL] TICL in reconstruction #29081

rovere · 2020-03-03T15:07:14Z

Add TICL to Phase2 Reconstruction

This is a PR for the inclusion of TICL into the standard Phase2 reconstruction.
It will run TICL by default and save its output products into the output file for several data-tiers.

The objects reconstructed by TICL are not injected into ParticleFlow by default. A customization function has been provided (injectTICLintoPF) in order to swap the sim-Assisted reconstruction in HGCAL with the one coming from TICL.

Non-optimal performances on complex events have to be expected out of the box (i.e. quite some development will have to be implemented on top of what is provided in this PR).

Results based on the development contained in this PR have been shown at a recent HGCAL DPG Meeting: link

A special thanks to @gouskos for helping during the development and testing phase.

PR validation:

The limited Matrix run w/o issues.

cmsbuild · 2020-03-03T15:07:44Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-03-03T15:15:11Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29081/13966

This PR adds an extra 216KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/EventContent/python/EventContent_cff.py modified in PR(s): PPS: rename directories with misleading names #29037
- File Configuration/StandardSequences/python/Reconstruction_cff.py modified in PR(s): PPS: rename directories with misleading names #29037, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29081/13966/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29081/13966/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

slava77 · 2020-03-03T15:45:06Z

The final scram build code-format has not been run, yet: we will run it last once the PR is final.

tests can not be started without passing code-checks.
review would be a bit easier if tests can run as well; that's unless this is not expected to run yet.

rovere · 2020-03-03T16:33:24Z

The final scram build code-format has not been run, yet: we will run it last once the PR is final.

tests can not be started without passing code-checks.
review would be a bit easier if tests can run as well; that's unless this is not expected to run yet.

Thanks, @slava77 I was not aware that code-checking was a requirement to run the tests.
I'll apply and push the commit.

cmsbuild · 2020-03-03T16:43:49Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-03-03T16:49:43Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29081/13975

This PR adds an extra 228KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/EventContent/python/EventContent_cff.py modified in PR(s): PPS: rename directories with misleading names #29037
- File Configuration/StandardSequences/python/Reconstruction_cff.py modified in PR(s): PPS: rename directories with misleading names #29037, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

cmsbuild · 2020-03-03T16:50:10Z

A new Pull Request was created by @rovere (Marco Rovere) for master.

It involves the following packages:

Configuration/EventContent
Configuration/StandardSequences
DataFormats/HGCalReco
RecoHGCal/Configuration
RecoHGCal/TICL
Validation/HGCalValidation

The following packages do not have a category, yet:

RecoHGCal/Configuration
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@perrotta, @andrius-k, @kmaeshima, @schneiml, @kpedro88, @cmsbuild, @silviodonato, @franzoni, @jfernan2, @fioriNTU, @slava77, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @vandreev11, @lecriste, @sethzenz, @bsunanda, @makortel, @felicepantaleo, @riga, @GiacomoSguazzoni, @lgray, @cseez, @apsallid, @sobhatta, @pfs, @deguio, @hatakeyamak, @VinInn, @dgulhan, @clelange this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2020-03-04T10:57:38Z

The code-checks are being triggered in jenkins.

perrotta · 2020-04-08T07:55:55Z

full reports PR vs original can be found here:
http://fpantale.web.cern.ch/fpantale/diff_eventSize_step3_29081.html
http://fpantale.web.cern.ch/fpantale/diff_eventSize_inMINIAODSIM_29081.html

Thank you @felicepantaleo

Sizes blow up at 200 PU wrt the no-PU case in #29081 (comment) ...

Do you have the total event size numbers for the two samples (before and after this PR: I don't see them in your logs), so that we can get an idea of the relative size increase in output? It was 8.6% increase overall for the no-PU case.

perrotta · 2020-04-08T07:56:47Z

unhold
(I think this can be un-held @rovere , since the fix has been integrated already)

felicepantaleo · 2020-04-08T08:45:47Z

The overall step3.root file size in bytes when running wf 23306.0 on 100 ttbar + PU200 events:

original	PR	difference (%)
10252191486	11836149551	15.4

rovere · 2020-04-08T08:47:32Z

unhold
(I think this can be un-held @rovere , since the fix has been integrated already)

Yes, indeed, thanks @perrotta

perrotta · 2020-04-08T08:58:24Z

+1

The new producers to allow TICL in reconstruction are implemented here
Those products are stored in the reco output, but not yet used in the following reconstruction or PF
This is an ongoing work, and further optimizations and improvements are expected. The performance in terms of timing and additional event size are however already acceptable for having these products in the Phase2 productions
- Some slimming of the products which mostly contribute to the 15.4% overall expansion of the output size in the 200 PU case is however advisable at some point
Jenkisn tests pass and show no differences besides the added products (and the spurious ones reported above)

jfernan2 · 2020-04-08T08:59:49Z

+dqm

kpedro88 · 2020-04-08T13:14:35Z

+upgrade

silviodonato · 2020-04-09T07:28:39Z

+operations

cmsbuild · 2020-04-09T07:29:09Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-04-09T07:32:11Z

+1

perrotta · 2020-04-10T07:50:24Z

RecoHGCal/TICL/plugins/PFTICLProducer.cc

-  desc.add<edm::InputTag>("ticlCandidateSrc", edm::InputTag("ticlCandidateFromTrackstersProducer"));
-  descriptions.add("pfTICLProducer", desc);
+  desc.add<edm::InputTag>("ticlCandidateSrc", edm::InputTag("ticlCandidateFromTracksters"));
+  descriptions.add("pfTICL", desc);


I think the automatic replacement of *Producer in the product names inlcuded this one by mistake, see #29446

cmsbuild added this to the CMSSW_11_1_X milestone Mar 3, 2020

cmsbuild added code-checks-pending comparison-pending dqm-pending new-package-pending operations-pending orp-pending pending-signatures reconstruction-pending tests-pending upgrade-pending labels Mar 3, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Mar 3, 2020

cmsbuild added code-checks-pending and removed code-checks-rejected labels Mar 3, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 3, 2020

This was referenced Mar 4, 2020

Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

Closed

Revert "PPS: rename directories with misleading names" #29093

Merged

rovere force-pushed the TICL_inReco_111X_pf branch from 5f826c7 to 0db2fbc Compare March 4, 2020 10:57

cmsbuild added code-checks-pending and removed code-checks-approved labels Mar 4, 2020

cmsbuild removed the hold label Apr 8, 2020

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Apr 8, 2020

cmsbuild added dqm-approved and removed dqm-pending labels Apr 8, 2020

cmsbuild added upgrade-approved and removed upgrade-pending labels Apr 8, 2020

cmsbuild added fully-signed operations-approved and removed operations-pending pending-signatures labels Apr 9, 2020

cmsbuild added orp-approved and removed orp-pending labels Apr 9, 2020

cmsbuild merged commit 199f672 into cms-sw:master Apr 9, 2020

cmsbuild mentioned this pull request Apr 9, 2020

DQM: Add a local vs. global section to the README #29438

Merged

kpedro88 mentioned this pull request Apr 9, 2020

HGCal Trackster dictionary error #29444

Closed

silviodonato mentioned this pull request Apr 10, 2020

"No module named pfTICLProducer_cfi" in wf 20493.52 #29446

Closed

perrotta reviewed Apr 10, 2020

View reviewed changes

perrotta mentioned this pull request Apr 10, 2020

Remove TICL dedicated wfs, and adjust the name for the pfTICLProducer automatic config #29450

Merged

slava77 mentioned this pull request May 1, 2020

HGCAL clustering TICL modules to have well-identifiable name in production #27839

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HGCAL] TICL in reconstruction #29081

[HGCAL] TICL in reconstruction #29081

rovere commented Mar 3, 2020 •

edited

cmsbuild commented Mar 3, 2020

cmsbuild commented Mar 3, 2020

slava77 commented Mar 3, 2020

rovere commented Mar 3, 2020 •

edited

cmsbuild commented Mar 3, 2020

cmsbuild commented Mar 3, 2020

cmsbuild commented Mar 3, 2020

cmsbuild commented Mar 4, 2020

perrotta commented Apr 8, 2020

perrotta commented Apr 8, 2020

felicepantaleo commented Apr 8, 2020

rovere commented Apr 8, 2020

perrotta commented Apr 8, 2020

jfernan2 commented Apr 8, 2020

kpedro88 commented Apr 8, 2020

silviodonato commented Apr 9, 2020

cmsbuild commented Apr 9, 2020

silviodonato commented Apr 9, 2020

perrotta Apr 10, 2020

[HGCAL] TICL in reconstruction #29081

[HGCAL] TICL in reconstruction #29081

Conversation

rovere commented Mar 3, 2020 • edited

Add TICL to Phase2 Reconstruction

PR validation:

cmsbuild commented Mar 3, 2020

cmsbuild commented Mar 3, 2020

slava77 commented Mar 3, 2020

rovere commented Mar 3, 2020 • edited

cmsbuild commented Mar 3, 2020

cmsbuild commented Mar 3, 2020

cmsbuild commented Mar 3, 2020

cmsbuild commented Mar 4, 2020

perrotta commented Apr 8, 2020

perrotta commented Apr 8, 2020

felicepantaleo commented Apr 8, 2020

rovere commented Apr 8, 2020

perrotta commented Apr 8, 2020

jfernan2 commented Apr 8, 2020

kpedro88 commented Apr 8, 2020

silviodonato commented Apr 9, 2020

cmsbuild commented Apr 9, 2020

silviodonato commented Apr 9, 2020

perrotta Apr 10, 2020

Choose a reason for hiding this comment

rovere commented Mar 3, 2020 •

edited

rovere commented Mar 3, 2020 •

edited