Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add evaluation of SpecPar numeric values in Phase II Tracker XMLs #32479

Merged
merged 1 commit into from Dec 16, 2020

Conversation

ghugo83
Copy link
Contributor

@ghugo83 ghugo83 commented Dec 14, 2020

This could help speeding-up the DD4hep workflows.

A recent change was added to the dd4hep-version of the FilteredView in CMSSW, in order to directly handle numerical values from XMLs, instead of first storing them as strings.

Mentioned in #32414
@ianna

NB 1: Sim And Reco Phase II XMLs already had evaluated numerical values, nothing to do there.

NB 2: I noticed that there are other XML files with non-evaluated numeric values, outside of Phase II
(except Geometry/TrackerCommonData/data/PhaseI/trackerStructureTopology.xml, which was already fixed in 32a0c74):
Geometry/TrackerCommonData/data/trackerStructureTopology.xml
Geometry/TrackerCommonData/data/Pilot/trackerStructureTopology.xml
Geometry/TrackerCommonData/data/CRack/trackerStructureTopology.xml
Geometry/TwentyFivePercentTrackerCommonData/data/trackerStructureTopology_twentyfivepercent.xml
Geometry/TrackerCommonData/data/PhaseI/PixelForward/trackerStructureTopology.xml
Is it worth it modifying them as well, as they are either unused or pre-Run 3 ? I assume it is not.

… XMLs. This will help speed-up the DD4hep workflows (no time lost with string handling).
@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32479/20385

  • This PR adds an extra 32KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @ghugo83 for master.

It involves the following packages:

Geometry/TrackerCommonData

@civanch, @Dr15Jones, @makortel, @cvuosalo, @ianna, @mdhildreth, @cmsbuild can you please review it and eventually sign? Thanks.
@vargasa, @makortel, @JanFSchulte, @VinInn, @ebrondol, @mtosi, @fabiocos, @venturia this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@cvuosalo
Copy link
Contributor

@cmsbuild please test

@cvuosalo
Copy link
Contributor

Should any of the values be given units? For example, is PixelROC_X a length?

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3eef8b/11644/summary.html
CMSSW: CMSSW_11_3_X_2020-12-14-1100/slc7_amd64_gcc900

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 36
  • DQMHistoTests: Total histograms compared: 2747284
  • DQMHistoTests: Total failures: 7
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2747255
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 35 files compared)
  • Checked 153 log files, 37 edm output root files, 36 DQM output files

@ghugo83
Copy link
Contributor Author

ghugo83 commented Dec 15, 2020

Should any of the values be given units? For example, is PixelROC_X a length?

@cvuosalo No it is fine, all of these are without unit.
PixelROC_X is the number of ROCs in X for each sensor. PixelROCRows is the number of rows per ROC.
Similar reasoning for the variables in Y.

@ianna
Copy link
Contributor

ianna commented Dec 15, 2020

@ghugo83 - the way all these specpars are defined looks quite wasteful: thousands of repetitive lines and identical constants... I'd be surprised if a specpar name is used anywhere. I'd suggest we optimise these descriptions.

@ghugo83
Copy link
Contributor Author

ghugo83 commented Dec 15, 2020

@ianna I am looking at Run 3 for now, I have found ways of speeding up Run 3 DD4hep workflow very significantly :) Will PR it today / tomorrow.
Some of the speeding up for Run 3 will also work for Phase 2.

Regarding the performance of dd4hep workflow in general:
Sth surprising is that the exact same XMLs are used for old DD, where they are processed in a very reasonable time, and several orders of magnitude less than with DD4hep.
So would also need to look what is 'dd4hep-specic' about it.

Regarding these XMLs: the PixelROCRows constants blocks are indeed repeated for each sensor.
Instead of having numerous SpecPars blocks with one path, we could have few SpecPars blocks with many paths.
But in any case, all the sensors paths defined in the XMLs will be looped on, whatever the arrangement 'per SpecPar block' is.
But at least, reducing the size of the file will be beneficial anyway.
This file is automatically produced following what was done in Phase 1. I can easily change few lines in tkLayout, and have sth more compact automatically generated, and see the impact on DD4hep workflow perf.

I already plan to have a set of Phase 2 XMLs generated for DD4hep workflow anyway, at least for testing purposes, to re-arrange the materials in an order which works in the DD4hep case.

@cvuosalo
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@qliphy
Copy link
Contributor

qliphy commented Dec 16, 2020

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants