Migration to DD4hep for ME0 db Loader #33098

slomeo · 2021-03-08T07:35:04Z

PR description:

This PR (only for ME0) followed what I did for GEM in the PR #32982 .

PR validation:

Validation by using "cmsRun CondTools/Geometry/test/me0geometrywriter.py" (with DDD and DD4hep):
by checking printouts for both DD & DD4Hep build methods within the ME0GeometryParsFromDD Class in order to compare some DetId parameters (see below):

//DDD

(0) ME0GeometryParsFromDD - DDD
(1) DDD, dimension dx1 109.951, dx2 263.652, dy 435.84, dz 147.5
(2) DDD, tran vector 1059.4 -2.37076e-12 5393.5
(3) DDD, rot matrix 2.44929e-16 -1 2.44929e-16 4.89859e-16 3.67394e-16 -1 1 -3.67394e-16 -1.22465e-16
(4) DDD, Chamber DetID 704643078 Name ME0Box
(1) DDD, dimension dx1 109.951, dx2 263.652, dy 435.84, dz 2.9875
(2) DDD, tran vector 1059.4 -2.32539e-12 5270
(3) DDD, rot matrix 2.44929e-16 -1 2.44929e-16 4.89859e-16 3.67394e-16 -1 1 -3.67394e-16 -1.22465e-16
(5) DDD, Layer DetID 704643334 Name ME0L
(1) DDD, dimension dx1 237.625, dx2 263.652, dy 73.803, dz 2.9875
(2) DDD, tran vector 1421.44 -2.4584e-12 5270
(3) DDD, rot matrix 2.44929e-16 -1 2.44929e-16 4.89859e-16 3.67394e-16 -1 1 -3.67394e-16 -1.22465e-16
(6) DDD, Eta Partion DetID 704651526 Name GHA001 nStrips 384 nPads 192

and so on...

//DD4Hep

(0) ME0GeometryParsFromDD - DD4HEP
(1) DD4HEP, dimension dx1 109.951, dx2 263.652, dy 435.84, dz 147.5
(2) DD4HEP, tran vector 1059.4 0 5393.5
(3) DD4HEP, rot matrix -6.12323e-17 -1 -6.12323e-17 0 -1.22465e-16 -1 1 1.22465e-16 1.83697e-16
(4) DD4HEP, Chamber DetID 704643078 Name ME0Box
(1) DD4HEP, dimension dx1 109.951, dx2 263.652, dy 435.84, dz 2.9875
(2) DD4HEP, tran vector 1059.4 -1.51244e-14 5270
(3) DD4HEP, rot matrix -6.12323e-17 -1 -6.12323e-17 0 -1.22465e-16 -1 1 1.22465e-16 1.83697e-16
(5) DD4HEP, Layer DetID 704643334 Name ME0L
(1) DD4HEP, dimension dx1 237.625, dx2 263.652, dy 73.803, dz 2.9875
(2) DD4HEP, tran vector 1421.44 2.92124e-14 5270
(3) DD4HEP, rot matrix -6.12323e-17 -1 -6.12323e-17 0 -1.22465e-16 -1 1 1.22465e-16 1.83697e-16
(6) DD4HEP, Eta Partion DetID 704651526 Name GHA001 nStrips 384 nPads 192

and so on..

The DD part of the code has not been touched by this PR (except for the printouts). Added "/ dd4hep::mm;", where it was necessary, in the DD4Hep part of the code.

Validation by using a dump script, as I did for the PRs (Update Migration to DD4Hep for CSC and RPC db Loader #32992 Migration to DD4hep for GEM DB loader #32982 Migration to DD4hep for DT db Loader #32781 ), that reads a local .mb file produced by Payload scripts has been not performed because some problems are present. Please read the FIXME comment in CondTools/Geometry/test/writehelpers/geometryExtended2026_writer.py

This second validation step will be perfomed when all the problems with Payloads will be fixed.

if this PR is a backport please specify the original PR and why you need to backport that PR:

nothing special

…ended2026_writer.py

cmsbuild · 2021-03-08T07:40:00Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33098/21425

This PR adds an extra 24KB to repository

cmsbuild · 2021-03-08T07:40:23Z

A new Pull Request was created by @slomeo (Sergio Lo Meo) for master.

It involves the following packages:

CondTools/Geometry
Geometry/GEMGeometryBuilder

@civanch, @Dr15Jones, @makortel, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @srimanob, @kpedro88, @ggovi can you please review it and eventually sign? Thanks.
@cvuosalo, @watson-ij, @jshlee, @dildick, @mmusich, @fabiocos this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

civanch · 2021-03-08T20:31:30Z

please test

cvuosalo · 2021-03-08T20:33:58Z

CondTools/Geometry/test/me0geometrywriter.py

@@ -14,6 +24,9 @@

 process.ME0GeometryWriter = cms.EDAnalyzer("ME0RecoIdealDBLoader")

+#process.ME0GeometryWriter = cms.EDAnalyzer("ME0RecoIdealDBLoader",
+#                                           fromDD4Hep = cms.untracked.bool(False))


Shouldn't these lines be uncommented now, and line 25 deleted? The old DD way should work with fromDD4hep set to false, right?

@cvuosalo The old DD way is the default (i.e fromDD4Hep_ = iC.getUntrackedParameter("fromDD4Hep", false); inside the ME0RecoIdealDBLoader Class) ad I did for GEM. I deleted the comments inside me0geometrywriter.py

cvuosalo · 2021-03-08T20:36:39Z

CondTools/Geometry/test/writehelpers/splitExtended2026Database.sh

@@ -1,6 +1,6 @@
 #!/bin/sh

-conddb_import -f sqlite_file:myfile.db -c sqlite_file:GeometryFileExtended2026D41.db -t XMLFILE_Geometry_TagXX_Extended2026D41_mc -i XMLFILE_Geometry_TagXX_Extended2026D41_mc
+conddb_import -f sqlite_file:myfile.db -c sqlite_file:GeometryFileExtended2026D49.db -t XMLFILE_Geometry_TagXX_Extended2026D49_mc -i XMLFILE_Geometry_TagXX_Extended2026D49_mc


All the conddb_import lines should be replaced with conddb, as in splitExtended2021Database.sh. conddb_import is deprecated.

cmsbuild · 2021-03-08T23:03:31Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79be25/13348/summary.html
COMMIT: e022e35
CMSSW: CMSSW_11_3_X_2021-03-08-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33098/13348/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-79be25/34634.0_TTbar_14TeV+2026D76+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-79be25/34834.999_TTbar_14TeV+2026D76PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU

Summary:

No significant changes to the logs found
Reco comparison results: 3 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2849195
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2849166
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 37 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 160 log files, 37 edm output root files, 38 DQM output files

cmsbuild · 2021-03-10T14:42:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33098/21491

This PR adds an extra 28KB to repository

cmsbuild · 2021-03-10T14:43:21Z

Pull request #33098 was updated. @civanch, @Dr15Jones, @makortel, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @srimanob, @kpedro88, @ggovi can you please check and sign again.

cvuosalo · 2021-03-10T20:16:54Z

@cmsbuild please test

cvuosalo · 2021-03-10T20:17:01Z

+1

cmsbuild · 2021-03-10T22:56:42Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79be25/13431/summary.html
COMMIT: dcc4f56
CMSSW: CMSSW_11_3_X_2021-03-09-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33098/13431/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 3 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2849195
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2849166
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 37 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 160 log files, 37 edm output root files, 38 DQM output files

srimanob · 2021-03-11T02:18:15Z

+Upgrade

slomeo · 2021-03-15T08:35:10Z

please, can "db" and "orp" take a look and eventually sign this PR? do you see any problems? Thanks in advance

silviodonato · 2021-03-15T09:58:06Z

kind reminder @ggovi

ggovi · 2021-03-18T13:10:56Z

+1

cmsbuild · 2021-03-18T13:11:21Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-03-18T13:14:31Z

+1

slomeo added 7 commits March 8, 2021 07:30

added printouts for DDD part

c7ac080

added DD4HEP part

add6f52

added writer and xmlwriter

6c7d61c

fixed a typo

66b4771

fixed a problem with CondTools/Geometry/test/writehelpers/geometryExt…

a772852

…ended2026_writer.py

added a FIXME

cf0fd93

fixed a typo

e022e35

cmsbuild added this to the CMSSW_11_3_X milestone Mar 8, 2021

cmsbuild added code-checks-pending db-pending geometry-pending orp-pending pending-signatures tests-pending upgrade-pending labels Mar 8, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 8, 2021

cmsbuild added tests-started and removed tests-pending labels Mar 8, 2021

cvuosalo reviewed Mar 8, 2021

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Mar 8, 2021

fixed Carl V. requests

dcc4f56

cmsbuild removed code-checks-approved tests-approved labels Mar 10, 2021

cmsbuild added code-checks-pending tests-pending labels Mar 10, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 10, 2021

cmsbuild added geometry-approved tests-started and removed geometry-pending tests-pending labels Mar 10, 2021

cmsbuild added tests-approved and removed tests-started labels Mar 10, 2021

cmsbuild added upgrade-approved and removed upgrade-pending labels Mar 11, 2021

cmsbuild added db-approved fully-signed and removed db-pending pending-signatures labels Mar 18, 2021

cmsbuild added orp-approved and removed orp-pending labels Mar 18, 2021

cmsbuild merged commit 3367af4 into cms-sw:master Mar 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migration to DD4hep for ME0 db Loader #33098

Migration to DD4hep for ME0 db Loader #33098

slomeo commented Mar 8, 2021

cmsbuild commented Mar 8, 2021

cmsbuild commented Mar 8, 2021

civanch commented Mar 8, 2021

cvuosalo Mar 8, 2021

slomeo Mar 10, 2021

cvuosalo Mar 8, 2021

slomeo Mar 10, 2021

cmsbuild commented Mar 8, 2021

cmsbuild commented Mar 10, 2021

cmsbuild commented Mar 10, 2021

cvuosalo commented Mar 10, 2021

cvuosalo commented Mar 10, 2021

cmsbuild commented Mar 10, 2021

srimanob commented Mar 11, 2021

slomeo commented Mar 15, 2021

silviodonato commented Mar 15, 2021

ggovi commented Mar 18, 2021

cmsbuild commented Mar 18, 2021

qliphy commented Mar 18, 2021

Migration to DD4hep for ME0 db Loader #33098

Migration to DD4hep for ME0 db Loader #33098

Conversation

slomeo commented Mar 8, 2021

cmsbuild commented Mar 8, 2021

cmsbuild commented Mar 8, 2021

civanch commented Mar 8, 2021

cvuosalo Mar 8, 2021

Choose a reason for hiding this comment

slomeo Mar 10, 2021

Choose a reason for hiding this comment

cvuosalo Mar 8, 2021

Choose a reason for hiding this comment

slomeo Mar 10, 2021

Choose a reason for hiding this comment

cmsbuild commented Mar 8, 2021

Comparison Summary

cmsbuild commented Mar 10, 2021

cmsbuild commented Mar 10, 2021

cvuosalo commented Mar 10, 2021

cvuosalo commented Mar 10, 2021

cmsbuild commented Mar 10, 2021

Comparison Summary

srimanob commented Mar 11, 2021

slomeo commented Mar 15, 2021

silviodonato commented Mar 15, 2021

ggovi commented Mar 18, 2021

cmsbuild commented Mar 18, 2021

qliphy commented Mar 18, 2021