DD4Hep migration (SolidShapes) for TrackerNumberingBuilder (+TkAl & related CondTool/Geometry plugins) #29905

adewit · 2020-05-19T12:21:48Z

PR description:

This changes:

DDSolidShapes -> cms::DDSolidShapes in TrackerNumberingBuilder.
cms::DDSolidShapes enum to match DDSolidShapes
DDCompactView -> cms::DDCompactView in TkAl - TrackerGeometryCompare and tracker-related CondTools/Geometry plugins

PR validation:

Unit tests run successfully; for tracker alignment plugin output was compared before and after these changes

if this PR is a backport please specify the original PR and why you need to backport that PR:

Not a backport

cmsbuild · 2020-05-19T12:22:14Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-05-19T12:27:27Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29905/15496

This PR adds an extra 44KB to repository

cmsbuild · 2020-05-19T12:27:48Z

A new Pull Request was created by @adewit for master.

It involves the following packages:

Alignment/OfflineValidation
CondTools/Geometry
DetectorDescription/DDCMS
Geometry/MTDNumberingBuilder
Geometry/TrackerNumberingBuilder

@civanch, @Dr15Jones, @makortel, @cvuosalo, @christopheralanwest, @ianna, @mdhildreth, @cmsbuild, @tocheng, @tlampen, @ggovi, @pohsun, @kpedro88 can you please review it and eventually sign? Thanks.
@fabiocos, @vargasa, @makortel, @tocheng, @VinInn, @tlampen, @mschrode, @mmusich, @ebrondol, @slomeo, @venturia this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

tlampen · 2020-05-19T12:59:28Z

please test

cmsbuild · 2020-05-19T12:59:53Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6421/console Started: 2020/05/19 15:02

cmsbuild · 2020-05-19T14:25:51Z

+1
Tested at: 7cfc76b
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbdd42/6421/summary.html
CMSSW: CMSSW_11_1_X_2020-05-18-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-05-19T14:25:54Z

Comparison job queued.

fabiocos

I have a few questions about this update, which is interesting also for MTD, where I should forward port it

fabiocos · 2020-05-19T14:42:36Z

Alignment/OfflineValidation/plugins/TrackerGeometryCompare.cc

@@ -360,7 +361,7 @@ void TrackerGeometryCompare::createROOTGeometry(const edm::EventSetup& iSetup) {
  }

  //accessing the initial geometry
-  edm::ESTransientHandle<DDCompactView> cpv;
+  edm::ESTransientHandle<cms::DDCompactView> cpv;


@adewit @cvuosalo @ianna isn't DDD supposed to coexist for the time being with DDCMS? Here I see a replacement, please correct me in case

Fabio is right. We want both. An example of how to handle both old DD and DD4hep in the same file is here:

https://cmssdt.cern.ch/lxr/source/Geometry/MTDGeometryBuilder/plugins/MTDParametersESModule.cc

Should be fixed for the plugins now

fabiocos · 2020-05-19T14:42:58Z

CondTools/Geometry/plugins/PGeometricDetBuilder.cc

@@ -10,11 +10,15 @@
 #include "CondFormats/GeometryObjects/interface/PGeometricDet.h"
 #include "Geometry/Records/interface/IdealGeometryRecord.h"
 #include "Geometry/TrackerNumberingBuilder/interface/GeometricDet.h"
-#include "DetectorDescription/Core/interface/DDCompactView.h"
+#include "DetectorDescription/DDCMS/interface/DDCompactView.h"


here ase well

fabiocos · 2020-05-19T14:43:16Z

CondTools/Geometry/plugins/PGeometricDetExtraBuilder.cc

@@ -10,7 +10,7 @@
 #include "CondFormats/GeometryObjects/interface/PGeometricDetExtra.h"
 #include "Geometry/Records/interface/PGeometricDetExtraRcd.h"
 #include "Geometry/TrackerNumberingBuilder/interface/GeometricDetExtra.h"
-#include "DetectorDescription/Core/interface/DDCompactView.h"
+#include "DetectorDescription/DDCMS/interface/DDCompactView.h"


here as well

fabiocos · 2020-05-19T14:43:24Z

CondTools/Geometry/plugins/PTrackerParametersDBBuilder.cc

@@ -5,7 +5,7 @@
 #include "FWCore/ServiceRegistry/interface/Service.h"
 #include "CondCore/DBOutputService/interface/PoolDBOutputService.h"
 #include "CondFormats/GeometryObjects/interface/PTrackerParameters.h"
-#include "DetectorDescription/Core/interface/DDCompactView.h"
+#include "DetectorDescription/DDCMS/interface/DDCompactView.h"


here as well

fabiocos · 2020-05-19T14:47:19Z

Geometry/MTDNumberingBuilder/src/GeometricTimingDet.cc

@@ -223,5 +223,5 @@ GeometricTimingDet::Rotation GeometricTimingDet::rotationBounds() const {
 std::unique_ptr<Bounds> GeometricTimingDet::bounds() const {
  const std::vector<double>& par = params_;
  TrackerShapeToBounds shapeToBounds;
-  return std::unique_ptr<Bounds>(shapeToBounds.buildBounds(shape_, par));
+  return std::unique_ptr<Bounds>(shapeToBounds.buildBounds(cms::DDSolidShape(static_cast<int>(shape_)), par));


@adewit @ianna there are two kind of volumes which do not seem to have a correspondent in DD4hep, do we have use of them and is this compatible?

Hmm, thinking about it again I think you're right that this would break down if we had a ddpolycone_rrz/ddpolyhedra_rrz (in old dd) - the corresponding int would be 7 or 8 and that doesn't give a corresponding type in cms::DDSolidShape. Think there is a map to the single ddpolycone/ddpolyhedra that can be used to get this correct, need to check.

@adewit Sorry, I had forgotten about the DDSolidShape map. It is in DetectorDescription/DDCMS/interface/DDSolidShapes.h. @ianna I think we need a new template function for a cms::DDSolidShape to old DDSolidShape mapping. Could you please advise?

@cvuosalo and @ianna - I've added the inverse of the value function (called value_reverse to use with the LegacySolidShapeMap to do cms::DDSolidShape➔ DDSolidShape). That seems to do the job; is that what you had in mind?

fabiocos · 2020-05-19T14:47:44Z

Geometry/TrackerNumberingBuilder/bin/stubs/GeometricDetLoader.cc

@@ -10,7 +10,8 @@

 #include "Geometry/Records/interface/IdealGeometryRecord.h"
 #include "Geometry/TrackerNumberingBuilder/interface/GeometricDet.h"
-#include <DetectorDescription/Core/interface/DDCompactView.h>
+//#include <DetectorDescription/Core/interface/DDCompactView.h>
+#include <DetectorDescription/DDCMS/interface/DDCompactView.h>


again this looks a simple replacement

fabiocos · 2020-05-19T14:49:27Z

Geometry/TrackerNumberingBuilder/src/TrackerShapeToBounds.cc

      return buildOpen(_par);
      break;
-    case DDSolidShape::ddpolycone_rrz:
+    case cms::DDSolidShape::ddpolycone:


@adewit @ianna ok, is this supposed to handle the old ddpolycone_rrz in a transparent way? Is it mapped into ddpolycone by DD4hep?

This code appears to do more than it actually does. It should be simplified for clarity.

case cms::DDSolidShape::ddtubs: case cms::DDSolidShape::ddpolycone: case cms::DDSolidShape::ddsubtraction: return buildOpen(); break;

Note that buildOpen() should be changed to take no parameter. If you check the definition of the method, it doesn't use its parameter.

vargasa

Hello @adewit, you may need to double check with @cvuosalo. It looks to me the changes are made directly and as I understand there should be a way to switch between Core/DDCMS (*) until the migration is completed/validated. If the changes were already validated it should be fine tho (imo) but it is good to double check. Thank you for your work :)

[*] http://cms.cern.ch/iCMS/jsp/openfile.jsp?tp=draft&files=8001_cmsdd4hep.pdf

vargasa · 2020-05-19T14:43:58Z

Geometry/TrackerNumberingBuilder/bin/stubs/GeometricDetLoader.cc

@@ -10,7 +10,8 @@

 #include "Geometry/Records/interface/IdealGeometryRecord.h"
 #include "Geometry/TrackerNumberingBuilder/interface/GeometricDet.h"
-#include <DetectorDescription/Core/interface/DDCompactView.h>
+//#include <DetectorDescription/Core/interface/DDCompactView.h>


If headers are not needed anymore please remove them

vargasa · 2020-05-19T14:47:29Z

Geometry/TrackerNumberingBuilder/bin/stubs/GeometricDetLoader.cc

@@ -37,7 +38,7 @@ void GeometricDetLoader::beginRun(edm::Run const& /* iEvent */, edm::EventSetup
    std::cout << "PoolDBOutputService unavailable" << std::endl;


When possible use LogVerbatim. see #29610

There are a load of couts in all these files - maybe changing them over to LogVerbatim should be decoupled from this PR?

@adewit Yes, the cout fix can be done later.

vargasa · 2020-05-19T14:48:35Z

Geometry/TrackerNumberingBuilder/interface/GeometricDet.h

@@ -4,6 +4,7 @@
 #include "CondFormats/GeometryObjects/interface/PGeometricDet.h"
 #include "DetectorDescription/Core/interface/DDExpandedView.h"
 #include "DetectorDescription/Core/interface/DDSolidShapes.h"


Core headers, see my comment above

Andres is right. There are still problems here. I think line 6 needs to be deleted. I don't see the old DDSolidShape being used here.
Also, migration code has to provided for the old DDExpandedView.

Ha - I missed this. L6 can indeed go. As for DDExpandedView - what it provides for this file is two types that are effectively vectors of ints, so I've moved those defs into this file. However, I also see using GeoHistory = std::vector<DDExpandedNode>;, even though GeoHistory is not used in this file. Perhaps that can be removed - then DDExpandedView is not needed in this file at all.

I assume there is a good reason there's no direct dd4hep copy of DDExpandedView / DDExpandedNode (yet) - but if central changes are needed there they could then happen outside of this PR. In any case I don't think I'm the right person to make those changes (I definitely don't know enough about the requirements to be able to do this correctly)

cmsbuild · 2020-05-19T15:55:32Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbdd42/6421/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2694466
DQMHistoTests: Total failures: 48
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2694369
DQMHistoTests: Total skipped: 49
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 150 log files, 16 edm output root files, 35 DQM output files

cmsbuild · 2020-07-10T18:52:08Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-07-10-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/7872/console Started: 2020/07/10 20:57

cmsbuild · 2020-07-10T20:28:34Z

+1
Tested at: 637ae06
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbdd42/7872/summary.html
CMSSW: CMSSW_11_2_X_2020-07-10-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-07-10T20:28:36Z

Comparison job queued.

cmsbuild · 2020-07-10T22:19:32Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbdd42/7872/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2787429
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2787378
DQMHistoTests: Total skipped: 50
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 154 log files, 17 edm output root files, 37 DQM output files

christopheralanwest · 2020-07-13T18:52:09Z

+1

I performed tests similar to those in #29905 (comment). For the dumped versions of the tracker reco geometry, the only differences in the content of the <shape> tag correspond to a handful of interchanges between 5 <-> 7. The other interchanges are now absent. I am assured in #29905 (comment) and #29905 (comment) that these are harmless, which seems to be consistent with the results of the comparison tests.

fabiocos · 2020-07-15T14:53:42Z

I verified that this PR introduces no change in the output of the MTD geometry tests (extended version, where the solid shape is printed as well)

cvuosalo · 2020-07-15T17:04:59Z

@ggovi Please review this PR.

ggovi · 2020-07-16T05:50:58Z

+1

cmsbuild · 2020-07-16T05:51:23Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-07-16T07:53:16Z

+1

cmsbuild added this to the CMSSW_11_1_X milestone May 19, 2020

cmsbuild added alca-pending code-checks-pending comparison-pending db-pending geometry-pending orp-pending pending-signatures tests-pending upgrade-pending labels May 19, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels May 19, 2020

cmsbuild added tests-started and removed tests-pending labels May 19, 2020

cmsbuild added tests-approved and removed tests-started labels May 19, 2020

fabiocos reviewed May 19, 2020

View reviewed changes

vargasa reviewed May 19, 2020

View reviewed changes

cmsbuild added comparison-available and removed comparison-pending labels May 19, 2020

smuzaffar modified the milestones: CMSSW_11_1_X, CMSSW_11_2_X May 22, 2020

cmsbuild added comparison-pending tests-pending and removed tests-approved labels Jul 10, 2020

cmsbuild added tests-started and removed tests-pending labels Jul 10, 2020

cmsbuild added tests-approved and removed tests-started labels Jul 10, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jul 10, 2020

cmsbuild added alca-approved and removed alca-pending labels Jul 13, 2020

cmsbuild mentioned this pull request Jul 15, 2020

HLTTauDQM: conditional plotting #30243

Closed

cmsbuild added db-approved fully-signed and removed db-pending pending-signatures labels Jul 16, 2020

cmsbuild added orp-approved and removed orp-pending labels Jul 16, 2020

cmsbuild merged commit d5e7107 into cms-sw:master Jul 16, 2020

fabiocos mentioned this pull request Jul 20, 2020

MTD Geometry: updated GeometricTimingDet to remove the residual dependency on DDSolidShape #30834

Merged

		@@ -37,7 +38,7 @@ void GeometricDetLoader::beginRun(edm::Run const& /* iEvent */, edm::EventSetup
		std::cout << "PoolDBOutputService unavailable" << std::endl;

DD4Hep migration (SolidShapes) for TrackerNumberingBuilder (+TkAl & related CondTool/Geometry plugins) #29905

DD4Hep migration (SolidShapes) for TrackerNumberingBuilder (+TkAl & related CondTool/Geometry plugins) #29905

Conversation

adewit commented May 19, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented May 19, 2020

cmsbuild commented May 19, 2020

cmsbuild commented May 19, 2020

tlampen commented May 19, 2020

cmsbuild commented May 19, 2020 • edited

cmsbuild commented May 19, 2020

cmsbuild commented May 19, 2020

fabiocos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vargasa left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsbuild commented May 19, 2020

cmsbuild commented Jul 10, 2020 • edited

cmsbuild commented Jul 10, 2020

cmsbuild commented Jul 10, 2020

cmsbuild commented Jul 10, 2020

christopheralanwest commented Jul 13, 2020

fabiocos commented Jul 15, 2020

cvuosalo commented Jul 15, 2020

ggovi commented Jul 16, 2020

cmsbuild commented Jul 16, 2020

silviodonato commented Jul 16, 2020

cmsbuild commented May 19, 2020 •

edited

cmsbuild commented Jul 10, 2020 •

edited