Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DD4HEP GEM migration #28796

Merged
merged 9 commits into from Feb 6, 2020
Merged

DD4HEP GEM migration #28796

merged 9 commits into from Feb 6, 2020

Conversation

slomeo
Copy link
Contributor

@slomeo slomeo commented Jan 27, 2020

PR description:

DD4HEP GEM migration

PR validation:

by validateGEMGeometry_cfg.py script (see pdf attached)
GEM1.pdf
GEM2.pdf
GEM3.pdf
GEM4.pdf
GEM5.pdf
GEM6.pdf
GEM7.pdf
GEM8.pdf

if this PR is a backport please specify the original PR:

Before submitting your pull requests, make sure you followed this checklist:

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28796/13484

  • This PR adds an extra 8488KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28796/13485

  • This PR adds an extra 24852KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

1 similar comment
@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28796/13487

  • This PR adds an extra 8500KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @slomeo (Sergio Lo Meo) for master.

It involves the following packages:

DetectorDescription/DDCMS
Geometry/GEMGeometryBuilder
Geometry/MuonNumbering
cmsRecoGeom-2021.root

The following packages do not have a category, yet:

cmsRecoGeom-2021.root
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@civanch, @Dr15Jones, @makortel, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @kpedro88 can you please review it and eventually sign? Thanks.
@jshlee, @ptcox, @dildick, @vargasa this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

Copy link
Contributor

@ianna ianna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, exclude the root file from this PR. Files as such can be added to the data repository if necessary.

#include "Geometry/MuonNumbering/interface/DD4hep_MuonNumbering.h"
#include "Geometry/MuonNumbering/interface/MuonDDDNumbering.h"
#include "Geometry/MuonNumbering/interface/MuonBaseNumber.h"
#include <DetectorDescription/DDCMS/interface/DDCompactView.h>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slomeo - please, use quotes "" instead of <>

<library file="GEMGeometryESModule.cc">
<flags EDM_PLUGIN="1"/>
</library>
<library file="ME0GeometryESModule.cc">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slomeo - the ME0 geometry builder is dropped... Please, restore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ianna There is a problem if I restore the ME0 Geometry Builder in Geometry/GEMGeometryBuilder/plugins/BuildFile.xml even if I made a "scram b clean" and then a "scram b -j 10".
The messages I see is:
....

Building shared library tmp/slc7_amd64_gcc820/src/DetectorDescription/DDCMS/plugins/DetectorDescriptionDD4HepPlugins/libDetectorDescriptionDD4HepPlugins.so
/cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: tmp/slc7_amd64_gcc820/src/Geometry/GEMGeometryBuilder/plugins/ME0GeometryESModule/ME0GeometryESModule.cc.o: in function ME0GeometryESModule::produce(MuonGeometryRecord const&)': ME0GeometryESModule.cc:(.text+0x3f7): undefined reference to ME0GeometryBuilderFromDDD::ME0GeometryBuilderFromDDD()'
/cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: ME0GeometryESModule.cc:(.text+0x413): undefined reference to ME0GeometryBuilderFromDDD::build(DDCompactView const*, MuonDDDConstants const&)' /cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: ME0GeometryESModule.cc:(.text+0x41e): undefined reference to ME0GeometryBuilderFromDDD::~ME0GeometryBuilderFromDDD()'
/cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: ME0GeometryESModule.cc:(.text+0x5fa): undefined reference to ME0GeometryBuilderFromCondDB::ME0GeometryBuilderFromCondDB()' /cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: ME0GeometryESModule.cc:(.text+0x605): undefined reference to ME0GeometryBuilderFromCondDB::build(RecoIdealGeometry const&)'
/cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: ME0GeometryESModule.cc:(.text+0x610): undefined reference to ME0GeometryBuilderFromCondDB::~ME0GeometryBuilderFromCondDB()' /cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: tmp/slc7_amd64_gcc820/src/Geometry/GEMGeometryBuilder/plugins/ME0GeometryESModule/ME0GeometryESModule.cc.o: in function ME0GeometryESModule::produce(MuonGeometryRecord const&) [clone .cold.341]':
ME0GeometryESModule.cc:(.text.unlikely+0x31c): undefined reference to ME0GeometryBuilderFromDDD::~ME0GeometryBuilderFromDDD()' /cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: ME0GeometryESModule.cc:(.text.unlikely+0x368): undefined reference to ME0GeometryBuilderFromCondDB::~ME0GeometryBuilderFromCondDB()'
/cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: tmp/slc7_amd64_gcc820/src/Geometry/GEMGeometryBuilder/plugins/ME0GeometryESModule/ME0GeometryESModule.cc.o: in function edm::eventsetup::ProxyArgumentFactoryTemplate<edm::eventsetup::CallbackProxy<edm::eventsetup::Callback<ME0GeometryESModule, std::unique_ptr<ME0Geometry, std::default_delete<ME0Geometry> >, MuonGeometryRecord, edm::eventsetup::CallbackSimpleDecorator<MuonGeometryRecord> >, MuonGeometryRecord, std::unique_ptr<ME0Geometry, std::default_delete<ME0Geometry> > >, edm::eventsetup::Callback<ME0GeometryESModule, std::unique_ptr<ME0Geometry, std::default_delete<ME0Geometry> >, MuonGeometryRecord, edm::eventsetup::CallbackSimpleDecorator<MuonGeometryRecord> > >::makeKey(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const': ME0GeometryESModule.cc:(.text._ZNK3edm10eventsetup28ProxyArgumentFactoryTemplateINS0_13CallbackProxyINS0_8CallbackI19ME0GeometryESModuleSt10unique_ptrI11ME0GeometrySt14default_deleteIS6_EE18MuonGeometryRecordNS0_23CallbackSimpleDecoratorISA_EEEESA_S9_EESD_E7makeKeyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZNK3edm10eventsetup28ProxyArgumentFactoryTemplateINS0_13CallbackProxyINS0_8CallbackI19ME0GeometryESModuleSt10unique_ptrI11ME0GeometrySt14default_deleteIS6_EE18MuonGeometryRecordNS0_23CallbackSimpleDecoratorISA_EEEESA_S9_EESD_E7makeKeyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0xd): undefined reference to char const* edm::typelookup::className()'
/cvmfs/cms-ib.cern.ch/nweek-02613/slc7_amd64_gcc820/external/gcc/8.2.0-pafccj/bin/../lib/gcc/x86_64-unknown-linux-gnu/8.3.1/../../../../x86_64-unknown-linux-gnu/bin/ld: ME0GeometryESModule.cc:(.text._ZNK3edm10eventsetup28ProxyArgumentFactoryTemplateINS0_13CallbackProxyINS0_8CallbackI19ME0GeometryESModuleSt10unique_ptrI11ME0GeometrySt14default_deleteIS6_EE18MuonGeometryRecordNS0_23CallbackSimpleDecoratorISA_EEEESA_S9_EESD_E7makeKeyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZNK3edm10eventsetup28ProxyArgumentFactoryTemplateINS0_13CallbackProxyINS0_8CallbackI19ME0GeometryESModuleSt10unique_ptrI11ME0GeometrySt14default_deleteIS6_EE18MuonGeometryRecordNS0_23CallbackSimpleDecoratorISA_EEEESA_S9_EESD_E7makeKeyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x16): undefined reference to `std::type_info const& edm::typelookup::classTypeInfo()'
collect2: error: ld returned 1 exit status
gmake: *** [config/SCRAM/GMake/Makefile.rules:1742: tmp/slc7_amd64_gcc820/src/Geometry/GEMGeometryBuilder/plugins/ME0GeometryESModule/libME0GeometryESModule.so] Error 1
gmake: *** Waiting for unfinished jobs....
Copying tmp/slc7_amd64_gcc820/src/Geometry/HcalCommonData/src/GeometryHcalCommonData/libGeometryHcalCommonData.so to productstore area:
Copying tmp/slc7_amd64_gcc820/src/Geometry/HGCalCommonData/src/GeometryHGCalCommonData/libGeometryHGCalCommonData.so to productstore area:
Leaving library rule at src/DetectorDescription/DDCMS/plugins
Leaving library rule at src/DetectorDescription/DDCMS/plugins
Leaving library rule at Geometry/HGCalCommonData
Leaving library rule at Geometry/HcalCommonData
Leaving library rule at src/DetectorDescription/DDCMS/plugins
--- Registered EDM Plugin: RPCGeometryValidationPlugins
--- Registered EDM Plugin: DTGeometryValidationPlugins
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is because you do not link it against Geometry/GEMGeometryBuilder

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ianna : done

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 5, 2020

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4518/console Started: 2020/02/05 21:42

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 5, 2020

+1
Tested at: a6175cd
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-94b539/4518/summary.html
CMSSW: CMSSW_11_1_X_2020-02-05-1100
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 5, 2020

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 5, 2020

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-94b539/4518/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 2698043
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2697696
  • DQMHistoTests: Total skipped: 346
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 147 log files, 16 edm output root files, 34 DQM output files

@kpedro88
Copy link
Contributor

kpedro88 commented Feb 6, 2020

+upgrade

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 6, 2020

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@silviodonato
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit a31b233 into cms-sw:master Feb 6, 2020
@smuzaffar
Copy link
Contributor

Looks like this PR is the reason behind the massive RelVal errors in IBs.

[a]

Geometry option Extended2026D41 unknown.
Traceback (most recent call last):
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/bin/slc7_amd64_gcc820/cmsDriver.py", line 56, in &lt;module&gt;
    run()
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/bin/slc7_amd64_gcc820/cmsDriver.py", line 28, in run
    configBuilder.prepare()
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/python/Configuration/Applications/ConfigBuilder.py", line 2125, in prepare
    self.addStandardSequences()
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/python/Configuration/Applications/ConfigBuilder.py", line 736, in addStandardSequences
    self.loadAndRemember(self.GeometryCFF)
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/python/Configuration/Applications/ConfigBuilder.py", line 325, in loadAndRemember
    self.process.load(includeFile)
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/python/FWCore/ParameterSet/Config.py", line 683, in load
    module = __import__(moduleName)
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/python/Configuration/Geometry/GeometryExtended2026D41Reco_cff.py", line 6, in &lt;module&gt;
    from Configuration.Geometry.GeometryExtended2026D41_cff import *
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/python/Configuration/Geometry/GeometryExtended2026D41_cff.py", line 11, in &lt;module&gt;
    from Geometry.HGCalCommonData.hgcalParametersInitialization_cfi import *
  File "/cvmfs/cms-ib.cern.ch/nweek-02614/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_ROOT620_X_2020-02-06-2300/python/Geometry/HGCalCommonData/hgcalParametersInitialization_cfi.py", line 5, in &lt;module&gt;
    from Configuration.Eras.Modifier_dd4hep_cff import dd4hep
ImportError: No module named Modifier_dd4hep_cff

@makortel
Copy link
Contributor

makortel commented Feb 7, 2020

@smuzaffar I can't understand how this PR could have broken the IB, in configuration side it only adds two test configuration files. #28847 must be somehow involved since it moved the dd4hep from Eras to ProcessModifiers, but it alone did not break the IB (CMSSW_11_1_X_2020-02-05-2300).

I see now that #28847 indeed missed to migrate two cfi files. Also the CMSSW_11_1_X_2020-02-06-2300 seems to be the first full build since #28847 got merged, so could it be that the old Modifier_dd4hep_cff.pyc was somehow reachable from the previous full build?

(anyway, I can make a quick fix)

@fabiocos
Copy link
Contributor

fabiocos commented Feb 7, 2020

@makortel @smuzaffar yes, unfortunately I missed to update two HGCal fragments in my migration, sorry for this. I see that #28881 fixes this, @silviodonato as this is trivial and straightforward please just merge it for next IB. As far as I can see in https://cmssdt.cern.ch/lxr/search?%21v=CMSSW_11_1_X_2020-02-05-2300&_filestring=&_string=dd4hep_cff I would say that should be enough.

@silviodonato
Copy link
Contributor

silviodonato commented Feb 7, 2020

@slomeo @fabiocos this PR is also the reason of the failure of the following unit test
https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc820/CMSSW_11_1_X_2020-02-07-1100/unitTestLogs/Geometry/DTGeometryBuilder#/

I checked with

@fabiocos
Copy link
Contributor

fabiocos commented Feb 7, 2020

@silviodonato yes, I can confirm (and there is nothing else touching the geometry in the IB that might justify the failure). Looking at the PR the reason is not immediately obvious to me.

@fabiocos
Copy link
Contributor

fabiocos commented Feb 7, 2020

ok, I believe the cause is a change in the output of the dump after this PR, that causes the compariosn with the reference log file to fail. It should be checked, and in case the log file replaced.

@@ -29,7 +29,7 @@ void MuonBaseNumber::addBase(const LevelBaseNumber& num) {
}
sortedBaseNumber.insert(cur, num);

#ifdef LOCAL_DEBUG
//#ifdef LOCAL_DEBUG
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slomeo @cvuosalo @silviodonato this update of MuonBaseNumber adds lots of (I guess) unwanted printouts, this changes the log file (but if LogInfo is silenced this is not noticed but in the summary), and as the output is different wrt the reference, the testDTGeometry unit test fails. @slomeo could you please fix this? Why removing the ifdef?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet