VecGeom: Update to version 1.2.7 #8864

smuzaffar · 2023-12-08T06:29:11Z

This fixes #8863

smuzaffar · 2023-12-08T06:29:19Z

please test

cmsbuild · 2023-12-08T06:29:35Z

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_14_0_X/master.

@smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@antoniovilela, @rappoccio, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

cmsbuild · 2023-12-08T06:48:52Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-33304d/36374/summary.html
COMMIT: 0cb1cb9
CMSSW: CMSSW_14_0_X_2023-12-07-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36374/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

CMakeLists.txt:49 (include)


-- Configuring incomplete, errors occurred!
See also "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/geant4/11.1.2-786016dae37a6040739c2d65e31d2e7d/build/CMakeFiles/CMakeOutput.log".
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.jJN9Sd (%build)


RPM build errors:
line 37: It's not recommended to have unversioned Obsoletes: Obsoletes: external+geant4+11.1.2-786016dae37a6040739c2d65e31d2e7d
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.jJN9Sd (%build)

smuzaffar · 2023-12-08T07:57:23Z

please test

cmsbuild · 2023-12-08T07:57:37Z

Pull request #8864 was updated.

cmsbuild · 2023-12-08T08:15:46Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-33304d/36378/summary.html
COMMIT: 251079a
CMSSW: CMSSW_14_0_X_2023-12-07-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36378/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

CMakeLists.txt:49 (include)


-- Configuring incomplete, errors occurred!
See also "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/geant4/11.1.2-dcc51fd8055a6ad1899e3335c5739db0/build/CMakeFiles/CMakeOutput.log".
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.pRoOUp (%build)


RPM build errors:
line 37: It's not recommended to have unversioned Obsoletes: Obsoletes: external+geant4+11.1.2-dcc51fd8055a6ad1899e3335c5739db0
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.pRoOUp (%build)

smuzaffar · 2023-12-08T09:35:35Z

please test

cmsbuild · 2023-12-08T09:35:40Z

Pull request #8864 was updated.

cmsbuild · 2023-12-08T13:24:41Z

-1

Failed Tests: UnitTests RelVals AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-33304d/36380/summary.html
COMMIT: 02d5429
CMSSW: CMSSW_14_0_X_2023-12-07-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 6 errors in the following unit tests:

---> test Miscellanea had ERRORS
---> test DD4hepGeometryTestDriver had ERRORS
---> test testG4PropagatorAnalyzer had ERRORS
and more ...

RelVals

8.0A fatal system signal has occurred: abort signal
25.0A fatal system signal has occurred: abort signal
7.3A fatal system signal has occurred: abort signal

Expand to see more relval errors ...

AddOn Tests

hlt_mc_Fake

A fatal system signal has occurred: abort signal

hlt_mc_Fake

----- Begin Fatal Exception 08-Dec-2023 13:22:00 CET-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   Additional Info:
      [a] Input file file:RelVal_Raw_Fake_MC.root could not be opened.
      [b] Fatal Root Error: @SUB=TStorageFactoryFile::ReadBuffer
read from Storage::xread returned 256. Asked to read n bytes: 300 from offset: 0 with file size: 256

----- End Fatal Exception -------------------------------------------------

hlt_mc_Fake

----- Begin Fatal Exception 08-Dec-2023 13:23:17 CET-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   Additional Info:
      [a] Input file file:RelVal_Raw_Fake_MC.root could not be opened.
      [b] Fatal Root Error: @SUB=TStorageFactoryFile::ReadBuffer
read from Storage::xread returned 256. Asked to read n bytes: 300 from offset: 0 with file size: 256

----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

smuzaffar · 2023-12-08T13:29:12Z

@civanch , I am trying to update vecgeom to version 1.2.6 but G4 fails with error below. Any idea if it is problem with cmssw or our G4 version is not compatible with new vecgeom ?

https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-33304d/36380/runTheMatrix-results/8.0_BeamHalo/step1_BeamHalo.log

-------- EEEE ------- G4Exception-START -------- EEEE -------
*** G4Exception : GeomSolids0002
      issued by : G4UTrap::CheckPlanarity()
Side face ~-X is not planar for solid: TIBModKaptonBoxSter
Discrepancy: -7.78701 mm
UnplacedTrapezoid { 39mm, 0.1rad, 0rad, 0.15mm, 60.2817mm, 60.2817mm, 0rad, 0.15mm, 60.2817mm, 60.2817mm, 0rad }
 

-------- EEEE -------- G4Exception-END --------- EEEE -------

failed stack is

#5  0x00002ae7ee693acf in raise () from /lib64/libc.so.6
#6  0x00002ae7ee666ea5 in abort () from /lib64/libc.so.6
#7  0x00002ae7edf01a49 in __gnu_cxx::__verbose_terminate_handler () at ../../../../libstdc++-v3/libsupc++/vterminate.cc:95
#8  0x00002ae7edf0d06a in __cxxabiv1::__terminate (handler=<optimized out>) at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48
#9  0x00002ae7edf0d0d5 in std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:58
#10 0x00002ae7edf0d328 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x2ae7ec24b628 <typeinfo for cms::Exception>, dest=0x2ae7ec214e20 <cms::Exception::~Exception()>) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:98
#11 0x00002ae82227e909 in ExceptionHandler::Notify(char const*, char const*, G4ExceptionSeverity, char const*) [clone .cold] () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#12 0x00002ae8231afb7e in G4Exception(char const*, char const*, G4ExceptionSeverity, char const*) [clone .constprop.0] () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#13 0x00002ae822b2cab8 in G4Trap::G4Trap(G4String const&, double, double, double, double, double, double, double, double, double, double, double) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#14 0x00002ae822588e4a in DDG4SolidConverter::trap(DDSolid const&) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#15 0x00002ae82257a73a in DDG4Builder::convertSolid(DDSolid const&) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#16 0x00002ae822583d4c in DDG4Builder::convertLV(DDLogicalPart const&) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#17 0x00002ae8225841c7 in DDG4Builder::BuildGeometry(SensitiveDetectorCatalog&) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#18 0x00002ae822588730 in DDDWorld::DDDWorld(DDCompactView const*, cms::DDCompactView const*, SensitiveDetectorCatalog&, int, bool, bool) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#19 0x00002ae8225145c6 in RunManagerMT::initG4(DDCompactView const*, cms::DDCompactView const*, HepPDT::ParticleDataTable const*) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so
#20 0x00002ae822516277 in OscarMTMasterThread::OscarMTMasterThread(edm::ParameterSet const&)::{lambda()#1}::operator()() const () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36380/CMSSW_14_0_X_2023-12-07-2300/biglib/el8_amd64_gcc12/pluginSimulation.so

civanch · 2023-12-08T18:50:18Z

@smuzaffar , G4 developers tried this shape and in their tests everything is working. Would it be possible to make modification in two steps: in the first do not change anything in two files of this PR but only vecgeom.spec with different version?

civanch · 2023-12-09T08:31:10Z

@smuzaffar , Gabriele advice to have a look into version 2.0.0-rc , https://gitlab.cern.ch/VecGeom/VecGeom/-/tree/v2.0.0-rc2?ref_type=tags

It seems (see release notes https://gitlab.cern.ch/VecGeom/VecGeom/-/blob/v2.0.0-rc2/ReleaseNotes.txt?ref_type=tags) that it has full support for Celeritas and remove obsolete vectorization options.

smuzaffar · 2023-12-20T18:27:12Z

@civanch , I see there are many DQM/reco comparison differences, are these expected with Vecgeom update?

smuzaffar · 2023-12-21T09:42:55Z

please test

cmsbuild · 2023-12-21T17:39:11Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-33304d/36621/summary.html
COMMIT: 4fbb511
CMSSW: CMSSW_14_0_X_2023-12-20-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36621/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 79 lines from the logs
Reco comparison results: 2898 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3247277
DQMHistoTests: Total failures: 1788
DQMHistoTests: Total nulls: 20
DQMHistoTests: Total successes: 3245447
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 200 log files, 161 edm output root files, 48 DQM output files
TriggerResults: found differences in 1 / 46 workflows

smuzaffar · 2023-12-22T10:06:06Z

please test

civanch · 2023-12-22T10:19:55Z

@smuzaffar , there are two types of differences: random 1200 in MkFit WF and about 1700 in 2026D98 ttbar WF. From my point of view, number of differences is small. This means the same simulation histories in Run2,3 and some minor effect in Phase2, which may be connected with some fixes between 1.2.1 and 1.2.7, which do not change histories itself, because in the last cases number of differences will be much more.

I would accept this external for the master after extra test you restarted just now.

smuzaffar · 2023-12-22T10:22:47Z

thanks @civanch . I have started #8909 ( just rebuilding vecgeom 1.2.1) to see what type of DQM/reco differences we get.

cmsbuild · 2023-12-22T17:35:30Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-33304d/36655/summary.html
COMMIT: 4fbb511
CMSSW: CMSSW_14_0_X_2023-12-21-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8864/36655/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 253 lines from the logs
Reco comparison results: 2898 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3247277
DQMHistoTests: Total failures: 1785
DQMHistoTests: Total nulls: 20
DQMHistoTests: Total successes: 3245450
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 200 log files, 161 edm output root files, 48 DQM output files
TriggerResults: found differences in 1 / 46 workflows

civanch · 2023-12-23T06:49:57Z

@smuzaffar , looking into differences in 2026D98 WF, it seems that there are statistical fluctuations but no change of shape of any distribution in new/old histograms.

smuzaffar · 2023-12-23T08:46:54Z

+externals

thanks @civanch

cmsbuild · 2023-12-23T08:47:12Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_14_0_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

VecGeom: Update to version 1.2.6

0cb1cb9

cmsbuild added externals-pending pending-signatures tests-started orp-pending labels Dec 8, 2023

cmsbuild added tests-rejected and removed tests-started labels Dec 8, 2023

set VecGeom_GIT_DESCRIBE to avoid vecgeom searching for git tag

251079a

cmsbuild added tests-started and removed tests-rejected labels Dec 8, 2023

cmsbuild added tests-rejected and removed tests-started labels Dec 8, 2023

set VecCore_DIr; fix vectorization lib path

02d5429

cmsbuild added tests-started and removed tests-rejected labels Dec 8, 2023

cmsbuild added tests-rejected and removed tests-started labels Dec 8, 2023

whokion mentioned this pull request Dec 8, 2023

Add celeritas as an external package #8867

Closed

cmsbuild added the tests-approved label Dec 20, 2023

cmsbuild added tests-started and removed tests-approved labels Dec 21, 2023

cmsbuild added tests-approved and removed tests-started labels Dec 21, 2023

cmsbuild added tests-started and removed tests-approved labels Dec 22, 2023

cmsbuild added tests-approved and removed tests-started labels Dec 22, 2023

smuzaffar merged commit cae7006 into IB/CMSSW_14_0_X/master Dec 23, 2023
10 checks passed

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Dec 23, 2023

smuzaffar deleted the smuzaffar-patch-6 branch December 23, 2023 09:06

This was referenced Dec 23, 2023

[geant4] Cleanup spec #8910

Merged

Added celeritas version 0.4.1 #8912

Merged

Add Phase-2 Level-1 trigger ML MET emulator cms-sw/cmssw#43633

Merged

[json] Version 3.11.3; build with cmake #8911

Merged

Reduce symbol search cms-sw/root#193

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VecGeom: Update to version 1.2.7 #8864

VecGeom: Update to version 1.2.7 #8864

smuzaffar commented Dec 8, 2023

smuzaffar commented Dec 8, 2023

cmsbuild commented Dec 8, 2023 •

edited

Loading

cmsbuild commented Dec 8, 2023

smuzaffar commented Dec 8, 2023

cmsbuild commented Dec 8, 2023

cmsbuild commented Dec 8, 2023

smuzaffar commented Dec 8, 2023

cmsbuild commented Dec 8, 2023

cmsbuild commented Dec 8, 2023

smuzaffar commented Dec 8, 2023

civanch commented Dec 8, 2023

civanch commented Dec 9, 2023

smuzaffar commented Dec 20, 2023

smuzaffar commented Dec 21, 2023

cmsbuild commented Dec 21, 2023

smuzaffar commented Dec 22, 2023

civanch commented Dec 22, 2023

smuzaffar commented Dec 22, 2023

cmsbuild commented Dec 22, 2023

civanch commented Dec 23, 2023

smuzaffar commented Dec 23, 2023

cmsbuild commented Dec 23, 2023

VecGeom: Update to version 1.2.7 #8864

VecGeom: Update to version 1.2.7 #8864

Conversation

smuzaffar commented Dec 8, 2023

smuzaffar commented Dec 8, 2023

cmsbuild commented Dec 8, 2023 • edited Loading

cmsbuild commented Dec 8, 2023

External Build

smuzaffar commented Dec 8, 2023

cmsbuild commented Dec 8, 2023

cmsbuild commented Dec 8, 2023

External Build

smuzaffar commented Dec 8, 2023

cmsbuild commented Dec 8, 2023

cmsbuild commented Dec 8, 2023

Unit Tests

RelVals

AddOn Tests

smuzaffar commented Dec 8, 2023

civanch commented Dec 8, 2023

civanch commented Dec 9, 2023

smuzaffar commented Dec 20, 2023

smuzaffar commented Dec 21, 2023

cmsbuild commented Dec 21, 2023

Comparison Summary

smuzaffar commented Dec 22, 2023

civanch commented Dec 22, 2023

smuzaffar commented Dec 22, 2023

cmsbuild commented Dec 22, 2023

Comparison Summary

civanch commented Dec 23, 2023

smuzaffar commented Dec 23, 2023

cmsbuild commented Dec 23, 2023

cmsbuild commented Dec 8, 2023 •

edited

Loading