[Geant4] Code adjustment for new Geant4 version #28637

mrodozov · 2019-12-17T12:08:54Z

PR description:

Changes to get new Geant4 10.6 external for the cmsdist Geant4 branch.
See cms-externals/geant4#42 , #28626 and #28629. This PR joins 28626 and 28629 + the requirements discussed in them.

PR validation:

Builds for both cmsdist/master (production "old" geant) and cmsdist/geant4 (10.6 geant)

The previous two cmssw PRs discussion suggests to use int counters to avoid the original build error and from other concerns. This PR builds with both master of cmdist and geant4 branch (works with both geant versions)

cmsbuild · 2019-12-17T12:09:29Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-12-17T12:15:05Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28637/13185

This PR adds an extra 96KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File SimG4CMS/Calo/src/CaloSD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/Calo/src/HcalTestAnalysis.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/FP420/plugins/FP420SD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/FP420/plugins/FP420Test.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/Forward/src/BscTest.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/Forward/src/TimingSD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/Forward/src/TotemSD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/HcalTestBeam/plugins/HcalTB04Analysis.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/PPS/src/PPSPixelSD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/ShowerLibraryProducer/src/CastorShowerLibraryMaker.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/ShowerLibraryProducer/src/HcalForwardAnalysis.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4Core/GFlash/src/GflashHadronWrapperProcess.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4Core/PrintGeomInfo/src/PrintGeomInfoAction.cc modified in PR(s): [Geant] Updates needed for Geant 10.6 #28626
- File Validation/EcalHits/src/EcalSimHitsValidProducer.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File Validation/HcalHits/src/SimG4HcalValidation.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28637/13185/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28637/13185/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2019-12-17T12:22:49Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-12-17T12:22:59Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#5424,cms-externals/geant4#43
Test Parameters:

BUILD_FULL_CMSSW = true
RUN_ON_SLAVE = cmsbuild20
RELEASE_FORMAT = CMSSW_11_1_GEANT4_X
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4026/console Started: 2019/12/17 13:26

cmsbuild · 2019-12-17T12:28:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28637/13186

This PR adds an extra 96KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File SimG4CMS/Calo/src/CaloSD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/Calo/src/HcalTestAnalysis.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/FP420/plugins/FP420SD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/FP420/plugins/FP420Test.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/Forward/src/BscTest.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/Forward/src/TimingSD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/Forward/src/TotemSD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/HcalTestBeam/plugins/HcalTB04Analysis.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/PPS/src/PPSPixelSD.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/ShowerLibraryProducer/src/CastorShowerLibraryMaker.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4CMS/ShowerLibraryProducer/src/HcalForwardAnalysis.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4Core/GFlash/src/GflashHadronWrapperProcess.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File SimG4Core/PrintGeomInfo/src/PrintGeomInfoAction.cc modified in PR(s): [Geant] Updates needed for Geant 10.6 #28626
- File Validation/EcalHits/src/EcalSimHitsValidProducer.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629
- File Validation/HcalHits/src/SimG4HcalValidation.cc modified in PR(s): [Geant] Additional changes for Geant 10.6 #28629

cmsbuild · 2019-12-17T12:29:24Z

A new Pull Request was created by @mrodozov (Mircho Rodozov) for master.

It involves the following packages:

SimG4CMS/Calo
SimG4CMS/FP420
SimG4CMS/Forward
SimG4CMS/HcalTestBeam
SimG4CMS/PPS
SimG4CMS/ShowerLibraryProducer
SimG4Core/GFlash
SimG4Core/PrintGeomInfo
Validation/EcalHits
Validation/HcalHits

@andrius-k, @kmaeshima, @schneiml, @civanch, @mdhildreth, @jfernan2, @fioriNTU can you please review it and eventually sign? Thanks.
@makortel, @argiro, @hatakeyamak this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2019-12-17T19:17:38Z

-1

Tested at: 06c80ed

CMSSW: CMSSW_11_1_GEANT4_X_2019-12-12-2300
SCRAM_ARCH: slc7_amd64_gcc820

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

[HGCAL trigger] Remove HFNose from tower maps #28609 @PFCal-dev: [HGCAL trigger] Remove HFNose from tower maps
Prevent segfault if dd4hep::Solid is not valid #28623 @cvuosalo: Prevent segfault if dd4hep::Solid is not valid
PPS: configurable timing-tracking cut #28621 @CTPPS: PPS: configurable timing-tracking cut
Static analysis changes for DataFormats/FWLite #28631 @Dr15Jones: Static analysis changes for DataFormats/FWLite
Add helper checkConfigParameters function in CSC emulator #28472 @dildick: Add helper checkConfigParameters function in CSC emulator
Add CMS_SA_ALLOW to Core classes #28618 @Dr15Jones: Add CMS_SA_ALLOW to Core classes
Use CmsRun python module for Validation/Geometry tests #28619 @Dr15Jones: Use CmsRun python module for Validation/Geometry tests
Define CLCT patterns for Run-3 (CCLUT-1) #28600 @dildick: Define CLCT patterns for Run-3
Fix dependency and use root tools instead of explicit lib names #28583 @smuzaffar: Fix dependency and use root tools instead of explicit lib names
PPS: address comments #28615 @CTPPS: PPS: address comments
[DQM] Bin by bin normalization off by default #28616 @andrius-k: [DQM] Bin by bin normalization off by default
Make PixelChanneldentifier::thePacking constexpr #28617 @Dr15Jones: Make PixelChanneldentifier::thePacking constexpr
Remove FWCore/GuiBrowsers #28584 @fabiocos: Remove FWCore/GuiBrowsers
add missing dependencies in DataForamts buildfiles #28606 @davidlange6: add missing dependencies in DataForamts buildfiles

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-48a978/4026/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-48a978/4026/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-48a978/4026/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests RelVals AddOn

Unit Tests:

I found errors in the following unit tests:

---> test materialBudgetTrackerPlots had ERRORS
---> test testTauEmbeddingProducers had ERRORS

RelVals:

When I ran the RelVals I found an error in the following workflows:
101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

11634.0 step1

runTheMatrix-results/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

250202.181 step2

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step2_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_25ns --fileout file:RelVal_Raw_Fake1_MC.root : FAILED - time: date Tue Dec 17 20:05:24 2019-date Tue Dec 17 19:58:27 2019 s - exit: 34304
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_2018 --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018 --fileout file:RelVal_Raw_2018_MC.root : FAILED - time: date Tue Dec 17 20:11:46 2019-date Tue Dec 17 19:58:31 2019 s - exit: 34304
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run3 --fileout file:RelVal_Raw_GRun_MC.root : FAILED - time: date Tue Dec 17 20:11:24 2019-date Tue Dec 17 19:58:34 2019 s - exit: 34304
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run3 --fileout file:RelVal_Raw_PRef_MC.root : FAILED - time: date Tue Dec 17 20:06:10 2019-date Tue Dec 17 19:58:40 2019 s - exit: 34304
cmsDriver.py TTbar_Tauola_8TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --fileout file:RelVal_Raw_Fake_MC.root : FAILED - time: date Tue Dec 17 20:05:51 2019-date Tue Dec 17 19:58:46 2019 s - exit: 34304
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2018_pp_on_AA --fileout file:RelVal_Raw_HIon_MC.root : FAILED - time: date Tue Dec 17 20:07:45 2019-date Tue Dec 17 19:58:51 2019 s - exit: 34304
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake2 --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016 --fileout file:RelVal_Raw_Fake2_MC.root : FAILED - time: date Tue Dec 17 20:06:01 2019-date Tue Dec 17 19:58:55 2019 s - exit: 34304
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run3 --fileout file:RelVal_Raw_PIon_MC.root : FAILED - time: date Tue Dec 17 20:08:23 2019-date Tue Dec 17 20:03:32 2019 s - exit: 34304

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

[HGCAL trigger] Remove HFNose from tower maps #28609 @PFCal-dev: [HGCAL trigger] Remove HFNose from tower maps
Prevent segfault if dd4hep::Solid is not valid #28623 @cvuosalo: Prevent segfault if dd4hep::Solid is not valid
PPS: configurable timing-tracking cut #28621 @CTPPS: PPS: configurable timing-tracking cut
Static analysis changes for DataFormats/FWLite #28631 @Dr15Jones: Static analysis changes for DataFormats/FWLite
Add helper checkConfigParameters function in CSC emulator #28472 @dildick: Add helper checkConfigParameters function in CSC emulator
Add CMS_SA_ALLOW to Core classes #28618 @Dr15Jones: Add CMS_SA_ALLOW to Core classes
Use CmsRun python module for Validation/Geometry tests #28619 @Dr15Jones: Use CmsRun python module for Validation/Geometry tests
Define CLCT patterns for Run-3 (CCLUT-1) #28600 @dildick: Define CLCT patterns for Run-3
Fix dependency and use root tools instead of explicit lib names #28583 @smuzaffar: Fix dependency and use root tools instead of explicit lib names
PPS: address comments #28615 @CTPPS: PPS: address comments
[DQM] Bin by bin normalization off by default #28616 @andrius-k: [DQM] Bin by bin normalization off by default
Make PixelChanneldentifier::thePacking constexpr #28617 @Dr15Jones: Make PixelChanneldentifier::thePacking constexpr
Remove FWCore/GuiBrowsers #28584 @fabiocos: Remove FWCore/GuiBrowsers
add missing dependencies in DataForamts buildfiles #28606 @davidlange6: add missing dependencies in DataForamts buildfiles

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-48a978/4026/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-48a978/4026/git-merge-result

cmsbuild · 2019-12-17T19:17:46Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

cmsbuild · 2019-12-17T21:19:19Z

+1
Tested at: 06c80ed
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d3805f/4037/summary.html
CMSSW: CMSSW_11_1_X_2019-12-17-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2019-12-17T21:19:24Z

Comparison job queued.

cmsbuild · 2019-12-17T23:09:09Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d3805f/4037/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2798405
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2798062
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

smuzaffar · 2019-12-18T07:41:28Z

this looks good to go in master IB. Once it is merged then we will merge cmsdist geant4 changes cms-sw/cmsdist#5424

jfernan2 · 2019-12-18T07:50:08Z

+1

civanch · 2019-12-18T10:21:00Z

+1

cmsbuild · 2019-12-18T10:21:16Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-12-19T09:55:46Z

SimG4CMS/ShowerLibraryProducer/src/HcalForwardAnalysis.cc

  if (idHC >= 0 && theHC != nullptr) {
-    std::cout << "FiberhitSize " << theHC->entries() << std::endl;
-    for (j = 0; j < theHC->entries(); j++) {
+    std::cout << "FiberhitSize " << thehc_entries << std::endl;


cout is not ideal, but this code is likely not part of official workflows meant to run in multithreaded mode

fabiocos · 2019-12-19T09:56:54Z

+1

technical update

cmsbuild · 2019-12-19T21:49:52Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#5424,cms-externals/geant4#43
Test Parameters:

BUILD_FULL_CMSSW = true
RUN_ON_SLAVE = cmsbuild20
RELEASE_FORMAT = CMSSW_11_1_GEANT4_X
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4069/console Started: 2019/12/19 22:53

[Geant4] Code adjustment for new Geant4 version

c3a6cfd

cmsbuild added this to the CMSSW_11_1_X milestone Dec 17, 2019

cmsbuild added code-checks-pending comparison-pending dqm-pending orp-pending pending-signatures simulation-pending tests-pending labels Dec 17, 2019

mrodozov mentioned this pull request Dec 17, 2019

[G4] Update geant4 to 4.10.6.0 cms-sw/cmsdist#5424

Merged

cmsbuild added code-checks-rejected and removed code-checks-pending labels Dec 17, 2019

remove weird symbols

06c80ed

cmsbuild added code-checks-pending and removed code-checks-rejected labels Dec 17, 2019

cmsbuild mentioned this pull request Dec 17, 2019

Fixed rare numerical problems cms-externals/geant4#43

Merged

cmsbuild added requires-external tests-started and removed tests-pending labels Dec 17, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 17, 2019

cmsbuild removed the comparison-pending label Dec 17, 2019

cmsbuild added tests-started and removed tests-pending labels Dec 17, 2019

cmsbuild added tests-approved and removed tests-started labels Dec 17, 2019

cmsbuild added comparison-available and removed comparison-pending labels Dec 17, 2019

cmsbuild added dqm-approved and removed dqm-pending labels Dec 18, 2019

This was referenced Dec 18, 2019

[Geant] Updates needed for Geant 10.6 #28626

Closed

Switch GEANT4 branch to Geant4 10.6 cms-externals/geant4#42

Closed

cmsbuild added fully-signed simulation-approved and removed pending-signatures simulation-pending labels Dec 18, 2019

fabiocos reviewed Dec 19, 2019

View reviewed changes

cmsbuild added orp-approved and removed orp-pending labels Dec 19, 2019

cmsbuild merged commit fb33c64 into cms-sw:master Dec 19, 2019

cmsbuild added requires-external tests-started and removed tests-approved labels Dec 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Geant4] Code adjustment for new Geant4 version #28637

[Geant4] Code adjustment for new Geant4 version #28637

mrodozov commented Dec 17, 2019 •

edited

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019 •

edited

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

smuzaffar commented Dec 18, 2019

jfernan2 commented Dec 18, 2019

civanch commented Dec 18, 2019

cmsbuild commented Dec 18, 2019

fabiocos Dec 19, 2019

fabiocos commented Dec 19, 2019

cmsbuild commented Dec 19, 2019 •

edited

[Geant4] Code adjustment for new Geant4 version #28637

[Geant4] Code adjustment for new Geant4 version #28637

Conversation

mrodozov commented Dec 17, 2019 • edited

PR description:

PR validation:

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019 • edited

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

cmsbuild commented Dec 17, 2019

smuzaffar commented Dec 18, 2019

jfernan2 commented Dec 18, 2019

civanch commented Dec 18, 2019

cmsbuild commented Dec 18, 2019

fabiocos Dec 19, 2019

Choose a reason for hiding this comment

fabiocos commented Dec 19, 2019

cmsbuild commented Dec 19, 2019 • edited

mrodozov commented Dec 17, 2019 •

edited

cmsbuild commented Dec 17, 2019 •

edited

cmsbuild commented Dec 19, 2019 •

edited