HCAL LUT generation and validation #19864

akhukhun · 2017-07-21T17:08:13Z

The PR includes

code to generate current HCAL LUTs: i.e. 2017 input/compression LUTs and HF channel mask, adc threshold and tdc mask tables.
Simple Hcal LUT analyzer for validation
Increased precision for L1TriggerObject dumping that removes discrepancies due to rounding.

The changes will not affect TP emulation.

cmsbuild · 2017-07-21T17:08:30Z

A new Pull Request was created by @akhukhun for master.

It involves the following packages:

CalibCalorimetry/CaloTPG
CalibCalorimetry/HcalAlgos
CalibCalorimetry/HcalTPGAlgos
CaloOnlineTools/HcalOnlineDb

@ghellwig, @arunhep, @cerminar, @cmsbuild, @rekovic, @franzoni, @ggovi, @davidlange6, @mulhearn, @lpernie can you please review it and eventually sign? Thanks.
@ghellwig, @mariadalfonso, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

rekovic · 2017-07-25T06:27:39Z

@akhukhun
cout is not thread safe. Please use LogInfo or similar.

rekovic · 2017-07-25T06:32:44Z

CaloOnlineTools/HcalOnlineDb/src/WriteL1TriggerObjectsTxt.cc

+    const HcalLutMetadata *metadata = conditions->getHcalLutMetadata();
+    const HcalTopology *topo = metadata->topo();
+
+    std::auto_ptr<HcalL1TriggerObjects> HcalL1TrigObjCol(new HcalL1TriggerObjects);


auto_ptr is depricated in C++11. Please use unique_ptr

rekovic · 2017-07-25T06:34:21Z

CalibCalorimetry/CaloTPG/src/CaloTPGTranscoderULUT.cc

@@ -70,6 +70,7 @@ void CaloTPGTranscoderULUT::loadHCALCompress(HcalLutMetadata const& lutMetadata,
 	bool isHBHE = (abs(ieta) < theTrigTowerGeometry.firstHFTower(version)); 

        unsigned int lutsize = getOutputLUTSize(id);
+	outputLUT_[index].resize(lutsize);

        for (unsigned int i = 0; i < threshold; ++i)


please use range-based loop.

rekovic · 2017-07-25T06:34:41Z

CalibCalorimetry/HcalTPGAlgos/src/HcaluLUTTPGCoder.cc

@@ -267,6 +268,8 @@ void HcaluLUTTPGCoder::update(const HcalDbService& conditions) {
     float gain = 0;
     uint32_t status = 0;

+     QIEType_[lutId]=conditions.getHcalQIEType(cell)->getValue();
+
     if (LUTGenerationMode_){
        const HcalCalibrations& calibrations = conditions.getHcalCalibrations(cell);
        for (int capId = 0; capId < 4; ++capId){


use range-based loop

rekovic · 2017-07-25T06:35:03Z

CalibCalorimetry/HcalTPGAlgos/src/HcaluLUTTPGCoder.cc

+					   upgradeQIE11LUT_ ;
+    return lut.at(lutId);
+}
+
 void HcaluLUTTPGCoder::lookupMSB(const HBHEDataFrame& df, std::vector<bool>& msb) const{
  msb.resize(df.size());
  for (int i=0; i<df.size(); ++i)


use range-based loop.

rekovic · 2017-07-25T06:35:28Z

CaloOnlineTools/HcalOnlineDb/bin/hcalLUT.cc

+    int nzero[7]; 
+    int nreal[7]; 
+    int extra[7];
+    for(int i=0; i<7; ++i) {


use range-based loop

rekovic · 2017-07-25T06:35:56Z

CaloOnlineTools/HcalOnlineDb/bin/hcalLUT.cc

+	}
+	bool zero=true;
+	const std::vector<unsigned int>& lut2 = xml2->second;
+	for(size_t i=0; i<lut2.size(); ++i){


use range-based loop

rekovic · 2017-07-25T06:36:10Z

CaloOnlineTools/HcalOnlineDb/bin/hcalLUT.cc

+	else     nreal[subdet]++;
+    }
+    cout << Form("%3s:  %8s  %8s  %8s  %8s", "Det", "total", "nonzeros", "zeroes", "extra") << endl; 
+    for(int i=1; i<6; ++i) cout << Form("%3s:  %8d  %8d  %8d  %8d", DET[i], ntota[i], nreal[i], nzero[i], extra[i]) << endl;


use range-based loop

rekovic · 2017-07-25T06:36:24Z

CaloOnlineTools/HcalOnlineDb/bin/hcalLUT.cc

+    cout << Form("%3s:  %8s  %8s  %8s  %8s", "Det", "total", "nonzeros", "zeroes", "extra") << endl; 
+    for(int i=1; i<6; ++i) cout << Form("%3s:  %8d  %8d  %8d  %8d", DET[i], ntota[i], nreal[i], nzero[i], extra[i]) << endl;
+    cout << "--------------------------------------------" << endl;
+    for(int i=0; i<7; ++i) {


use range-based loop

and everywhere else in this file.

rekovic · 2017-07-25T06:37:58Z

CaloOnlineTools/HcalOnlineDb/src/HcalLutManager.cc

@@ -1550,6 +1643,11 @@ int HcalLutManager::createLutXmlFiles_HBEFFromCoder_HOFromAscii_ZDC( std::string
  //
  const std::map<int, boost::shared_ptr<LutXml> > _comp_lut_xml = getCompressionLutXmlFromCoder( _transcoder, _tag, split_by_crate );
  addLutMap( xml, _comp_lut_xml );
+
+  for(int masktype=0; masktype<3; ++masktype){


use range-based loop

rekovic · 2017-07-25T06:38:25Z

CaloOnlineTools/HcalOnlineDb/src/WriteL1TriggerObjectsTxt.cc

+	float gain = 0.0;
+	float ped = 0.0;
+
+	for (int i=0; i<4; ++i) {


use range-based loop

franzoni · 2017-07-25T14:31:21Z

please test

cmsbuild · 2017-07-25T14:31:36Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21736/console Started: 2017/07/25 16:32

cmsbuild · 2017-07-25T15:43:29Z

-1

Tested at: 959c308

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
939e204
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/21736/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/21736/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/21736/summary.html

I found follow errors while testing this PR

Failed tests: ClangBuild

Clang:

I found a compilation error while trying to compile with clang:
I used this command:
scram b vclean && scram build -k -j 16 USER_CXXFLAGS='-fsyntax-only' COMPILER='llvm compile'

Entering library rule at src/CalibCalorimetry/CaloTPG/plugins
Entering library rule at CaloOnlineTools/HcalOnlineDb
>> Compiling edm plugin /build/cmsbld/jenkins-workarea/workspace/ib-any-integration/CMSSW_9_3_X_2017-07-25-1100/src/CalibCalorimetry/CaloTPG/plugins/CaloTPGTranscoderULUTs.cc 
>> Compiling  /build/cmsbld/jenkins-workarea/workspace/ib-any-integration/CMSSW_9_3_X_2017-07-25-1100/src/CaloOnlineTools/HcalOnlineDb/src/ZdcLut.cc 
In file included from /build/cmsbld/jenkins-workarea/workspace/ib-any-integration/CMSSW_9_3_X_2017-07-25-1100/src/CalibCalorimetry/HcalTPGAlgos/src/LutXml.cc:20:
/build/cmsbld/jenkins-workarea/workspace/ib-any-integration/CMSSW_9_3_X_2017-07-25-1100/src/CalibCalorimetry/HcalTPGAlgos/interface/LutXml.h:86:99: error: 'auto' not allowed in function prototype
  XERCES_CPP_NAMESPACE::DOMElement * addData( std::string _elements, std::string _encoding, const auto& _lut );
                                                                                                  ^~~~
/build/cmsbld/jenkins-workarea/workspace/ib-any-integration/CMSSW_9_3_X_2017-07-25-1100/src/CalibCalorimetry/HcalTPGAlgos/src/LutXml.cc:157:83: error: 'auto' not allowed in function prototype
DOMElement * LutXml::addData( std::string _elements, std::string _encoding, const auto& _lut )
                                                                                  ^~~~

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
939e204
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/21736/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/21736/git-merge-result

cmsbuild · 2017-07-25T15:43:31Z

Comparison job queued.

cmsbuild · 2017-07-25T21:48:47Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/21736/summary.html

Comparison Summary:

You potentially added 5 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 23
DQMHistoTests: Total histograms compared: 2363781
DQMHistoTests: Total failures: 14627
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2348988
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 93 log files, 14 edm output root files, 23 DQM output files

cmsbuild · 2017-08-03T09:44:05Z

Pull request #19864 was updated. @ghellwig, @arunhep, @cerminar, @cmsbuild, @rekovic, @franzoni, @ggovi, @mulhearn, @lpernie can you please check and sign again.

kpedro88 · 2017-08-03T13:07:37Z

please test

cmsbuild · 2017-08-03T13:07:53Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22050/console Started: 2017/08/03 16:05

cmsbuild · 2017-08-03T15:22:09Z

+1
Tested at: bc6ca06
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/22050/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
f2e1b83
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/22050/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/22050/git-merge-result

cmsbuild · 2017-08-03T15:22:13Z

Comparison job queued.

cmsbuild · 2017-08-03T18:23:07Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19864/22050/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 25
DQMHistoTests: Total histograms compared: 2639325
DQMHistoTests: Total failures: 44406
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2594738
DQMHistoTests: Total skipped: 181
DQMHistoTests: Total Missing objects: 0
Checked 102 log files, 14 edm output root files, 25 DQM output files

arunhep · 2017-08-08T13:46:39Z

+1

ggovi · 2017-08-15T12:39:48Z

+1

davidlange6 · 2017-08-17T15:33:49Z

+1

rekovic · 2017-08-22T15:38:06Z

+1

cmsbuild · 2017-08-22T15:38:27Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

cmsbuild added this to the CMSSW_9_3_X milestone Jul 21, 2017

cmsbuild added alca-pending comparison-pending db-pending l1-pending orp-pending pending-signatures tests-pending labels Jul 21, 2017

rekovic reviewed Jul 25, 2017

View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Jul 25, 2017

cmsbuild added tests-rejected and removed tests-started labels Jul 25, 2017

cmsbuild removed the comparison-pending label Jul 25, 2017

cmsbuild added tests-started and removed tests-pending labels Aug 3, 2017

cmsbuild added tests-approved and removed tests-started labels Aug 3, 2017

cmsbuild added comparison-available and removed comparison-pending labels Aug 3, 2017

cmsbuild added alca-approved and removed alca-pending labels Aug 8, 2017

cmsbuild added db-approved and removed db-pending labels Aug 15, 2017

cmsbuild added orp-approved and removed orp-pending labels Aug 17, 2017

smuzaffar added the code-checks-pending label Aug 22, 2017

cmsbuild added fully-signed l1-approved and removed l1-pending pending-signatures labels Aug 22, 2017

cmsbuild merged commit 09041a6 into cms-sw:master Aug 22, 2017

akhukhun deleted the new-hcal-lut-generation branch February 22, 2018 09:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HCAL LUT generation and validation #19864

HCAL LUT generation and validation #19864

akhukhun commented Jul 21, 2017

cmsbuild commented Jul 21, 2017

rekovic commented Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

rekovic Jul 25, 2017

franzoni commented Jul 25, 2017

cmsbuild commented Jul 25, 2017 •

edited

cmsbuild commented Jul 25, 2017

cmsbuild commented Jul 25, 2017

cmsbuild commented Jul 25, 2017

cmsbuild commented Aug 3, 2017

kpedro88 commented Aug 3, 2017

cmsbuild commented Aug 3, 2017 •

edited

cmsbuild commented Aug 3, 2017

cmsbuild commented Aug 3, 2017

cmsbuild commented Aug 3, 2017

arunhep commented Aug 8, 2017

ggovi commented Aug 15, 2017

davidlange6 commented Aug 17, 2017

rekovic commented Aug 22, 2017

cmsbuild commented Aug 22, 2017

HCAL LUT generation and validation #19864

HCAL LUT generation and validation #19864

Conversation

akhukhun commented Jul 21, 2017

cmsbuild commented Jul 21, 2017

rekovic commented Jul 25, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

franzoni commented Jul 25, 2017

cmsbuild commented Jul 25, 2017 • edited

cmsbuild commented Jul 25, 2017

cmsbuild commented Jul 25, 2017

cmsbuild commented Jul 25, 2017

cmsbuild commented Aug 3, 2017

kpedro88 commented Aug 3, 2017

cmsbuild commented Aug 3, 2017 • edited

cmsbuild commented Aug 3, 2017

cmsbuild commented Aug 3, 2017

cmsbuild commented Aug 3, 2017

arunhep commented Aug 8, 2017

ggovi commented Aug 15, 2017

davidlange6 commented Aug 17, 2017

rekovic commented Aug 22, 2017

cmsbuild commented Aug 22, 2017

cmsbuild commented Jul 25, 2017 •

edited

cmsbuild commented Aug 3, 2017 •

edited