output adler32 checksumming propagated to JSON file #5236

smorovic · 2014-09-09T10:33:03Z

This includes:

support for calculating adler32 checksum of micro-merged file with adler32_combine function
optional recheck of data read back from disk
write information to output json file (adding this field also to DQM histogram output analyzer)

cmsbuild · 2014-09-09T10:38:04Z

A new Pull Request was created by @smorovic (Srecko Morovic) for CMSSW_7_2_X.

output adler32 checksumming propagated to JSON file

It involves the following packages:

DQMServices/Components
EventFilter/Utilities

@nclopezo, @danduggan, @emeschi, @rovere, @cmsbuild, @deguio, @mommsen, @ojeda can you please review it and eventually sign? Thanks.
@barvic this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

cmsbuild · 2014-09-09T14:01:24Z

-1
Tested at: 39f739a
When I ran the RelVals I found an error in the following worklfows:
8.0 step1

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step1_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log
----- Begin Fatal Exception 09-Sep-2014 15:32:10 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named BeamHalo_cfi_py_GEN_SIM.py
Exception Message:
python encountered the error: 
edm::FileInPath unable to find file SimG4CMS/Calo/data/HFShowerLibrary_oldpmt_noatt_eta4_16en.root anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/CMSSW_7_2_X_2014-09-07-0200/src:/build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/CMSSW_7_2_X_2014-09-07-0200/external/slc6_amd64_gcc481/data:/afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc6_amd64_gcc481/cms/cmssw/CMSSW_7_2_X_2014-09-07-0200/src:/afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc6_amd64_gcc481/cms/cmssw/CMSSW_7_2_X_2014-09-07-0200/external/slc6_amd64_gcc481/data
Current directory is: /build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS
----- End Fatal Exception -------------------------------------------------

9.0 step1

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step1_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log
----- Begin Fatal Exception 09-Sep-2014 15:32:12 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named H200ChargedTaus_Tauola_8TeV_cfi_GEN_SIM.py
Exception Message:
python encountered the error: 
edm::FileInPath unable to find file SimG4CMS/Calo/data/HFShowerLibrary_oldpmt_noatt_eta4_16en.root anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/CMSSW_7_2_X_2014-09-07-0200/src:/build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/CMSSW_7_2_X_2014-09-07-0200/external/slc6_amd64_gcc481/data:/afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc6_amd64_gcc481/cms/cmssw/CMSSW_7_2_X_2014-09-07-0200/src:/afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc6_amd64_gcc481/cms/cmssw/CMSSW_7_2_X_2014-09-07-0200/external/slc6_amd64_gcc481/data
Current directory is: /build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST
----- End Fatal Exception -------------------------------------------------

25.0 step1

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step1_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log
----- Begin Fatal Exception 09-Sep-2014 15:32:13 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named TTbar_Tauola_8TeV_cfi_GEN_SIM.py
Exception Message:
python encountered the error: 
edm::FileInPath unable to find file SimG4CMS/Calo/data/HFShowerLibrary_oldpmt_noatt_eta4_16en.root anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/CMSSW_7_2_X_2014-09-07-0200/src:/build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/CMSSW_7_2_X_2014-09-07-0200/external/slc6_amd64_gcc481/data:/afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc6_amd64_gcc481/cms/cmssw/CMSSW_7_2_X_2014-09-07-0200/src:/afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc6_amd64_gcc481/cms/cmssw/CMSSW_7_2_X_2014-09-07-0200/external/slc6_amd64_gcc481/data
Current directory is: /build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT
----- End Fatal Exception -------------------------------------------------

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log
----- Begin Fatal Exception 09-Sep-2014 15:32:34 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named SingleElectronE120EHCAL_cfi_GEN_SIM_DIGI_DIGI2RAW_RAW2DIGI_L1Reco_RECO_EI.py
Exception Message:
python encountered the error: 
edm::FileInPath unable to find file SimG4CMS/Calo/data/HFShowerLibrary_oldpmt_noatt_eta4_16en.root anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/CMSSW_7_2_X_2014-09-07-0200/src:/build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/CMSSW_7_2_X_2014-09-07-0200/external/slc6_amd64_gcc481/data:/afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc6_amd64_gcc481/cms/cmssw/CMSSW_7_2_X_2014-09-07-0200/src:/afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc6_amd64_gcc481/cms/cmssw/CMSSW_7_2_X_2014-09-07-0200/external/slc6_amd64_gcc481/data
Current directory is: /build/cmsbuild/jenkins-workarea/workspace/ib-integration-CMSSW_7_2_X-slc6_amd64_gcc481/runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5236/876/summary.html

cmsbuild · 2014-09-11T09:41:15Z

+1
Tested at: 39f739a
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5236/902/summary.html

cmsbuild · 2014-09-12T10:01:14Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5236/902/summary.html

mommsen · 2014-09-15T13:59:57Z

+1

cmsbuild · 2014-09-15T14:02:55Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_2_X IBs unless changes (tests are also fine).

output adler32 checksumming propagated to JSON file

adler32 checksumming propagated to JSON file,including DQMFileSaver

39f739a

smorovic mentioned this pull request Sep 9, 2014

output adler32 checksumming propagated to JSON file (71X backport) #5237

Merged

cmsbuild added this to the Next CMSSW_7_2_X milestone Sep 9, 2014

cmsbuild added comparison-pending daq-pending dqm-pending pending-signatures tests-pending labels Sep 9, 2014

cmsbuild added comparison-pending daq-pending dqm-pending pending-signatures tests-rejected and removed comparison-pending daq-pending dqm-pending pending-signatures tests-pending labels Sep 9, 2014

cmsbuild added comparison-pending daq-pending dqm-pending pending-signatures and removed comparison-pending daq-pending dqm-pending pending-signatures tests-rejected labels Sep 11, 2014

cmsbuild added dqm-approved pending-signatures tests-approved labels Sep 11, 2014

cmsbuild added comparison-available daq-pending dqm-approved pending-signatures tests-approved and removed comparison-pending daq-pending dqm-approved pending-signatures tests-approved labels Sep 12, 2014

cmsbuild added comparison-available daq-approved dqm-approved fully-signed tests-approved and removed comparison-available daq-pending dqm-approved pending-signatures tests-approved labels Sep 15, 2014

cmsbuild mentioned this pull request Sep 17, 2014

output adler32 checksumming propagated to JSON file #5385

Merged

davidlange6 added a commit that referenced this pull request Sep 17, 2014

Merge pull request #5236 from smorovic/output-checksum-72X

8af6432

output adler32 checksumming propagated to JSON file

davidlange6 merged commit 8af6432 into cms-sw:CMSSW_7_2_X Sep 17, 2014

smorovic deleted the output-checksum-72X branch October 7, 2014 08:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

output adler32 checksumming propagated to JSON file #5236

output adler32 checksumming propagated to JSON file #5236

smorovic commented Sep 9, 2014

cmsbuild commented Sep 9, 2014

cmsbuild commented Sep 9, 2014

cmsbuild commented Sep 11, 2014

cmsbuild commented Sep 12, 2014

mommsen commented Sep 15, 2014

cmsbuild commented Sep 15, 2014

output adler32 checksumming propagated to JSON file #5236

output adler32 checksumming propagated to JSON file #5236

Conversation

smorovic commented Sep 9, 2014

cmsbuild commented Sep 9, 2014

cmsbuild commented Sep 9, 2014

cmsbuild commented Sep 11, 2014

cmsbuild commented Sep 12, 2014

mommsen commented Sep 15, 2014

cmsbuild commented Sep 15, 2014