Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

short names for matrix workflows #40106

Merged
merged 7 commits into from Dec 3, 2022

Conversation

vlimant
Copy link
Contributor

@vlimant vlimant commented Nov 17, 2022

PR description:

while going through cms-sw/cms-bot#1872 we discussed the painful long workflow names and subsequent long directory names.
this PR uses the mostly unused feature of defining a nickname for a workflow using the first string
wf[num] = ['anickname', ['step1name', ...] ]

the visual of --show and the runall-report-step123-.log are mostly unchanged

but I'd like to check on opinions about this, and the interaction with cms-bot

PR validation:

configuring a couple of workflows with runTheMatrix.py -l limited -j 0 went smoothly.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40106/33090

  • This PR adds an extra 20KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @vlimant (vlimant) for master.

It involves the following packages:

  • Configuration/PyReleaseValidation (pdmv, upgrade)

@bbilin, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @sunilUIET can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @fabiocos, @Martin-Grunewald, @missirol, @trtomei, @beaucero, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@smuzaffar
Copy link
Contributor

please test

@vlimant
Copy link
Contributor Author

vlimant commented Nov 22, 2022

please test

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40106/33123

  • This PR adds an extra 32KB to repository

@cmsbuild
Copy link
Contributor

Pull request #40106 was updated. @bbilin, @srimanob, @sunilUIET, @kskovpen, @AdrianoDee can you please check and sign again.

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-353010/29173/summary.html
COMMIT: e52dd9d
CMSSW: CMSSW_12_6_X_2022-11-21-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40106/29173/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

----- Begin Fatal Exception 22-Nov-2022 11:24:18 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 346512 lumi: 250 event: 243042266 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Prefetching for module DQMMessageLogger/'DQMMessageLogger'
   [3] Prefetching for module LogErrorHarvester/'logErrorHarvester'
   [4] Prefetching for module HcalNoiseInfoProducer/'hcalnoise'
   [5] Prefetching for module FastjetJetProducer/'ak4PFJets'
   [6] Prefetching for module PFLinker/'particleFlow'
   [7] Prefetching for module PFProducer/'particleFlowTmp'
   [8] Prefetching for module PFBlockProducer/'particleFlowBlock'
   [9] Prefetching for module PFElecTkProducer/'pfTrackElec'
   [10] Prefetching for module GsfTrackProducer/'electronGsfTracks'
   [11] Prefetching for module CkfTrackCandidateMaker/'electronCkfTrackCandidates'
   [12] Prefetching for module MeasurementTrackerEventProducer/'MeasurementTrackerEvent'
   [13] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [14] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [15] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [16] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [17] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [18] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_seed_cleaner_name' not found
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 22-Nov-2022 11:24:33 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 346512 lumi: 250 event: 243042266 stream: 0
   [1] Running path 'dqmoffline_13_step'
   [2] Prefetching for module SMPDQM/'SMPDQM'
   [3] Prefetching for module MuonProducer/'muons'
   [4] Prefetching for module MuonIdProducer/'muons1stStep'
   [5] Prefetching for module DuplicateListMerger/'generalTracks'
   [6] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [7] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [8] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [9] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [10] Prefetching for module TrackProducer/'initialStepTracks'
   [11] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [12] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [13] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [14] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [15] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [16] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [17] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [18] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [19] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [20] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_seed_cleaner_name' not found
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 22-Nov-2022 11:26:43 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 301998 lumi: 9 event: 8792128 stream: 0
   [1] Running path 'dqmoffline_9_step'
   [2] Prefetching for module SMPDQM/'SMPDQM'
   [3] Prefetching for module MuonProducer/'muons'
   [4] Prefetching for module MuonIdProducer/'muons1stStep'
   [5] Prefetching for module HBHEIsolatedNoiseReflagger/'hbhereco@cpu'
   [6] Prefetching for module TrackExtrapolator/'trackExtrapolator'
   [7] Prefetching for module DuplicateListMerger/'generalTracks'
   [8] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [9] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [10] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [11] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [12] Prefetching for module TrackProducer/'initialStepTracks'
   [13] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [14] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [15] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [16] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [17] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [18] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [19] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [20] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [21] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [22] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_seed_cleaner_name' not found
----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 138.4138.4_PromptCollisions/step2_PromptCollisions.log
  • 138.5138.5_ExpressCollisions/step2_ExpressCollisions.log
  • 140.56140.56_RunHI2018/step2_RunHI2018.log
Expand to see more relval errors ...

AddOn Tests

----- Begin Fatal Exception 22-Nov-2022 11:30:34 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 2 stream: 1
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module ReducedRecHitCollectionProducer/'reducedEcalRecHitsEB'
   [4] Prefetching for module InterestingDetIdCollectionProducer/'interestingEcalDetIdPFEB'
   [5] Prefetching for module PFECALSuperClusterProducer/'particleFlowSuperClusterECAL'
   [6] Prefetching for module RecoChargedRefCandidatePrimaryVertexSorter/'offlinePrimaryVertices'
   [7] Prefetching for module ChargedRefCandidateProducer/'trackRefsForJetsBeforeSorting'
   [8] Prefetching for module TrackWithVertexRefSelector/'trackWithVertexRefSelectorBeforeSorting'
   [9] Prefetching for module DuplicateListMerger/'generalTracks'
   [10] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [11] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [12] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [13] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [14] Prefetching for module TrackProducer/'initialStepTracks'
   [15] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [16] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [17] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [18] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [19] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [20] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [21] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [22] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [23] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [24] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_seed_cleaner_name' not found
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 22-Nov-2022 11:28:56 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 1
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module ReducedRecHitCollectionProducer/'reducedEcalRecHitsEB'
   [4] Prefetching for module InterestingDetIdCollectionProducer/'interestingEcalDetIdPFEB'
   [5] Prefetching for module PFECALSuperClusterProducer/'particleFlowSuperClusterECAL'
   [6] Prefetching for module RecoChargedRefCandidatePrimaryVertexSorter/'offlinePrimaryVertices'
   [7] Prefetching for module ChargedRefCandidateProducer/'trackRefsForJetsBeforeSorting'
   [8] Prefetching for module TrackWithVertexRefSelector/'trackWithVertexRefSelectorBeforeSorting'
   [9] Prefetching for module DuplicateListMerger/'generalTracks'
   [10] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [11] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [12] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [13] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [14] Prefetching for module TrackProducer/'initialStepTracks'
   [15] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [16] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [17] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [18] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [19] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [20] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [21] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [22] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [23] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [24] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_seed_cleaner_name' not found
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 22-Nov-2022 11:28:02 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 2
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module ReducedRecHitCollectionProducer/'reducedEcalRecHitsEB'
   [4] Prefetching for module InterestingDetIdCollectionProducer/'interestingEcalDetIdPFEB'
   [5] Prefetching for module PFECALSuperClusterProducer/'particleFlowSuperClusterECAL'
   [6] Prefetching for module RecoChargedRefCandidatePrimaryVertexSorter/'offlinePrimaryVertices'
   [7] Prefetching for module ChargedRefCandidateProducer/'trackRefsForJetsBeforeSorting'
   [8] Prefetching for module TrackWithVertexRefSelector/'trackWithVertexRefSelectorBeforeSorting'
   [9] Prefetching for module DuplicateListMerger/'generalTracks'
   [10] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [11] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [12] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [13] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [14] Prefetching for module TrackProducer/'initialStepTracks'
   [15] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [16] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [17] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [18] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [19] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [20] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [21] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [22] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [23] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [24] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_seed_cleaner_name' not found
----- End Fatal Exception -------------------------------------------------
Expand to see more addon errors ...

@AdrianoDee
Copy link
Contributor

please test with cms-sw/cms-bot#1897
(CMSSW_13_0_X_2022-12-01-1100 is out)

@cmsbuild
Copy link
Contributor

cmsbuild commented Dec 1, 2022

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-353010/29393/summary.html
COMMIT: c469770
CMSSW: CMSSW_13_0_X_2022-12-01-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40106/29393/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3421159
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3421137
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 206 log files, 158 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@AdrianoDee
Copy link
Contributor

+upgrade

@vlimant
Copy link
Contributor Author

vlimant commented Dec 2, 2022

@cms-sw/pdmv-l2 please resign

@kskovpen
Copy link
Contributor

kskovpen commented Dec 2, 2022

+pdmv

@cmsbuild
Copy link
Contributor

cmsbuild commented Dec 2, 2022

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

perrotta commented Dec 3, 2022

@smuzaffar
Copy link
Contributor

@vlimant , looks like after this change Relvals logs from IB page are not accessible. The IB page reads runall-report-step123-.log files to get the workflows names and this file still contains the long names [a]. Can you please update https://github.com/cms-sw/cmssw/blob/master/Configuration/PyReleaseValidation/python/WorkFlowRunner.py#L210 to also use short names?

[a] https://cmssdt.cern.ch/buildlogs/el8_amd64_gcc11/www/sat/13.0-sat-11/CMSSW_13_0_X_2022-12-03-1100/pyRelValMatrixLogs/run/runall-report-step123-.log

2500.311_NANOmc106Xul17v2+TTbarMINIAOD10.6_UL17v2+NANO_mc10.6ul17v2+HRV_NANO_mc Step0-PASSED Step1-PASSED Step2-PASSED  - time date Sat Dec  3 14:06:15 2022-date Sat Dec  3 14:06:12 2022; exit: 0 0 0
11634.914_TTbar_14TeV+2021_DDDDB+TTbar_14TeV_TuneCP5_2021_GenSim_DDDDB+Digi_DDDDB_2021+RecoNano_DDDDB_2021+HARVESTNano_DDDDB_2021+ALCA_DDDDB_2021 Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED Step4-PASSED  - time date Sat Dec  3 14:05:49 2022-date Sat Dec  3 14:05:49 2022; exit: 0 0 0 0 0
11650.914_ZMM_14+2021_DDDDB+ZMM_14TeV_TuneCP5_2021_GenSim_DDDDB+Digi_DDDDB_2021+RecoNano_DDDDB_2021+HARVESTNano_DDDDB_2021+ALCA_DDDDB_2021 Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED Step4-PASSED  - time date Sat Dec  3 14:05:49 2022-date Sat Dec  3 14:05:49 2022; exit: 0 0 0 0 0

@vlimant
Copy link
Contributor Author

vlimant commented Dec 4, 2022

I had kept the step list in the final report so that one could see what step was failing directly. I overlooked that something was grabbing the directory names from that file (and honestly I was expecting that some script would).
I am preparing a PR for this, however, would it be more natural to have the IB page do a listing of directories using the wf id from the report ?

@vlimant vlimant deleted the runthematrix_shortnames branch December 4, 2022 11:26
@smuzaffar
Copy link
Contributor

smuzaffar commented Dec 4, 2022

IBs page uses runall-report-step123-.log to get the results of each workflow e.g. how many steps are there and what are the exit codes for every step and previously the name in this file were matching the wf directory and log names.

@vlimant
Copy link
Contributor Author

vlimant commented Dec 4, 2022

I mean, can the parsing be done and mapping to the actual directory done by wf id, instead of removing the step list in #40235

@smuzaffar
Copy link
Contributor

yes can be done but I would suggest to keep the runall-report-step123-.log and wf directory in sync. No idea how many other users might be using runall-report-step123-.log. If you really want to have a full list of steps then I would suggest to generate a separate report file e.g. runall-report-steplist123.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants