Add overrideGUID option to PoolOutputModule #37806

makortel · 2022-05-04T19:17:27Z

PR description:

This PR adds overrideGUID parameter to PoolOutputModule. The intended use case is to ensure consistency of the GUID in the file name and in the file for the manual recovery at Tier0, requested in #37742 (comment).

PR validation:

Unit tests run.

cmsbuild · 2022-05-04T19:23:46Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37806/29724

This PR adds an extra 40KB to repository

cmsbuild · 2022-05-04T19:24:07Z

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

FWCore/Utilities (core)
IOPool/Output (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@felicepantaleo, @wddgit this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

makortel · 2022-05-04T19:27:32Z

@cmsbuild, please test

makortel · 2022-05-04T19:27:46Z

@Dr15Jones, @dan131riley please review

cmsbuild · 2022-05-04T23:31:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25a044/24459/summary.html
COMMIT: a2b9ee4
CMSSW: CMSSW_12_4_X_2022-05-04-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37806/24459/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

There are some workflows for which there are errors in the baseline:
39434.501 step 3
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3700548
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3700518
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 205 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

Dr15Jones · 2022-05-05T14:02:29Z

FWCore/Utilities/interface/GlobalIdentifier.h

@@ -4,6 +4,8 @@
 #include <string>
 namespace edm {
  std::string createGlobalIdentifier(bool binary = false);
-}
+
+  bool isValidGlobalIdentifier(std::string const& guid);


When can we have a non-valid one? Maybe a comment here?

OK, now I see this is testing that the string matches the required format.

IOPool/Output/src/RootOutputFile.cc

cmsbuild · 2022-05-09T20:26:50Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37806/29845

This PR adds an extra 20KB to repository

cmsbuild · 2022-05-09T20:27:13Z

Pull request #37806 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

makortel · 2022-05-09T20:32:19Z

@cmsbuild, please test

cmsbuild · 2022-05-10T00:28:27Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25a044/24552/summary.html
COMMIT: 42dc4a6
CMSSW: CMSSW_12_4_X_2022-05-09-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37806/24552/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test SiStripDAQ_O2O_test had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3696633
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3696597
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 48 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 206 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

makortel · 2022-05-10T15:29:04Z

The unit test fails also in IBs (#37888).

makortel · 2022-05-10T23:57:54Z

+1

cmsbuild · 2022-05-10T23:58:12Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2022-05-11T01:09:28Z

+1

qliphy · 2022-05-11T01:09:57Z

merge

cmsbuild added this to the CMSSW_12_4_X milestone May 4, 2022

cmsbuild added code-checks-pending core-pending orp-pending pending-signatures tests-pending labels May 4, 2022

This was referenced May 4, 2022

Add overrideGUID option to PoolOutputModule cms-sw/framework-team#398

Closed

A tool to change the GUID of a file manually for 8034-FileNameInconsistentWithGUID issues #37742

Closed

cmsbuild added code-checks-approved and removed code-checks-pending labels May 4, 2022

cmsbuild added tests-started and removed tests-pending labels May 4, 2022

cmsbuild added tests-approved and removed tests-started labels May 4, 2022

Dr15Jones suggested changes May 5, 2022

View reviewed changes

makortel added 2 commits May 9, 2022 22:18

Add overrideGUID option to PoolOutputModule

de84af7

Limit the GUID override to the first output file only

42dc4a6

makortel force-pushed the overrideGUID branch from a2b9ee4 to 42dc4a6 Compare May 9, 2022 20:19

cmsbuild added code-checks-pending tests-pending and removed tests-approved code-checks-approved labels May 9, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 9, 2022

cmsbuild added tests-started and removed tests-pending labels May 9, 2022

cmsbuild added tests-rejected and removed tests-started labels May 10, 2022

AdrianoDee mentioned this pull request May 10, 2022

L1T phase-2: particle flow and correlator updates (code reorganized) #37536

Closed

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels May 10, 2022

cmsbuild added orp-approved and removed orp-pending labels May 11, 2022

cmsbuild merged commit 89e3d94 into cms-sw:master May 11, 2022

This was referenced May 11, 2022

DeepSC algorithm for ECAL SuperClusters #37115

Merged

Changes to enable new DeepTauId trainig #37892

Merged

makortel deleted the overrideGUID branch July 12, 2022 16:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add overrideGUID option to PoolOutputModule #37806

Add overrideGUID option to PoolOutputModule #37806

makortel commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

makortel commented May 4, 2022

makortel commented May 4, 2022

cmsbuild commented May 4, 2022

Dr15Jones May 5, 2022

cmsbuild commented May 9, 2022

cmsbuild commented May 9, 2022

makortel commented May 9, 2022

cmsbuild commented May 10, 2022

makortel commented May 10, 2022

makortel commented May 10, 2022

cmsbuild commented May 10, 2022

qliphy commented May 11, 2022

qliphy commented May 11, 2022

Add overrideGUID option to PoolOutputModule #37806

Add overrideGUID option to PoolOutputModule #37806

Conversation

makortel commented May 4, 2022

PR description:

PR validation:

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

makortel commented May 4, 2022

makortel commented May 4, 2022

cmsbuild commented May 4, 2022

Comparison Summary

Dr15Jones May 5, 2022

Choose a reason for hiding this comment

cmsbuild commented May 9, 2022

cmsbuild commented May 9, 2022

makortel commented May 9, 2022

cmsbuild commented May 10, 2022

Unit Tests

Comparison Summary

makortel commented May 10, 2022

makortel commented May 10, 2022

cmsbuild commented May 10, 2022

qliphy commented May 11, 2022

qliphy commented May 11, 2022