Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use "best matching" EDAlias in module dependence checks when placing ConditionalTask modules in Path #39409

Merged

Conversation

makortel
Copy link
Contributor

@makortel makortel commented Sep 16, 2022

PR description:

This PR fixes the problem reported in #36938 (comment). If the EDAlias has wildcards for one module, and more specific aliases for another module, we should use the one that has the most constrained match. There is one ambiguous case left, for which an exception is thrown. I hope we won't encounter that situation in practice.

The problem was encountered with a SwitchProducer having the EDAlias case, but I think the problem could happen also without SwitchProducer.

PR validation:

Framework unit test passed in 12_4_8 (where I developed this change originally)

@cmsbuild
Copy link
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39409/32120

ERROR: Build errors found during clang-tidy run.

FWCore/Framework/src/StreamSchedule.cc:655:31: error: no matching function for call to 'findBestMatchingAlias' [clang-diagnostic-error]
            auto foundAlias = findBestMatchingAlias(conditionalModuleBranches, aliasMap, productModuleLabel, ci);
                              ^
FWCore/Framework/src/StreamSchedule.cc:162:32: note: candidate function not viable: no known conversion from 'const std::unordered_multimap<std::string, const edm::BranchDescription *>' (aka 'const unordered_multimap<basic_string<char>, const edm::BranchDescription *>') to 'const std::multimap<std::string, const edm::BranchDescription *>' (aka 'const multimap<basic_string<char>, const edm::BranchDescription *>') for 1st argument
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:129: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39409/32121

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

  • FWCore/Framework (core)
  • FWCore/Integration (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@missirol, @wddgit this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@makortel
Copy link
Contributor Author

@cmsbuild, please test

@makortel
Copy link
Contributor Author

@Dr15Jones please check

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3d54ae/27585/summary.html
COMMIT: cf5e630
CMSSW: CMSSW_12_6_X_2022-09-15-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39409/27585/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test-das-selected-lumis had ERRORS

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 3618332
  • DQMHistoTests: Total failures: 5
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3618305
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
  • Checked 212 log files, 49 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

@makortel
Copy link
Contributor Author

The unit test failure is

===== Test "test-das-selected-lumis" ====
+ export CMS_BOT_USE_DASGOCLIENT=true
+ CMS_BOT_USE_DASGOCLIENT=true
+ QUERY='lumi,file dataset=/HIHardProbes/HIRun2018A-v1/RAW run=326479'
+ dasgoclient --limit 0 --query 'lumi,file dataset=/HIHardProbes/HIRun2018A-v1/RAW run=326479' --format json
+ das-selected-lumis.py 1,23
+ grep '^/store/'
2022/09/16 04:42:30 wrong fileLumi record

---> test test-das-selected-lumis had ERRORS

and is unrelated to this PR.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39409/32292

  • This PR adds an extra 40KB to repository

@cmsbuild
Copy link
Contributor

Pull request #39409 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

@makortel
Copy link
Contributor Author

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3d54ae/27820/summary.html
COMMIT: 5b46812
CMSSW: CMSSW_12_6_X_2022-09-28-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39409/27820/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 3624368
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3624340
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
  • Checked 212 log files, 49 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

@makortel
Copy link
Contributor Author

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@makortel
Copy link
Contributor Author

type bug-fix

@perrotta
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 53e38c4 into cms-sw:master Sep 28, 2022
@makortel makortel deleted the fixSwitchProducerEDAliasConditionalTask branch September 28, 2022 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants