Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Transformer ability to modules #38454

Merged
merged 3 commits into from Sep 10, 2022
Merged

Conversation

Dr15Jones
Copy link
Contributor

PR description:

Added ability to register tranform methods to modules. This allows a module to have a method which takes a data product already put into the Event by the same module and then create a new data product from it which is also put into the Event.

The plan is to use this ability to implement an automatic conversion from GPU to CPU data products.

PR validation:

Code compiles and new unit test passes.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38454/30666

  • This PR adds an extra 200KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

  • DataFormats/Provenance (core)
  • FWCore/Framework (core)
  • FWCore/Integration (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@makortel, @wddgit, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@Dr15Jones
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

-1

Failed Tests: Build ClangBuild
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-49175f/25673/summary.html
COMMIT: 7346305
CMSSW: CMSSW_12_5_X_2022-06-21-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38454/25673/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-49175f/25673/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-49175f/25673/git-merge-result

Build

I found compilation error when building:

>> Leaving Package AnalysisAlgos/SiStripClusterInfoProducer
>> Package AnalysisAlgos/SiStripClusterInfoProducer built
Entering library rule at src/AnalysisAlgos/SiStripClusterInfoProducer/plugins
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-06-21-1100/src/AnalysisAlgos/SiStripClusterInfoProducer/plugins/SiStripProcessedRawDigiProducer.cc
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-06-21-1100/src/AnalysisAlgos/SiStripClusterInfoProducer/plugins/SiStripProcessedRawDigiProducer.cc: In member function 'void SiStripProcessedRawDigiProducer::pr_process(const edm::DetSetVector&, edm::DetSetVector&, const SiStripGain&)':
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-06-21-1100/src/AnalysisAlgos/SiStripClusterInfoProducer/plugins/SiStripProcessedRawDigiProducer.cc:126:5: error: 'virtual void edm::stream::EDProducerBase::transform(std::size_t, edm::EventForTransformer&) const' is private within this context
  126 |     transform(
      |     ^~~~~~~~~
In file included from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-06-21-1100/src/FWCore/Framework/interface/stream/implementors.h:31,
                 from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-06-21-1100/src/FWCore/Framework/interface/stream/AbilityToImplementor.h:25,
                 from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-06-21-1100/src/FWCore/Framework/interface/stream/EDProducer.h:22,


Clang Build

I found compilation error while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

>> Entering Package RecoMuon/MuonSeedGenerator
>> Entering Package RecoMuon/StandAloneMuonProducer
>> Entering Package SimG4Core/Application
>> Entering Package SimGeneral/MixingModule
>> Compile sequence completed for CMSSW CMSSW_12_5_X_2022-06-21-1100
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 1
+ eval scram build outputlog '&&' '(python3' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/cms-bot/buildLogAnalyzer.py --ignoreWarning=Wdeprecated-declarations --logDir /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-06-21-1100/tmp/el8_amd64_gcc10/cache/log/src '||' 'true)'
++ scram build outputlog
>> Entering Package DataFormats/Provenance
Entering library rule at DataFormats/Provenance
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-06-21-1100/src/DataFormats/Provenance/src/BranchChildren.cc


@Dr15Jones
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38454/30682

  • This PR adds an extra 20KB to repository

@cmsbuild
Copy link
Contributor

Pull request #38454 was updated. @makortel, @smuzaffar, @Dr15Jones can you please check and sign again.

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 9, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38454/32060

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 9, 2022

Pull request #38454 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

@makortel
Copy link
Contributor

makortel commented Sep 9, 2022

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 9, 2022

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-49175f/27448/summary.html
COMMIT: 0084b15
CMSSW: CMSSW_12_6_X_2022-09-09-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38454/27448/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestIOPoolInputNoParentDictionary had ERRORS

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 10 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 3618326
  • DQMHistoTests: Total failures: 14
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3618290
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
  • Checked 212 log files, 49 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

@makortel
Copy link
Contributor

So TestIOPoolInputNoParentDictionary is failing again with

===== Test "TestIOPoolInputNoParentDictionary" ====
WARNING: In non-interactive mode release checks e.g. deprecated releases, production architectures are disabled.
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_6_X_2022-09-09-1100/TestIOPoolInputNoParentDictionary/CMSSW_12_6_X_2022-09-09-1100/src /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_6_X_2022-09-09-1100/TestIOPoolInputNoParentDictionary
Failed to copy DataFormats/TestObjects from local or release area

---> test TestIOPoolInputNoParentDictionary had ERRORS

while in #38454 (comment) and #38454 (comment) the test succeeded. The failure is not caused by the developments of this PR, but gets somehow triggered based on what packages get checked out and if the IB is patch or full release (CMSSW_12_6_X_2022-09-09-1100 is a patch release). I suggest we go a head with this PR, and investigate this failure mode of TestIOPoolInputNoParentDictionary test separately.

@makortel
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

+1

@perrotta
Copy link
Contributor

merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants