Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix for Beam Spot txt file mess-up #20102

Merged
merged 4 commits into from Aug 22, 2017
Merged

fix for Beam Spot txt file mess-up #20102

merged 4 commits into from Aug 22, 2017

Conversation

rmanzoni
Copy link
Contributor

@rmanzoni rmanzoni commented Aug 9, 2017

Fix needed to restore the appropriate txt format needed by, in cascade, if I understand correctly: DIP, WBM, scalers, HLT.

It heals a simultaneous attempt to write in the same /nfshome0/dqmpro/BeamMonitorDQM/BeamFitResults.txt
file.

@vanbesien @arunhep @sarafiorendi @sikler

@cmsbuild cmsbuild added this to the CMSSW_9_2_X milestone Aug 9, 2017
@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 9, 2017 •
edited

A new Pull Request was created by @rmanzoni (Riccardo Manzoni) for CMSSW_9_2_X.

It involves the following packages:

Calibration/TkAlCaRecoProducers
RecoVertex/BeamSpotProducer

@perrotta, @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @slava77, @lpernie can you please review it and eventually sign? Thanks.
@ghellwig, @makortel, @felicepantaleo, @GiacomoSguazzoni, @tocheng, @VinInn, @ebrondol, @rovere, @mmusich, @threus, @dgulhan this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

@rmanzoni rmanzoni mentioned this pull request Aug 9, 2017
Merged
@arunhep
Copy link
Contributor

arunhep commented Aug 9, 2017

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 9, 2017 •
edited

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22178/console Started: 2017/08/09 18:53

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 9, 2017

+1
Tested at: a5b3d27
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20102/22178/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
d9f7765
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20102/22178/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20102/22178/git-merge-result

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 9, 2017

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 9, 2017

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20102/22178/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 22
  • DQMHistoTests: Total histograms compared: 1791774
  • DQMHistoTests: Total failures: 15019
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 1776589
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 90 log files, 14 edm output root files, 22 DQM output files

@arunhep
Copy link
Contributor

arunhep commented Aug 9, 2017

please see the comment #20103 (comment)

@davidlange6 we should have this PR integrated in next 92X release or patch (whatever is coming earlier)

@arunhep
Copy link
Contributor

arunhep commented Aug 9, 2017

+1

@slava77
Copy link
Contributor

slava77 commented Aug 9, 2017

backport of #20103

slava77
slava77 reviewed Aug 9, 2017
View reviewed changes
Calibration/TkAlCaRecoProducers/plugins/AlcaBeamSpotHarvester.cc
@@ -155,7 +157,7 @@ void AlcaBeamSpotHarvester::endRun(const edm::Run& iRun, const edm::EventSetup&)
//poolDbService->appendSinceTime<BeamSpotObjects>(aBeamSpot, poolDbService->currentTime(),"BeamSpotObjectsRcd");
poolDbService->writeOne<BeamSpotObjects>(aBeamSpot, thisIOV, outputrecordName_);
if (dumpTxt_ && beamSpotOutputBase_ == "lumibased"){
beamspot::dumpBeamSpotTxt(outTxt, true, currentBS);
beamspot::dumpBeamSpotTxt(outFile, currentBS);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm lost a bit in which set of changes provides a fix.
The PR description mentions a problem of overwriting a file.
This change, if the outTxt existed already will overwrite now, while it was going to append in the previous version.
this seems to contradict the PR description

Copy link
Contributor Author

@rmanzoni rmanzoni Aug 10, 2017 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the origin of the problem comes from duplicating the file handling in the printer function, here
https://github.com/cms-sw/cmssw/blob/master/RecoVertex/BeamSpotProducer/interface/BeamSpotWrite2Txt.h#L22-L27

while this is already done here
https://github.com/cms-sw/cmssw/blob/master/RecoVertex/BeamSpotProducer/src/BeamFitter.cc#L636-L639

==============

As long as the outfile is used in append mode, everything is normal.

The problem showed up because, when the append option is not enabled, and this only happens for DIP, the outfile is first open here
https://github.com/cms-sw/cmssw/blob/master/RecoVertex/BeamSpotProducer/src/BeamFitter.cc#L636-L639

then reopened by the printer function here
https://github.com/cms-sw/cmssw/blob/master/RecoVertex/BeamSpotProducer/interface/BeamSpotWrite2Txt.h#L22-L27
the beam spot fit results are written, and then the file is closed here
https://github.com/cms-sw/cmssw/blob/master/RecoVertex/BeamSpotProducer/interface/BeamSpotWrite2Txt.h#L55
but then, back in the main BeamFitter.cc file, there's a concurrent attempt writing into the same file here
https://github.com/cms-sw/cmssw/blob/master/RecoVertex/BeamSpotProducer/src/BeamFitter.cc#L690-L701

Now, the idea is that the printer function does not need to handle the creation/closure and write mode of the outfile and let the code that calls it do it.

In this PRs the printer function receives a reference to the outfile
https://github.com/rmanzoni/cmssw/blob/a5b3d276baeaf1960a577b893410cb9f09842025/RecoVertex/BeamSpotProducer/interface/BeamSpotWrite2Txt.h#L20
and all it does is it writes.
The open/close file handling is delegated to the code that calls this function

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is clear now.

@cmsbuild
Copy link
Contributor

Pull request #20102 was updated. @perrotta, @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @slava77, @lpernie can you please check and sign again.

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 10, 2017 •
edited

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22187/console Started: 2017/08/10 11:28

@cmsbuild
Copy link
Contributor

+1
Tested at: 8c61497
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20102/22187/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
d9f7765
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20102/22187/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20102/22187/git-merge-result

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20102/22187/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 22
  • DQMHistoTests: Total histograms compared: 1791808
  • DQMHistoTests: Total failures: 15030
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 1776612
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 90 log files, 14 edm output root files, 22 DQM output files

@slava77
Copy link
Contributor

slava77 commented Aug 10, 2017

+1

for #20102 8c61497

  • the same changes as in 93X/master version in fix for Beam Spot txt file mess-up #20103 7ef3a00. to fix text file overwrite (and corruption?) in BeamMonitorDQM workflow online
  • jenkins tests pass and comparisons with baseline show no differences

@lpernie
Copy link
Contributor

lpernie commented Aug 11, 2017

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_9_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_9_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2)

@davidlange6
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 00000ba into cms-sw:CMSSW_9_2_X Aug 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@perrotta perrotta perrotta left review comments

@slava77 slava77 slava77 left review comments

Assignees
No one assigned
Labels
alca-approved backport-ok comparison-available fully-signed orp-approved reconstruction-approved tests-approved
Projects
None yet
Milestone
CMSSW_9_2_X
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@rmanzoni @cmsbuild @arunhep @slava77 @perrotta @lpernie @davidlange6