Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Luminosity producer using files from brilcalc #29271

Merged
merged 7 commits into from Mar 27, 2020

Conversation

plujan
Copy link
Contributor

@plujan plujan commented Mar 23, 2020

PR description:

This adds a new producer, LumiProducerFromBrilcalc, which will produce luminosity information in CMSSW using a csv file produced by brilcalc.

PR validation:

The included TestLumiProducerFromBrilcalc has been created and run to verify that it works properly (although it does not work as an automated test).

I have also checked the steps on the PR checklist and added some documentation in README.md.

Notes:

  • As discussed in the meeting, I will investigate using the conditions DB for bunch-by-bunch luminosity, but for total luminosity, I think including the file will be the simplest solution to make this available quickly.
  • As mentioned, this produces the luminosity information in the Event, although (by construction) it will always be the same for all events in a single luminosity section.

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29271/14322

  • This PR adds an extra 24KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @plujan (Paul Lujan) for master.

It involves the following packages:

RecoLuminosity/LumiProducer

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor

slava77 commented Mar 23, 2020

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 23, 2020

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5325/console Started: 2020/03/23 14:23

@cmsbuild
Copy link
Contributor

+1
Tested at: 321a67c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8c46fe/5325/summary.html
CMSSW: CMSSW_11_1_X_2020-03-23-1100
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8c46fe/5325/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 6 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 2692493
  • DQMHistoTests: Total failures: 33
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2692141
  • DQMHistoTests: Total skipped: 319
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 147 log files, 16 edm output root files, 34 DQM output files

class LumiProducerFromBrilcalc : public edm::stream::EDProducer<> {
public:
explicit LumiProducerFromBrilcalc(const edm::ParameterSet&);
~LumiProducerFromBrilcalc();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default destructor can be used instead

static void fillDescriptions(edm::ConfigurationDescriptions& descriptions);

private:
virtual void beginStream(edm::StreamID) override;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these methods but the produce() one do nothing.
Do they really need to be defined here? Is there any work in progress expected for eventually populating them?
Otherwise, you could simplify and just remove all them

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's just leftover boilerplate from the skeleton, so it can certainly be taken out.

virtual void produce(edm::Event&, const edm::EventSetup&) override;
virtual void endStream() override;
virtual void beginRun(edm::Run const&, edm::EventSetup const&) override;
//virtual void endRun(edm::Run const&, edm::EventSetup const&) override;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above: in particular, please remove the commented out methods, as well as their implementation, if not needed

//virtual void endLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&) override;

// ----------member data ---------------------------
std::string lumiFile_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const


// ----------member data ---------------------------
std::string lumiFile_;
bool throwIfNotFound_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const

// ----------member data ---------------------------
std::string lumiFile_;
bool throwIfNotFound_;
bool doBunchByBunch_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const

std::map<std::pair<int, int>, std::pair<float, float> > lumiData_;
};

//
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the unneeded commented out lines

std::string lumiFile_;
bool throwIfNotFound_;
bool doBunchByBunch_;
std::map<std::pair<int, int>, std::pair<float, float> > lumiData_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cannot be const, since it is initialized in the constructor.
Anyhow, the internal state of the producer is never modified while processing the events: the producer can become a global module, then

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, done (this also required changing the lumiData_[] accesses to lumiData_.at(), to keep const).

void LumiProducerFromBrilcalc::fillDescriptions(edm::ConfigurationDescriptions& descriptions) {
// Allowed parameters
edm::ParameterSetDescription desc;
desc.addUntracked<std::string>("lumiFile");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this is now used only in private analyses, therefore it is mostly a matter of taste whether to define a parameter tracked or untracked.
On the other hand, if this is going to be processed centrally, it will certainly become important to keep track of the lumi file used, or whether the missing runs in it had been given lumi=0 or not.
I think that all these parameters should be tracked, instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, sounds reasonable, changed.

* removed unused stuff from skeleton
* change to global module
* change parameters to tracked
@cmsbuild
Copy link
Contributor

Pull request #29271 was updated. @perrotta, @cmsbuild, @slava77 can you please check and sign again.

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 25, 2020

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5381/console Started: 2020/03/25 17:21

@cmsbuild
Copy link
Contributor

+1
Tested at: ba48f86
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8c46fe/5381/summary.html
CMSSW: CMSSW_11_1_X_2020-03-25-1100
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8c46fe/5381/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 2692110
  • DQMHistoTests: Total failures: 30
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2691761
  • DQMHistoTests: Total skipped: 319
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 147 log files, 16 edm output root files, 34 DQM output files

@perrotta
Copy link
Contributor

@plujan I assume that you checked that with the latest (small) updates the producer still runs as expected: could you please confirm (by running your test job, for example)?

@plujan
Copy link
Contributor Author

plujan commented Mar 26, 2020

Yes, the test job still runs successfully.

Begin processing the 1st record. Run 1, Event 103, LumiSection 2 on stream 0 at 25-Mar-2020 16:24:54.619 CET
Luminosity for 1 LS 2 is 16.4831

Also, I forgot to ask: should I also create a backport of this PR for 10_X?

@perrotta
Copy link
Contributor

+1

  • A new luminosity producer have been implemented, which reads a csv file produced by brilcalc.
  • It has to be used in analyses, not integrated in any central workflow
  • Jenkins tests pass

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

Also, I forgot to ask: should I also create a backport of this PR for 10_X?

I imagine that a backport in 10_6, to be used for the UL analyses, can be useful.
10_2 is currently used for the remaining pp analyses, and it could also profit of it.

All possible backports should be decided based on physics needs, though. Technically, this code automatically satisfies the no-change policy, and it is therefore viable to be backported

@silviodonato
Copy link
Contributor

+1

@plujan
Copy link
Contributor Author

plujan commented Mar 27, 2020

Thanks, OK, I've created backports to 10_2_X and 10_6_X as #29322 and #29323.

@plujan plujan deleted the LumiProducerFromBrilcalc branch March 27, 2020 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants