Adding merging type (DAQ) and changing defaults

[smorovic]

• Merge type parameter added to output JSON to reflect difference between streams created by different output modules (different file format). This will be used to display streams differently in monitoring. Optionally, a PSet name can be specified and, if found in the configuration, will be parsed for per-stream overrides of this parameter (can be used e.g. if more specific differentiation of DAT streams is required for monitoring or merging).
• by default activate empty-LS mode and disable micro-merging in CMSSW. This has been already configured as such in production HLT menus for some time.

[cmsbuild]

A new Pull Request was created by @smorovic (Srecko Morovic) for CMSSW_8_0_X.

It involves the following packages:

DQMServices/Components
EventFilter/Utilities
HLTrigger/JSONMonitoring

@perrotta, @cmsbuild, @cvuosalo, @fwyzard, @emeschi, @dmitrijus, @Martin-Grunewald, @deguio, @slava77, @mommsen, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @barvic this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

[mommsen]

Hi Srecko,

I would propose to define the default values for the different stream types in EvFDaqDirector::getStreamMergeType instead of returning an empty string and have the default (string) values scattered over different places.

Remi

[dmitrijus]

Hi Srecko,

I am trying to deprecate DQMFileSaver in favour of DQMFileSaverPB. So any changes done to former should go to the latter too:
https://github.com/cms-sw/cmssw/blob/CMSSW_8_1_X/DQMServices/FileIO/plugins/DQMFileSaverPB.cc#L99

Also, DQMFileSaver::fillJson is used here:
https://github.com/cms-sw/cmssw/blob/CMSSW_8_1_X/DQMServices/StreamerIO/plugins/JsonWritingTimeoutPoolOutputModule.cc#L42

(Used by Event Display stream written by the DQM)

Dmitrijus

[cmsbuild]

Pull request #14528 was updated. @perrotta, @cmsbuild, @cvuosalo, @fwyzard, @emeschi, @dmitrijus, @Martin-Grunewald, @deguio, @slava77, @mommsen, @vanbesien, @davidlange6 can you please check and sign again.

[smorovic]

Hi Remi,
I switched to using enums, with string values defined in static const vector in EvFDaqDirector. Modules are providing default enums which they want to use in case internal map doesn't contain their stream type.

Dmitrijus,
I'll add it to your new module.
Good point about JsonWritingTimeoutPoolOutputModule.cc. This module is not in HLT and doesn't use EvFDaqDirector. So I'll just put hardcoded "ROOT" there.

[cmsbuild]

Pull request #14528 was updated. @perrotta, @cmsbuild, @cvuosalo, @fwyzard, @emeschi, @dmitrijus, @Martin-Grunewald, @deguio, @slava77, @mommsen, @vanbesien, @davidlange6 can you please check and sign again.

[mommsen]

+1
Thanks @smorovic for implementing the change.

[cmsbuild]

The tests are being triggered in jenkins.

[dmitrijus]

+1

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/13018/console

[mommsen]

+1

[cmsbuild]

+1
Tested at: 02ebd55
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14528/13018/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14528/13018/summary.html

[Martin-Grunewald]

+1
PR for 81X?

[smorovic]

Created PR for 81X (#14543)

[slava77]

+1

for #14528 02ebd55

• offline reco is not affected

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

[dmitrijus]

Hi @smorovic,

do we need to apply this update to our (DQM) cluster, in order not to break the EventDisplay stream?
If so, when? I can imagine applying it too soon might break things too - these are positionals in a json array.

Dmitrijus

[smorovic]

Hi @dmitrijus, as we already discussed in private mail thread, this doesn't need to be synchronized (merger will handle both JSON formats), so DQM can apply it independently of HLT.