Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

transferSystem PSet support and other DAQ updates (74X) #7991

Merged
merged 15 commits into from Mar 2, 2015
Merged

transferSystem PSet support and other DAQ updates (74X) #7991

merged 15 commits into from Mar 2, 2015

Conversation

smorovic
Copy link
Contributor

74X PR identical to changes pushed for 73X in #7990

changes:

  • beginning of lumisection files written locally by FUs to indicate opening new LS from the input source to other DAQ components (introduced to create backpressure in case of delay in LS processing)
  • Each Ini file checksum is verified after writing to disk and before renaming into location where it is taken by hltd
  • transfer system PSet and selected destination support. Selected destination is passed as VarArgs parameter. A parameter is added to control whether exception is thrown when PSet is incomplete or parameter missing. New output json field is added to output module as well as JSON and DQMHistograms modules
  • fixed min. number of run digits in filenames used by TriggerJSONMonitoring (only relevant for test run numbers smaller than 100000).
  • Fake BU and Raw writer improvements:
  •    FED 1024 is handled in Raw writer

  •    closing last LS written when interrupted from shell

  •    updated schema of EoLS and Eor jsd/json

  •    ability to copy menu directory when starting a run by script

  •    fffParameters.jsn file handling

@smorovic
fake BU support for copying existing directory with HLT menu and othe…
953ca50 
…r parameters propagated through BU
@smorovic
creation of BoLS file
39486ca
@smorovic
ini file checksum: output module reads back and verifies checksum of the
7272709 
ini file. then file is moved from open subdirectory to the local run
directory
@smorovic
sleep and repeat check for EoL two times in case EoR appears without …
e18409a 
…EoL for current lumisection
@smorovic
nFeds can now be beyond 1024 in fake BU output module
1f31f39
@smorovic
*moving new fffparameters.jsn file
023beeb 
*updating to latest json metafile definitions as in BU
@smorovic
output TRANSFERS_PSET.jsn from the EvFDaqDirector service
6920cf7
@smorovic
*writing transfers pset file for all streams produced by CMSSW
b253304
@smorovic
std::string --> std::vstring (to have list of destinations per each
33aeda2 
stream and mode)
@smorovic
transfer destination added to output json for all streams (except str…
800e1c0 
…eamError which is external)
@smorovic
*passing selected transfer mode as untracked parameter
bce6dc9 
*updated cfi definition of EvFDaqDirector
*fixed stream destinations call in DQMFileSaver
@smorovic
fixed iSummary variable name (not iSummary_)
775e22d
@smorovic
*DQMFileSaver fixes:
7bdbd2c 
 - beginRun is const, so we must read transfer system string in beginJob
   (also could be done from constructor)
 - fillJson is static, so we pass transfer system string as parameter. Also adapted TimeoutPoolOutputModule to this.
@smorovic
*can not get process Pset in constructor of the service. Subsequently…
e158a09 
… also can

not get anything from constructor of the output module. beginJob is
at the moment safe to override because StreamerOutputModuleBase does not do anything
at that time, so moving the call there.
@smorovic
*removed unused functions constructing the transfer file path
79118e8 
*fake BU writing EoLS file after interrupted by signal
*supervisor thread makes a check for additional files when run is ended, delayed by 0.1 seconds.
  This makes sure that new raw data is not present. Also this slightly postpones a check
  for EoLS file in ramdisk in case it does not become visible before EoR file (when their timestamps are very close)
@cmsbuild cmsbuild added this to the Next CMSSW_7_4_X milestone Feb 27, 2015
@cmsbuild
Copy link
Contributor

A new Pull Request was created by @smorovic (Srecko Morovic) for CMSSW_7_4_X.

transferSystem PSet support and other DAQ updates (74X)

It involves the following packages:

DQMServices/Components
DQMServices/StreamerIO
EventFilter/Utilities
HLTrigger/JSONMonitoring

@perrotta, @cmsbuild, @nclopezo, @danduggan, @emeschi, @Martin-Grunewald, @deguio, @fwyzard, @mommsen can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @barvic this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

@fwyzard
Copy link
Contributor

fwyzard commented Feb 27, 2015 via email

@Martin-Grunewald
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

The tests are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 2, 2015

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_4_X IBs unless changes (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @ktf, @smuzaffar

@davidlange6
Copy link
Contributor

+1

cmsbuild added a commit that referenced this pull request Mar 2, 2015
@cmsbuild
Merge pull request #7991 from smorovic/daqupdates+transfersPSet-74X
2217e49 
transferSystem PSet support and other DAQ updates (74X)
@cmsbuild cmsbuild merged commit 2217e49 into cms-sw:CMSSW_7_4_X Mar 2, 2015
@smorovic smorovic deleted the daqupdates+transfersPSet-74X branch May 19, 2015 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comparison-available daq-approved dqm-approved fully-signed hlt-approved tests-approved
Projects
None yet
Milestone
Next CMSSW_7_4_X
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@smorovic @cmsbuild @fwyzard @Martin-Grunewald @deguio @emeschi @davidlange6