Added scripts for on-the-fly log file compression and reading.

[dmitrijus]

Added scripts for on-the-fly log file compression (ztee.py) and reading (ztail.py).

DQMMonitoringService now outputs per-ls stats, improved event rate calculation.

Refactored esMonitoring.py to pass environment variable (including GZIP_LOG).
And added on-fly-compression option.

I had to remove -Werror=unused-variable, otherwise build fails for this package:
there is an unused variable in boost include.

[cmsbuild]

A new Pull Request was created by @dmitrijus (Dmitrijus) for CMSSW_7_4_X.

Added scripts for on-the-fly log file compression and reading.

It involves the following packages:

DQMServices/StreamerIO

@cmsbuild, @danduggan, @nclopezo, @deguio can you please review it and eventually sign? Thanks.
@barvic this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

[deguio]

I know this one was tested on the DQM playback at P5.
was it submitted for 75 as well?

[dmitrijus]

I will make port to 75 after we finish migration at P5.

[ghost]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.

[cmsbuild]

+1
Tested at: 04397d9
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9332/5204/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9332/10023/summary.html

There are some workflows for which there are errors in the baseline:
50202.0 step 3
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_4_X IBs unless changes (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

[davidlange6]

@dmitrijus - why is this needed?

[dmitrijus]

Hi David,

because otherwise the build fails in a boost include:
/opt/offline/slc6_amd64_gcc491/external/boost/1.51.0-cms2/include/boost/asio/error.hpp:244:45: error: 'boost::asio::error::system_category' defined but not used [-Werror=unused-variable]
static const boost::system::error_category& system_category

Dmitrijus

[davidlange6]

+1