Improvements to APE tool: Usability and error fixes

[mteroerd]

This commit includes changes that were found neccessary while doing a multi-IOV APE measurement for the ReReco campaign. The changes include:

1. Reduced verbosity of some modules ( b467869 5b714bb )
   This makes the stderr files procuded by plugins/ApeEstimator.cc shorter and consume disk less space (was: 9mb per file per iteration per measurement). Now, these files can be searched for fatal errors that would not be caught otherwise, which helps tracing back job failures. test/cfgTemplate/apeLocalSetting_cfg.py produced a lot of unneccessary output which flooded the terminal.

2. Improved skimming tool ( fae3000 ccd3aca )
   Fixed a few errors. Also made starting a large number of skims easier.

3. More natural configuration of the automated submission tool, improved error detection ( 0357460 )
   The way that a config file is built was changed. The input file name definition is now more dynamic and easier. Finished or failed measurements are now logged. One can now specify the name of the file in which the current measurements are stored in case on wants to run multiple instances of autoSubmitter and cancel/resume any of them.

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-24448/6280

[cmsbuild]

A new Pull Request was created by @mteroerd for master.

It involves the following packages:

Alignment/APEEstimation

@arunhep, @tocheng, @cmsbuild, @franzoni, @pohsun, @lpernie can you please review it and eventually sign? Thanks.
@mschrode, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[tocheng]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/30233/console Started: 2018/09/03 18:44

[cmsbuild]

+1
Tested at: ccd3aca
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-24448/30233/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-24448/30233/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 32
• DQMHistoTests: Total histograms compared: 3143879
• DQMHistoTests: Total failures: 2
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3143680
• DQMHistoTests: Total skipped: 197
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
• Checked 133 log files, 14 edm output root files, 32 DQM output files

[tocheng]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

+1