Restructure validation scripts

[rovere]

No description provided.

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19190/console Started: 2017/04/15 06:58

[cmsbuild]

+1
Tested at: 93e278e
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18365/19190/summary.html

[cmsbuild]

Comparison job queued.

[ianna]

+1

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18365/19190/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 1612 differences found in the comparisons
• DQMHistoTests: Total files compared: 23
• DQMHistoTests: Total histograms compared: 1819408
• DQMHistoTests: Total failures: 25472
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 1793763
• DQMHistoTests: Total skipped: 173
• DQMHistoTests: Total Missing objects: 0
• Checked 94 log files, 14 edm output root files, 23 DQM output files

[dmitrijus]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @Muzaffar, @davidlange6, @smuzaffar

[davidlange6]

+1

[smuzaffar]

@rovere @ianna , looks like this PR broke the Material Budget IB tests https://cmssdt.cern.ch/jenkins/job/ib-run-material-budget/681/console
Running locally I see

cmsRun /cvmfs/cms-ib.cern.ch/week1/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_1_X_2017-04-29-1100/src/Validation/Geometry/test/runP_Tracker_cfg.py
----- Begin Fatal Exception 02-May-2017 14:56:50 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named /cvmfs/cms-ib.cern.ch/week1/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_1_X_2017-04-29-1100/src/Validation/Geometry/test/runP_Tracker_cfg.py
Exception Message:
python encountered the error: <type 'exceptions.ImportError'>
No module named plot_utils
----- End Fatal Exception -------------------------------------------------

reason is that the plot_utils.py is not in PYTHONPATH. @rovere , may be move Validation/Geometry/test/plot_utils.py to Validation/Geometry/python/plot_utils.py

[rovere]

I'll make a separate PR soonish. Thanks for reporting.

[smuzaffar]

@rovere , looks like Validation/Geometry/test/MaterialBudget.C is also needed by material budget tests

[rovere]

@ianna @smuzaffar
May I suggest having exact instructions about what is actually run at PR integration? Otherwise, we can go back and forth forever, wasting time uselessly.
Exact commands, environments, etc, would be really appreciated. It could also be the case that those commands have to be updated....

[smuzaffar]

this is what we run https://github.com/cms-sw/cms-bot/blob/master/run-material-budget
just create a CMSSW 91X dev area , set the env and run the above script.

For comparison we run https://github.com/cms-sw/cms-bot/blob/master/compare-material-budget

[rovere]

@smuzaffar @ianna
Hopefully this cms-sw/cms-bot#839 will fix all issues.

[smuzaffar]

thanks @rovere . cms-bot changes worked.