standardize implementation and offsets for special tracking workflows

[kpedro88]

In #18503 I tried to standardize the implementation of "special" workflows, but only propagated this to Phase2, ignoring Phase1.

This led to some offset numbers being used inconsistently between Phase1 and Phase2, as well as novel code in relval_2017 (which I would prefer to keep as a strict forwarding file to pass workflow numbers between the upgrade matrix and regular matrix).

I've moved all the special tracking workflows from relval_2017 into the new approach (all definitions/conditions in upgradeWorkflowComponents, relval_steps, and relval_upgrade). I checked the cmsDriver commands before and after the change using runTheMatrix.py --dryRun to make sure nothing was unintentionally altered.

In the process, I fixed a bug that the 2018 tracking workflows were actually using 2017 steps. I also changed the offset numbers for the Phase 2 timing and neutron workflows to be separate from the tracking workflows.

[cmsbuild]

The code-checks are being triggered in jenkins.

[kpedro88]

@makortel please take a look

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-21932/3000

[cmsbuild]

A new Pull Request was created by @kpedro88 (Kevin Pedro) for master.

It involves the following packages:

Configuration/PyReleaseValidation

@cmsbuild, @GurpreetSinghChahal, @prebello, @kpedro88, @fabozzi can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @ebrondol, @ghellwig, @Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[kpedro88]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/25582/console Started: 2018/01/23 20:46

[kpedro88]

please abort

[kpedro88]

please abort

[cmsbuild]

Jenkins tests are aborted.

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-21932/3001

[cmsbuild]

Pull request #21932 was updated. @cmsbuild, @GurpreetSinghChahal, @prebello, @kpedro88, @fabozzi can you please check and sign again.

[kpedro88]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/25583/console Started: 2018/01/23 20:52

[cmsbuild]

+1
Tested at: d679980
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21932/25583/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21932/25583/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-21932/1325.7_TTbar_13_94XNanoAODINPUT+TTbar_13_94XNanoAODINPUT+NANOEDMMC2017

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 26
• DQMHistoTests: Total histograms compared: 2467599
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2467429
• DQMHistoTests: Total skipped: 169
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 1.0999999999 KiB( 22 files compared)
• Checked 110 log files, 9 edm output root files, 26 DQM output files

[kpedro88]

@makortel ping

[fabozzi]

+1

[makortel]

@kpedro88 Sorry for the delay. Looks good to me (as far as I could understand by just reading). Thanks for the generalization.

[kpedro88]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

+1