Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated root to tip of branch v6-12-00-patches #4775

Merged
merged 1 commit into from Mar 14, 2019
Merged

Updated root to tip of branch v6-12-00-patches #4775

merged 1 commit into from Mar 14, 2019

Conversation

smuzaffar
Copy link
Contributor

No description provided.

@smuzaffar
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 13, 2019 •
edited

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/33560/console

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_10_6_X/gcc700.

@cmsbuild, @smuzaffar, @gudrutis, @mrodozov can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

@smuzaffar smuzaffar mentioned this pull request Mar 13, 2019
Closed
@cmsbuild
Copy link
Contributor

-1

Tested at: 0bdf673

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4775/33560/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

  • Unit Tests:

I found errors in the following unit tests:

---> test runtestPhysicsToolsPatAlgos had ERRORS

@cmsbuild
Copy link
Contributor

Comparison job queued.

@fabiocos
Copy link
Contributor 

13-Mar-2019 15:01:25 CET  Initiating request to open file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_2_2/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v1-v1/10000/8E7EE25F-294E-E711-A5CC-0025905B8610.root
13-Mar-2019 15:01:29 CET  Successfully opened file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_2_2/RelValTTbar_13/MINIAODSIM/PU25ns_92X_upgrade2017_realistic_v1-v1/10000/8E7EE25F-294E-E711-A5CC-0025905B8610.root
Begin processing the 1st record. Run 1, Event 54, LumiSection 2 on stream 0 at 13-Mar-2019 15:01:41.093 CET


A fatal system signal has occurred: segmentation violation
The following is the call stack containing the origin of the signal.

Wed Mar 13 15:01:42 CET 2019

Current Modules:

Module: PoolOutputModule:out (crashed)

A fatal system signal has occurred: segmentation violation
/build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_6_X_2019-03-12-1100/src/PhysicsTools/PatAlgos/test/runtests.sh: line 42: 12935 Segmentation fault      (core dumped) cmsRun ${LOCAL_TEST_DIR}/patTuple_updateMet_fromMiniAOD_cfg.py
Failure using patTuple_updateMet_fromMiniAOD_cfg.py: status 139
status = 35584

---> test runtestPhysicsToolsPatAlgos had ERRORS

@smuzaffar @Dr15Jones apparently this update does not work out of the box. I wonder whether it is reasonable to move forward and build 10_6_0_pre2 with what we have now, so far we have seen a failure only in one unit test. Otherwise we need to delay it further together with the validation. In this case perhaps it is better to temporarily revert cms-sw/#26053 , that apparently triggers this issue, build 10_6_0_pre2 to allow the validation of all the rest, and then go back to merge the PR and test further ROOT

@Dr15Jones
Copy link

I vote to just go forward with the one unit test failure and not do the revert.

@fabiocos
Copy link
Contributor

@Dr15Jones we have so far no evidence of other issues after several IBs. I am not so much concerned about the single unit test, that in any case needs to be fixed, but it is probably not crucial in itself for the pre-release test, but whether this could be the hint of a deeper problem.

@fabiocos
Copy link
Contributor

BTW, I see that the Validation/Geometry test is now running ok, so that problem looks fixed, but apparently there is another one, so it isn't clear that this latest version is a priori better than what we have so far in absolute terms, more checks are needed I would say

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4775/33560/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 197 differences found in the comparisons
  • DQMHistoTests: Total files compared: 32
  • DQMHistoTests: Total histograms compared: 3114829
  • DQMHistoTests: Total failures: 15
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3114617
  • DQMHistoTests: Total skipped: 197
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
  • Checked 133 log files, 14 edm output root files, 32 DQM output files

@smuzaffar smuzaffar merged commit 34d9daa into cms-sw:IB/CMSSW_10_6_X/gcc700 Mar 14, 2019
@slava77
Copy link
Contributor

slava77 commented Mar 14, 2019

Reco comparison results: 197 differences found in the comparisons

It looks like a similar issue reported in cms-sw/cmssw#25919 (comment)

why was this merged ???
It looks like the second attempt to ignore problems and still merge, similar to what was done on Oct 12.

Looking at the past merges:

I'd like to repeat my message from Oct 17:
#4417 (comment)

@smuzaffar
Copy link
Contributor Author

@slava77 , my bad. I though these 15 failure are standard failures we get while testing root.
I will revert it for next IB.

@slava77
Copy link
Contributor

slava77 commented Mar 14, 2019

@slava77 , my bad. I though these 15 failure are standard failures we get while testing root.
I will revert it for next IB.

Thanks.
BTW, the last time the crash in runtestPhysicsToolsPatAlgos also coincided with all validate.C used in the reco comparisons failing/crashing and prevented any practical non-trivial signoff by reco.

@smuzaffar smuzaffar mentioned this pull request Mar 14, 2019
Merged
@fabiocos
Copy link
Contributor

I preferred not to rush for pre2 to the newest version, and keep the previous one (we have an issue with unit tests there as well, but we saw stability in RelVals across several IBs)

@slava77
Copy link
Contributor

slava77 commented Mar 14, 2019

I preferred not to rush for pre2 to the newest version, and keep the previous one (we have an issue with unit tests there as well, but we saw stability in RelVals across several IBs)

It looks like with this measured approach a bigger disaster of this entering also in pre2 was averted.
Nice.

@smuzaffar
Copy link
Contributor Author

This is reverted by #4782. Today's 11h00 IB has been restarted to pick up the change.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4775/33560/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 197 differences found in the comparisons
  • DQMHistoTests: Total files compared: 32
  • DQMHistoTests: Total histograms compared: 3114829
  • DQMHistoTests: Total failures: 15
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3114617
  • DQMHistoTests: Total skipped: 197
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
  • Checked 133 log files, 14 edm output root files, 32 DQM output files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
externals-pending orp-pending pending-signatures tests-rejected
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@smuzaffar @cmsbuild @fabiocos @Dr15Jones @slava77