Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drop openssl and use it from system #6797

Merged
merged 2 commits into from May 29, 2021

Conversation

smuzaffar
Copy link
Contributor

@smuzaffar smuzaffar commented Apr 8, 2021

Backported system OpenSSL changes from IB/CMSSW_11_3_X/cc8 branch.
This should be included in CMSSW 12.0.Xand above

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 8, 2021

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_11_3_X/master.

@cmsbuild, @smuzaffar, @mrodozov can you please review it and eventually sign? Thanks.
cms-bot commands are listed here

@smuzaffar
Copy link
Contributor Author

test parameters:

  • full_cmssw = true
  • enable_test = threading

@smuzaffar
Copy link
Contributor Author

please test for slc7_ppc64le_gcc9

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 8, 2021

Pull request #6797 was updated.

@smuzaffar
Copy link
Contributor Author

please test for slc7_ppc64le_gcc9

@smuzaffar
Copy link
Contributor Author

please test

@smuzaffar
Copy link
Contributor Author

please test for slc7_aarch64_gcc9

@smuzaffar
Copy link
Contributor Author

@davidlange6 , as dropping openssl means rebuilt of all python based tools, should we consider updating pip, setuptools (for py3) and python3 itself e.g. python 3.9?

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 8, 2021

-1

Failed Tests: UnitTests RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14102/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_11_3_X_2021-04-07-2300/slc7_ppc64le_gcc9
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6797/14102/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test_PrepareInputDb had ERRORS
---> test test_MpsWorkFlow had ERRORS
---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
and more ...

RelVals

  • 11634.91111634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA/step1_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA.log

@smuzaffar
Copy link
Contributor Author

please test for slc7_ppc64le_gcc9

@smuzaffar
Copy link
Contributor Author

please test for slc7_aarch64_gcc9

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 8, 2021

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14103/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_11_3_X_2021-04-08-1100/slc7_amd64_gcc900
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6797/14103/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 38
  • DQMHistoTests: Total histograms compared: 2865506
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2865483
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
  • Checked 160 log files, 37 edm output root files, 38 DQM output files
  • TriggerResults: no differences found

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 9, 2021

-1

Failed Tests: UnitTests RelVals RelVals-THREADING
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14121/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_11_3_X_2021-04-07-2300/slc7_ppc64le_gcc9
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6797/14121/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14121/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14121/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test test_PrepareInputDb had ERRORS
---> test test_MpsWorkFlow had ERRORS
---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
and more ...

RelVals

  • 11634.91111634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA/step1_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA.log

RelVals-THREADING

  • 11634.91111634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA/step1_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA.log

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 9, 2021

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14122/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_11_3_X_2021-04-08-1100/slc7_aarch64_gcc9
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6797/14122/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14122/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14122/git-merge-result

Build

I found compilation error when building:

/cvmfs/cms-ib.cern.ch/nweek-02675/slc7_aarch64_gcc9/external/gcc/9.3.0/bin/../lib/gcc/aarch64-unknown-linux-gnu/9.3.0/../../../../aarch64-unknown-linux-gnu/bin/ld: tmp/slc7_aarch64_gcc9/src/SimTransport/TotemRPProtonTransportParametrization/src/SimTransportTotemRPProtonTransportParametrization/TMultiDimFet.cc.o: in function `TMultiDimFet::CheckTObjectHashConsistency() const':
TMultiDimFet.cc:(.text._ZNK12TMultiDimFet27CheckTObjectHashConsistencyEv[_ZNK12TMultiDimFet27CheckTObjectHashConsistencyEv]+0x9c): undefined reference to `TMultiDimFet::Class()'
/cvmfs/cms-ib.cern.ch/nweek-02675/slc7_aarch64_gcc9/external/gcc/9.3.0/bin/../lib/gcc/aarch64-unknown-linux-gnu/9.3.0/../../../../aarch64-unknown-linux-gnu/bin/ld: tmp/slc7_aarch64_gcc9/src/SimTransport/TotemRPProtonTransportParametrization/src/SimTransportTotemRPProtonTransportParametrization/TMultiDimFet.cc.o: in function `_GLOBAL__sub_I_TMultiDimFet.cc':
TMultiDimFet.cc:(.text.startup+0x48): undefined reference to `ROOT::GenerateInitInstance(TMultiDimFet const*)'
/cvmfs/cms-ib.cern.ch/nweek-02675/slc7_aarch64_gcc9/external/gcc/9.3.0/bin/../lib/gcc/aarch64-unknown-linux-gnu/9.3.0/../../../../aarch64-unknown-linux-gnu/bin/ld: tmp/slc7_aarch64_gcc9/src/SimTransport/TotemRPProtonTransportParametrization/src/SimTransportTotemRPProtonTransportParametrization/TMultiDimFet.cc.o:(.data.rel.ro._ZTV12TMultiDimFet[_ZTV12TMultiDimFet]+0x1e0): undefined reference to `TMultiDimFet::Streamer(TBuffer&)'
collect2: error: ld returned 1 exit status
gmake: *** [tmp/slc7_aarch64_gcc9/src/SimTransport/TotemRPProtonTransportParametrization/src/SimTransportTotemRPProtonTransportParametrization/libSimTransportTotemRPProtonTransportParametrization.so] Error 1
Leaving library rule at SimTransport/TotemRPProtonTransportParametrization
>> Leaving Package SimTransport/TotemRPProtonTransportParametrization
>> Package SimTransport/TotemRPProtonTransportParametrization built
>> Entering Package CondFormats/SiPixelTransient


@smuzaffar
Copy link
Contributor Author

please test for slc7_aarch64_gcc9

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14149/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_11_3_X_2021-04-09-1100/slc7_aarch64_gcc9
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6797/14149/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
---> test testFWCoreConcurrency had ERRORS
---> test InputTag_t had ERRORS
and more ...

@mrodozov
Copy link
Contributor

please test for slc7_aarch64_gcc9
retesting with the 2 unit test fixed

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests RelVals-THREADING
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14171/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_11_3_X_2021-04-11-2300/slc7_aarch64_gcc9
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/6797/14171/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14171/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14171/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test TestFWCoreServicesDriver had ERRORS
---> test TestRunnerFWCoreTFWLiteSelector had ERRORS
---> test PhiTest had ERRORS
---> test TestConfigDP had ERRORS
and more ...

RelVals-THREADING

  • 10042.010042.0_ZMM_13+2017+ZMM_13TeV_TuneCUETP8M1_GenSimINPUT+Digi+RecoFakeHLT+HARVESTFakeHLT+ALCA+Nano/step3_ZMM_13+2017+ZMM_13TeV_TuneCUETP8M1_GenSimINPUT+Digi+RecoFakeHLT+HARVESTFakeHLT+ALCA+Nano.log
  • 34634.034634.0_TTbar_14TeV+2026D76+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal/step3_TTbar_14TeV+2026D76+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal.log
  • 11634.91111634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA/step3_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA.log

@mrodozov mrodozov changed the base branch from IB/CMSSW_11_3_X/master to IB/CMSSW_12_0_X/master April 15, 2021 15:29
@mrodozov
Copy link
Contributor

please test

@mrodozov
Copy link
Contributor

please test
test it again for 12_0 and amd

@mrodozov
Copy link
Contributor

please test
openloops is fixed now

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14387/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_12_0_X_2021-04-20-2300/slc7_amd64_gcc900
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6797/14387/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14387/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14387/git-merge-result

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 38
  • DQMHistoTests: Total histograms compared: 2877046
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2877023
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
  • Checked 160 log files, 37 edm output root files, 38 DQM output files
  • TriggerResults: no differences found

@mrodozov
Copy link
Contributor

please test for slc7_aarch64_gcc9

@mrodozov
Copy link
Contributor

please test for slc7_ppc64le_gcc9

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14542/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_12_0_X_2021-04-22-2300/slc7_aarch64_gcc9
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6797/14542/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14542/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14542/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test import-yaml had ERRORS
---> test TestFWCoreServicesDriver had ERRORS
---> test PhiTest had ERRORS
---> test TestConfigDP had ERRORS
and more ...

@smuzaffar
Copy link
Contributor Author

please test for slc7_ppc64le_gcc9

@mrodozov
Copy link
Contributor

we are keeping this until pre2 (before I forget)

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests RelVals RelVals-THREADING AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e1aa0/14604/summary.html
COMMIT: 6a751fb
CMSSW: CMSSW_12_0_X_2021-04-26-2300/slc7_ppc64le_gcc9
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/6797/14604/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test_PrepareInputDb had ERRORS
---> test test_MpsWorkFlow had ERRORS
---> test testUnits had ERRORS
---> test testDD4hepFilteredView had ERRORS
and more ...

RelVals

RelVals-THREADING

AddOn Tests

  • unknownAddOnTest might have timed out: FAILED - secs

@smuzaffar
Copy link
Contributor Author

12.0.0.pre2 has been build, Iwould suggest to get this in now

@silviodonato
Copy link
Contributor

@smuzaffar @cms-sw/xpog-l2 has getting troubles with the NanoAOD integration tests.
In the latest test they are getting

>> Compiling  /builds/cms-nanoAOD/nanoAOD-integration/CMSSW_12_0_X_2021-05-31-2300/src/CondCore/CondDB/src/Binary.cc
/builds/cms-nanoAOD/nanoAOD-integration/CMSSW_12_0_X_2021-05-31-2300/src/CondCore/CondDB/src/Binary.cc:8:10: fatal error: openssl/sha.h: No such file or directory
    8 | #include <openssl/sha.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
gmake: *** [config/SCRAM/GMake/Makefile.rules:1694: tmp/slc7_amd64_gcc900/src/CondCore/CondDB/src/CondCoreCondDB/Binary.cc.o] Error 1
gmake: *** Waiting for unfinished jobs....
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

It seems that openssl is not visible anymore.
See https://gitlab.cern.ch/cms-nanoAOD/nanoAOD-integration/-/jobs/14197088

Is it a bug on CMSSW side? Do they need to install openssl locally?

@davidlange6
Copy link
Contributor

davidlange6 commented Jun 1, 2021 via email

@smuzaffar
Copy link
Contributor Author

correct, as announced

we now pick up openssl from system and one should have openssl-libs and openssl-devel packages installed on system

@smuzaffar smuzaffar deleted the slc7-drop-openssl branch June 2, 2021 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants