cleanup of py2 packages; make sure to install their dependencies #4482
Conversation
|
A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_10_4_X/gcc700. @cmsbuild, @smuzaffar, @gudrutis, @mrodozov can you please review it and eventually sign? Thanks. |
|
FYI @davidlange6 and @amaltaro |
|
please test |
|
The tests are being triggered in jenkins. |
|
Pull request #4482 was updated. |
|
please test |
|
The tests are being triggered in jenkins. |
|
hi @smuzaffar - cool this will really help. We talked some time ago about dependencies. You've removed the few declared dependencies that the py2*spec files had. So this means if package X really does depend at build time on package py2-Y, then it needs to declare that dependency and any others in the full dependency tree of py2-Y. Is that right? Hopefully we will continue to have o(1) such package (tensorflow) where this is important. |
|
@davidlange6 , yes package X needs to declare its dependencies and it will also recursively get all the dependencies from packages its depend on. I only added the dependencies which are needed. e.g. I removed 47 deps of |
|
Thanks Shahzad. Once this gets merged, I'll pick your build-with-pip changes and adapt according to the comp_gcc630 needs (aka PYTHONPATH only, since we don't have the same py2/py3 distinction as in cmssw). The question I had is about the pkgtools tag. Do you see any reason why we shouldn't start using the 00-32 tag ASAP in comp_gcc630? |
|
@amaltaro , there is no reason why you should not be able to move to V00-32. It is back word compatible. |
|
Ok, I built comp with the pkgtools |
|
-1 Tested at: f633865
I found compilation error when building: py2-networkx 2.2 py2-nose 1.3.7 py2-pymongo 3.7.2 py2-six 1.11.0 + exit 1 error: Bad exit status from /build/cmsbld/jenkins/workspace/ib-any-integration/testBuildDir/tmp/rpm-tmp.zRnok2 (%build) RPM build errors: Bad exit status from /build/cmsbld/jenkins/workspace/ib-any-integration/testBuildDir/tmp/rpm-tmp.zRnok2 (%build) You can see the results of the tests here: |
|
@amaltaro, V00-32 works with both (old and new) build-with-pip. |
|
hold I will update this PR later with the changes for build-with-pip |
|
Pull request has been put on hold by @smuzaffar |
|
please test |
|
The tests are being triggered in jenkins. |
|
Pull request #4482 was updated. |
|
please test |
|
The tests are being triggered in jenkins. |
|
Pull request #4482 was updated. |
|
Comparison job queued. |
|
Comparison is ready @slava77 comparisons for the following workflows were not done due to missing matrix map:
Comparison Summary:
|
|
+externals |
| ### RPM external py2-pyOpenSSL 18.0.0 | ||
| ## IMPORT build-with-pip | ||
|
|
||
| Requires: py2-cryptography |
There was a problem hiding this comment.
you missed a dependency on py2-six here
There was a problem hiding this comment.
py2-cryptography brings in that dependency
https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_10_4_X/gcc700/py2-cryptography.spec#L4
Cleanup for
py2-*packages. These changes work withpkgtools V00-32-XXbranch..in package name should be replaced with-e.g. pip packagebackports.lzmabecomespy2-backports-lzma.specpy2-*-toolfile.specand automatically generate these viacmssw-tool-conf.specpy2-*package satisfied and fail at build time if missingpy2-*packages to be of the formone day we can auto generate these from a single file. For packages with
<extra build/requires flags>can have py2-PipPackageName.file which can be included in the auto-generated package specs (using new## INCLUDE file-namefeature ofcmsBuild).build-with-pip.filegeneric enough so that we can customize the build from the package spec.