Use case-insensitive comparision for checking dependencies of python packages

[iarspider]

There is a problem with the way cmsdist checks python packages for missing dependencies.

The check is done by first getting package name from output of pip show ( https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_12_3_X/master/build-with-pip.file#L105 ), then looking for that name in output of pip check ( https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_12_3_X/master/build-with-pip.file#L107 ). The problem is in capitalization of PKG_NAME. E.g., for py3-cachecontrol, the value of PKG_NAME is CacheControl, but in the output of pip3 check the package is named cachecontrol (all lowercase), and the following grep ^${PKG_NAME} will not return anything

$ pip3 check --disable-pip-version-check cachecontrol
cachecontrol 0.12.6 requires msgpack, which is not installed.

We should do a case-insensitive grep for PKG_NAME.

[cmsbuild]

A new Issue was created by @iarspider .

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[smuzaffar]

Ivan Razumov
The check is done by first getting package name from output of pip show ( https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_12_3_X/master/build-with-pip.file#L105 ), then looking for that name in output of pip check ( https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_12_3_X/master/build-with-pip.file#L107 ). The problem is in capitalization of PKG_NAME. For py3-cachecontrol, the value of PKG_NAME is CacheControl, but in the output of pip3 check :
$ pip3 check --disable-pip-version-check cachecontrol
cachecontrol 0.12.6 requires msgpack, which is not installed.
the package is named cachecontrol (all lowercase), and the following grep ^${PKG_NAME} will not return anything.
I think making a case-insensitive grep (grep -i) should work. I will create a PR with that fix + any missing dependencies that are exposed after this fix.
One of the bigger missing dependencies is rust (for py3-setuptools-rust -> py3-cryptography).
to avoid building and distributing rust, we can try using pre-built wheel for py3-cryptography
same issue is with py3-pygithub: it's PKG_NAME name is PyGithub, but pip check shows it as pygithub.
a different issue with py3-keyring - it doesn't determine it's version correctly: the log file contains line Successfully installed keyring-0.0.0. Thus grepping for keyring 23.1.0 will not return anything

hi, yes you are right but I would suggest to not update build-with-pip.file which will trigger a rebuild of nearly all the cmsdist package. Issue is with cachecontrol itself. Just fix the name for it. e.g. create a file cmsdist/pip/cachecontrol.file with contents %define patchsrc0 sed -i -e 's|name="CacheControl"|name="cachecontrol"|' setup.py

Ivan Razumov
We make a temporary fix in 12_3_X, and when the time for 12_4_X comes we can include the proper fix (-i flag)

no, this should be done when we create a new arch or update gcc ( as that is going to trigger a rebuild of all packages anyway).
e.g. this can be done for gcc12 IBs

[smuzaffar]

#7696 now provides hooks which one can use to manipulate the dependencies