Consolidate Docker images for testing to cern/cc7-base:latest

[matthewfeickert]

At the moment most of the CI is running inside of gitlab-registry.cern.ch/linuxsupport/cc7-base:latest

$ git grep "cc7-base"
.github/workflows/build.yml:    container: gitlab-registry.cern.ch/linuxsupport/cc7-base:latest
.github/workflows/build.yml:    container: gitlab-registry.cern.ch/linuxsupport/cc7-base:latest
.github/workflows/build.yml:    container: gitlab-registry.cern.ch/linuxsupport/cc7-base:latest
.github/workflows/build.yml:    container: gitlab-registry.cern.ch/linuxsupport/cc7-base:latest
.github/workflows/build.yml:    container: gitlab-registry.cern.ch/linuxsupport/cc7-base:latest
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base
.gitlab-ci.yml:  image: gitlab-registry.cern.ch/linuxsupport/cc7-base

@simonmichal mentioned

Just for the record, the cern/cc7-base:latest to generate artefacts (meaning RPMs) for the platform, regarding tests AFAIR this part is doing the job:

xrootd/.gitlab-ci.yml

Lines 546 to 585 in 9292961

	pyxrootd_dockerimage:
	stage: build:dockerimage
	tags:
	- docker-image-build
	script:
	- ""
	variables:
	TO: gitlab-registry.cern.ch/dss/xrootd:pylatest
	DOCKER_FILE: xrootd-docker/Dockerfile-python.ci
	dependencies:
	- xrootd_docker_get
	only:
	- schedules
	- web
	except:
	- tags
	allow_failure: true
	pyxrootd_docker_test:
	stage: test
	script:
	- docker pull gitlab-registry.cern.ch/dss/xrootd:pylatest
	- sudo docker run -dit --privileged -e "container=docker" --name pyxrootd-container -h pyxrootd-container gitlab-registry.cern.ch/dss/xrootd:pylatest /sbin/init
	- docker exec pyxrootd-container systemctl start xrootd@standalone
	- docker exec pyxrootd-container sh -c "cd xrootd/bindings/python/tests && pytest test_file.py test_filesystem.py test_copy.py test_threads.py test_url.py"
	after_script:
	- sudo docker rm -f pyxrootd-container
	- sudo docker rmi -f gitlab-registry.cern.ch/dss/xrootd:pylatest
	tags:
	- shell-with-docker
	dependencies:
	- xrootd_docker_get
	only:
	- schedules
	- web
	except:
	- tags
	allow_failure: true

Originally posted by @simonmichal in #1585 (comment)

so given this is there interest in migrating the CI to run on cern/cc7-base:latest for consistency?

[simonmichal]

do you mean the GitHub actions? hmm I think it would be great if we could keep those builds that we currently have and add some additional for cc7. Let me know your thoughts!

[simonmichal]

@matthewfeickert : I just notice two problems after merging #1585:

• cannot uninstall the package anymore:

[root@77ad5541c633 xrootd]# python3 -m pip uninstall xrootd                  
Can't uninstall 'xrootd'. No files were found to uninstall.

• if I tag the repository, say: git tag -a v6.0.0 -m test it is no longer possible to install the package:

[root@77ad5541c633 xrootd]# cp packaging/wheel/* .
[root@77ad5541c633 xrootd]# ./publish.sh
[root@77ad5541c633 xrootd]# python3 -m pip install dist/xrootd-6.0.0.tar.gz 
WARNING: Running pip install with root privileges is generally not a good idea. Try `__main__.py install --user` instead.
Processing ./dist/xrootd-6.0.0.tar.gz
  Requirement already satisfied (use --upgrade to upgrade): xrootd==6.0.0 from file:///xrootd/dist/xrootd-6.0.0.tar.gz in /xrootd
Building wheels for collected packages: xrootd
  Running setup.py bdist_wheel for xrootd ... done
  Running setup.py clean for xrootd
Failed to build xrootd

Could you please fix those two regressions?

[matthewfeickert]

do you mean the GitHub actions? hmm I think it would be great if we could keep those builds that we currently have and add some additional for cc7. Let me know your thoughts!

I mean in general across all the CI. Let me ask it another way:

If you can run and build everything using cern/cc7-base:latest and that's what you're actually testing against when you make releases why is gitlab-registry.cern.ch/linuxsupport/cc7-base:latest used? Does it have some utilities in it that cern/cc7-base:latest is lacking?

[simonmichal]

@matthewfeickert : OK, I understand now the question, basically cern/cc7-base:latest and gitlab-registry.cern.ch/linuxsupport/cc7-base:latest should be the equivalent, it's just the latter happens to be hosted at CERN so in principle it should be faster for our CI to get the image. Hope that it makes sense.

[matthewfeickert]

basically cern/cc7-base:latest and gitlab-registry.cern.ch/linuxsupport/cc7-base:latest should be the equivalent, it's just the latter happens to be hosted at CERN so in principle it should be faster for our CI to get the image

Cool. I think that means that things can probably get left as is (as I don't think there's a huge time delay in the pull from CERN's GitLab container registry compared to Docker Hub).

Before I close out this Issue, if I was also going to add Debian based builds to the GitHub Actions CI is there a strict need to follow the .gitlab-ci.yml's use of Ubuntu images? Or could these test jobs just be done using debian:bullseye images? I assume that nothing Ubuntu specific is being used, but I thought I'd check.