Clean up RPM builds and fix i686 RPM file names #5241
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Our rpm/build_rpms.bsh script currently attempts to parse the /etc/os-release file to retrieve the major OS version number, and if that file does not exist, reads /etc/redhat-release instead; the latter logic has been in place since the introduction of the script in commit 4a71627 of PR git-lfs#332, and has been the fallback logic since commit 56ffe42 of PR git-lfs#555. However, /etc/os-release should exist on all the current versions of CentOS and Rocky Linux we support, so we do not need to retain the fallback parsing of /etc/redhat-release at this point.
In commit b560b85 of PR git-lfs#1298 and in commit git-lfs/build-dockers@64a3a9f of PR git-lfs/build-dockers#3 we dropped support for CentOS 5, so we can remove the logic in our rpm/build_rpms.bsh which installed a RedHat EPEL package only on this platform.
In commit cd3e92e of PR git-lfs#480 we introduced support for CentOS 6 into our rpm/build_rpms.bsh script by building an early version of Git from a local git.spec file. Then in commit d2d90a8 of PR git-lfs#891 this was replaced with installation from the setup.ius.io service. In commit git-lfs/build-dockers@7af5552 of PR git-lfs/build-dockers#19, however, the download and installation of Git 2.16 from github.com was added to the CentOS 6 Dockerfile; later this was adjusted to download from kernel.org. In commit git-lfs/build-dockers@f1f573f of PR git-lfs/build-dockers#29 we added support for CentOS 8, which supports a direct installation of Git using yum. And in commit git-lfs/build-dockers@898d9b0 of PR git-lfs/build-dockers#33 we dropped support for CentOS 6 entirely. Therefore we no longer have any need for our rpm/build_rpms.bsh script to install Git at all, especially not very old versions, so we simply remove that logic from our script.
Since the original introduction of the rpm/build_rpms.bsh script in commit 4a71627 of PR git-lfs#332 it has checked for the absence of the "go" binary and, if it is not found, installs the golang package, except on CentOS 5 where fetches and patches a golang.spec file and then builds a local RPM using that in order to install Go. However, in commit b560b85 of PR git-lfs#1298 and in commit git-lfs/build-dockers@64a3a9f of PR git-lfs/build-dockers#3 we dropped support for CentOS 5. And in commit 88430de of PR git-lfs#654 we updated our CentOS Dockerfiles (before they were moved to the git-lfs/build-dockers repository) to download and install Go from the googleapis.com service, and we still continue to do this but as of commit git-lfs/build-dockers@c638503 in PR git-lfs/build-dockers#35 we now use the golang.org service. This direct download obviates the logic in our rpm/build_rpms.bsh script to install the golang package, and the special handling for CentOS 5 is also obsolete, so we remove this logic from our script.
In commit git-lfs/build-dockers@544e07e of PR git-lfs/build-dockers#47 we updated our CentOS 7 Dockerfile to install Ruby 3.0 using the rh-ruby30-{ruby,build} packages and then run our CentOS build script with that version of Ruby as the default. We made this change to support our use of Asciidoctor in preference to ronn, which was completed in PR git-lfs#5054, and as part of that PR, in commit db9a821 we replaced our installation of ronn with one of asciidoctor in our rpm/build_rpms.bsh script. As a result, none of our CentOS or Rocky Linux container image builds now rely on Ruby 2.x, so we can remove the code which installed that version. We can also remove the related code which used to install Ruby 2.x when an even older version was the default, along with our local ruby.spec file for Ruby 2.2.2. Finally, we can remove the code which installed the rubygems-devel package on CentOS 8 because that is taken care of by the corresponding Dockerfile in our git-lfs/build-dockers project, which, like the one for Rocky Linux 9, installs ruby, ruby-devel, and rubygems-devel.
Since the original introduction of the rpm/build_rpms.bsh script in commit 4a71627 of PR git-lfs#332 it has installed a number of packages, including make, curl, tar, and bison. Then in commit a4c9db8 of PR git-lfs#480 the perl-Digest-SHA package was added as an install target as well. However, we no longer need to install any of these, either because the are already installed by our CentOS and Rocky Linux Dockerfiles in our git-lfs/build-dockers project, or because they are simply unused at present, so we remove these packages now.
In commit db9a821 of PR git-lfs#5054 we replaced our use of ronn with Asciidoctor, and as part of that change we updated our rpm/build_rpms.bsh script to only install the Asciidoctor Ruby gem instead of a set of four gems including one for ronn. As a consequence we no longer have any x86_64 Ruby gem RPMs but only "noarch" ones, so we can simplify our installation slightly, and also update our status messages to reflect that we are installing just one gem.
We no longer support either CentOS 5 or 6, per commit b560b85 of PR git-lfs#1298 and commit git-lfs/build-dockers@64a3a9f of PR git-lfs/build-dockers#3, and commit git-lfs/build-dockers@898d9b0 of PR git-lfs/build-dockers#33. We also now build the Asciidoctor Ruby gem in order to generate our man pages, rather than using ronn and several other gems, per commit db9a821 of PR git-lfs#5054. We therefore update the documentation for our RPM package build utilities and for our Docker container image build utilities to reflect these changes.
The SPEC variable is currently set with the relative path to our git-lfs.spec file, but is then not used in places where it could be, so we replace those with references to the variable.
In commit e2f770c of PR git-lfs#428 the rpm/build_rpms.bsh script was updated to run the rpmbuild command with the --nodeps option when the NODEPS environment variable was defined and set non-zero. However, in commit 88430de of PR git-lfs#654 the script was revised to always pass the --nodeps option to the rpmbuild command, so the NODEPS environment variable no longer had any effect, and so We can therefore remove our handling of this variable.
chrisd8088
added a commit
to chrisd8088/build-dockers
that referenced
this pull request
Dec 28, 2022
Since the original introduction of support for building CentOS RPMs in commit git-lfs/git-lfs@4a71627 of PR git-lfs/git-lfs#332, an rpm/clean.bsh script has been provided to perform the equivalent of a "git clean -xdf" command on the Git repository from which we build Git LFS. However, this script is no longer needed as all of the CentOS and Rocky Linux container image builds have a native Git package installed, so we remove our use of it, and also drop the script from the primary Git LFS project in PR git-lfs/git-lfs#5241.
chrisd8088
added a commit
to chrisd8088/build-dockers
that referenced
this pull request
Dec 28, 2022
Since the original introduction of support for building CentOS RPMs in commit git-lfs/git-lfs@4a71627 of PR git-lfs/git-lfs#332, an rpm/clean.bsh script has been provided to perform the equivalent of a "git clean -xdf" command on the Git repository from which we build Git LFS. However, this script is no longer needed as all of the CentOS and Rocky Linux container image builds have a native Git package installed, so we remove our use of it, and also drop the script from the primary Git LFS project in PR git-lfs/git-lfs#5241.
Since the original introduction of support for building CentOS RPMs in commit 4a71627 of PR git-lfs#332, an rpm/clean.bsh script has been provided to perform the equivalent of a "git clean -xdf" command on the Git repository from which we build Git LFS. However, this script is no longer needed as all of the CentOS and Rocky Linux container image builds have a native Git package installed, and so we remove the only use of it, in the centos_script.bsh script of the git-lfs/build-dockers project, in commit git-lfs/build-dockers@7be51a4 of PR git-lfs/build-dockers#54, and can therefore also drop the script from this repository as well.
In commit c2d25ee of PR git-lfs#511 we added support for building RPM packages for 32-bit platforms by updating the docker/centos_script.bsh script which was present at that time to call rpmbuild with a --target=i686 argument. Since commit 56ffe42 of PR git-lfs#555 both that script and the rpm/build_rpms.bsh script contained the same logic to parse the OS name and version in order to set a short suffix for the RPM filenames. However, the docker/centos_script.bsh script was subsequently moved into the git-lfs/build-dockers repository, where it has not been updated to match the rpm/build_rpms.bsh script, such as when parsing of the OS major version was added in commit e939409 of PR git-lfs#5054, which allows us to properly parse the version number on CentOS/Rocky Linux 8 and above, or when parsing of the Rocky Linux OS name was added in commit 723be34 of PR git-lfs#5144. The result is that at present we build 32-bit RPMs for CentOS 8 and Rocky Linux 9 (el8 and el9, respectively) without the platform short name suffix in their filenames, e.g., git-lfs-3.3.0-1.i686.rpm and git-lfs-3.3.0-1.i686.rpm, and then upload them to Packagecloud with those names. To resolve this problem and avoid later regressions between the two sets of parsing logic, we move the rpmbuild command for 32-bit packages into our rpm/build_rpms.bsh script, which ensures they will be built with the same context as our 64-bit packages. To do this we introduce an rpmbuild command with the --target=i686 argument into rpm/build_rpms.bsh, which allows us to also remove the rpmbuild command from the centos_script.bsh script in the git-lfs/build-dockers repository in PR git-lfs/build-dockers#54.
chrisd8088
added a commit
to chrisd8088/build-dockers
that referenced
this pull request
Dec 28, 2022
In commit git-lfs/git-lfs@c2d25ee of PR git-lfs/git-lfs#511 we added support for building RPM packages for 32-bit platforms by updating the docker/centos_script.bsh script which was present at that time to call rpmbuild with a --target=i686 argument. Since commit git-lfs/git-lfs@56ffe42 of PR git-lfs/git-lfs#555 both that script and the rpm/build_rpms.bsh script in the primary Git LFS project's repository contained the same logic to parse the OS name and version in order to set a short suffix for the RPM filenames. However, the docker/centos_script.bsh script was subsequently moved into this repository, where it has not been updated to match the rpm/build_rpms.bsh script in the primary project repository, such as when parsing of the OS major version was added in commit git-lfs/git-lfs@e939409 of PR git-lfs/git-lfs#5054, which allows us to properly parse the version number on CentOS/Rocky Linux 8 and above, or when parsing of the Rocky Linux OS name was added in commit git-lfs/git-lfs@723be34 of PR git-lfs/git-lfs#5144. The result is that at present we build 32-bit RPMs for CentOS 8 and Rocky Linux 9 (el8 and el9, respectively) without the platform short name suffix in their filenames, e.g., git-lfs-3.3.0-1.i686.rpm and git-lfs-3.3.0-1.i686.rpm, and then upload them to Packagecloud with those names. To resolve this problem and avoid later regressions between the two sets of parsing logic, we add an rpmbuild command with the --target=i686 argument for 32-bit packages to the rpm/build_rpms.bsh script in the primary Git LFS project repository in commit git-lfs/git-lfs@7830f04 of PR git-lfs/git-lfs#5241, which ensures they will be built with the same context as our 64-bit packages. We can therefore also remove the rpmbuild command with the --target=i686 argument from the centos_script.bsh script in this repository, along with the setup code which attempted to parse the OS version and name from either /etc/os-release or /etc/redhat-release, as these values are only used to set the RPM_DIST variable passed to rpmbuild, and the initialization code for that variable can be removed as well.
chrisd8088
changed the title
bk2204
approved these changes
Jan 3, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR cleans up various issues with our RPM build scripts, in particular removing leftover support for CentOS 5 and 6, and moving responsibility for building 32-bit RPMs into this project so as to fix a naming problem with the 32-bit RPMs for CentOS 8 and Rocky Linux 9.
Reviewing this PR commit-by-commit is recommended, and each commit should have a full description of its changes.
We expect to merge this PR prior to when its companion PR git-lfs/build-dockers#54 is merged, specifically because that PR drops its own build logic for 32-bit RPMs in deference to this project building those RPMs at the same time as the 64-bit ones and in the same context.
This PR's changes have been tested in conjunction with those of PR git-lfs/build-dockers#54 and confirmed to produce valid build artifacts. We used a branch of
git-lfs/git-lfswith the changes from this PR plus revisions to the GitHub Actions workflows so as to use the branch from PR git-lfs/build-dockers#54 and then to upload the resultant artifacts to GitHub Actions. These can be examined to confirm that they now correctly name the 32-bit RPM builds for CentOS 8 and Rocky Linux 9 and that they also include only Git LFS packages and no unrelated ones:Building 32-Bit RPMs
In commit c2d25ee of PR #511 we added support for building RPM packages for 32-bit platforms by updating the
docker/centos_script.bshscript which was present at that time to callrpmbuildwith a--target=i686argument.Since commit 56ffe42 of PR #555 both that script and the
rpm/build_rpms.bshscript contained the same logic to parse the OS name and version in order to set a short suffix for the RPM filenames.However, the
docker/centos_script.bshscript was subsequently moved into thegit-lfs/build-dockersrepository, where it has not been updated to match therpm/build_rpms.bshscript, such as when parsing of the OS major version was added in commit e939409 of PR #5054, which allows us to properly parse the version number on CentOS/Rocky Linux 8 and above, or when parsing of the Rocky Linux OS name was added in commit 723be34 of PR #5144.The result is that at present we build 32-bit RPMs for CentOS 8 and Rocky Linux 9 (
el8andel9, respectively) without the platform short name suffix in their filenames, e.g.,git-lfs-3.3.0-1.i686.rpmandgit-lfs-3.3.0-1.i686.rpm(instead ofgit-lfs-3.3.0-1.el9.i686.rpmandgit-lfs-3.3.0-1.el8.i686.rpm), and then upload them to Packagecloud with those names.To resolve this problem and avoid later regressions between the two sets of parsing logic, we add an
rpmbuildcommand with the--target=i686argument for 32-bit packages to therpm/build_rpms.bshscript of this repository, which ensures they will be built with the same context as our 64-bit packages.We therefore also remove the
rpmbuildcommand with the--target=i686argument from thecentos_script.bshscript in thegit-lfs/build-dockersrepository, in commit git-lfs/build-dockers@0b991e0 of PR git-lfs/build-dockers#54.Removing Substitute Repository Cleanup Script
Since the original introduction of support for building CentOS RPMs in commit 4a71627 of PR #332, an
rpm/clean.bshscript has been provided to perform the equivalent of agit clean -xdfcommand on the Git repository from which we build Git LFS.However, this script is no longer needed as all of the CentOS and Rocky Linux container image builds defined by the
git-lfs/build-dockersrepository'sDockerfiles have a native Git package installed (or a recent Git built, in the case of CentOS 7), so we remove the only use of it, in commit git-lfs/build-dockers@7be51a4 of PR git-lfs/build-dockers#54, and can therefore also drop the script from this repository as well.Removing CentOS 5 and 6 Build Steps
We no longer support either CentOS 5 or 6, per commit b560b85 of PR #1298 and commit git-lfs/build-dockers@64a3a9f of PR git-lfs/build-dockers#3, and commit git-lfs/build-dockers@898d9b0 of PR git-lfs/build-dockers#33.
We also now build the
asciidoctorRuby gem in order to generate our man pages, rather than usingronnand several other gems, per commit db9a821 of PR #5054.As a result, we can eliminate a number of build steps which only operated on systems that lacked native package manager support for Go, Git 2.x, Ruby 3.x, etc., as well as steps pertaining to the Ruby gem dependencies required for the
ronngem.