YUM mirrors tools
When operating infrastructure at large scale, ensuring that all servers maintain the same level of packages can be a tricky issue. As newer servers are added, they likely would get a newer version of kernel and other packages than the older machines that have not been updated yet.
This is a few scripts which help in managing mirroring infrastructure that can be staged (to be tested independently) then promoted. This will mean that you will have an always functional and common list of packages across all your fleet regardless when they were deployed.
There are a few tools that help with doin this already, however, they are too
much (require client agents, or other tooling) for the simple task we're trying
to accomplish here. This implies you'll be updating your
files via some existing configuration management, therefore no need for all that
This toolset include a configuration which can be used for deploying OpenStack
Ansible. You can create your own configuration by looking at what the
config/openstack-ansible.conf file looks like. Contributions are welcome for
You can control a few parameters where you'll be storing your data. You can do this by overriding the following environment variables
MIRROR_LOCATION: Storage path for all downloaded packages (defaults to
REPO_LOCATION: Path to store XML metadata for releases (defaults to
/var/www/html). This should be exposed by a webserver.
In addition, there is a few profiles available by default for mirroring
base.conf: CentOS packages
openstack-ansible.conf: CentOS packages + OpenStack-Ansible dependencies
Clone this repository on the machine that is going to be running the mirrors
$ git clone https://github.com/vexxhost/yum-mirror-tools /opt/yum-mirror-tools
Install dependencies (tooling to speed things up & create repositories). You should also setup an HTTP server which will be hosting your
$ yum -y install createrepo httpd $ rm -rfv /etc/httpd/conf.d/welcome.conf $ systemctl enable httpd $ systemctl start httpd
You can start the initial mirror (this will take quite sometime depending on your connection speed). In this example, we're using the
$ /opt/yum-mirror-tools/sync-packages.sh config/openstack-ansible.conf
The tool does not currently support running multiple configurations at once, patches welcome!
At this stage, you do not have a usable repository that you can point your users to. You need to make your first release of repositories. This shows an example of releasing your
$ /opt/yum-mirror-tools/create-release.sh 1.0.0
You should now have an accessible, frozen set of packages which you can use inside your
yumrepository configuration at
http://host/1.0.0/repoid. If you followed these instructions, you should be able to see all the repositories listed by opening that path in your browser.
Once you've tested tha teverything is working by deploying that path against a subset of machines, you can now setup a symbolic link (that always changes) towards the latest tested release.
promote-repo.shscript has a few more things added there which shutdown the web service during the promotion to make sure it kills off any HTTP clients and forces them to either fail or retry with the nwe package list.
$ /opt/yum-mirror-tools/promote-repo.sh 1.0.0