Skip to content
master
Go to file
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

README.md

This git repository contains the various scripts used in the CentOS Infra for mirrorlist service and isoredirect service.

Overview

mirrorlists schema

It contains the following kind of scripts:

Backend (crawler)

There are two Perl scripts for checking mirrors:

  • makemirrorlists-combined.pl for creating files for mirrorlist.centos.org
  • makeisolists-combined.pl for creating files for isoredirect.centos.org.

Both scripts can create lists for CentOS 6 and CentOS 7,including SIG and AltArch content. makemirrorlists-combined.pl will test each mirror separately for IPv4 and IPv6.

mirrorlist.centos.org will then be able to present only IPv6-capable mirrors to the clients when mirrorlist.centos.org is accessed over IPv6. More details about the internals of these scripts can be found in backend/mirrorlist_crawler_deployment_notes.txt

Frontend

All scripts are located in the frontend folder. The following items are needed for the mirrorlist/isoredirect service:

  • A http server (apache) using mod_proxy_balancer (see frontend/httpd/mirrorlist.conf vhost example)
  • python-bottle to run the {ml,isoredirect}.py code for various instances
  • Maxmind Geolite2 database : City version
  • python-geoip2 pkg (to consume those Geolite2 DB)
  • python-memcached (to cache results for GeoIP/Cloud providers)
  • For each worker, a specific instance/port can be initialized and added to Apache config for the proxy-balancer (see frontend/systemd/centos-ml-worker@.service)

Those services (mirrorlist/isoredirect) just consume mirrorlist files, pushed to those nodes, and updated in loop by the Crawler process (see Backend section above)

When a request is made to the service, the python script :

  • checks for IPv4 or IPv6 connectivity
  • checks if IP is in memcached (for country/cloud provider)
  • searches if IP is from cloud provider
  • computes Geolocation based on the origin IP
  • searches for validated mirrors in the same country/state for the request arch/repo/release
  • returns such list

About

Git repositories holding needed scripts for mirrors crawlers and mirrorlist/isoredirect nodes

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.