PSF Infrastructure Overview
The PSF runs a wide variety of infrastructure services to support its mission from the PyCon site to the CPython Mercurial server. The goal of this page is to enumerate all these services, where they run, and who the main contact points are.
The Infrastructure Team
The infrastructure team is ultimately responsible for maintaining PSF infrastructure. It is not, however, required to be a member of the infrastructure to run a PSF service. Indeed, the day to day operations of most services are handled by people not on the infrastructure team. The infrastructure team can assist in setting up new services and advise maintainers of individual services. Its members also generally handle changes to sensitive global systems such as DNS. The currrent team members are:
- Alex Gaynor (has no responsibilities)
- Benjamin Peterson
- Benjamin W. Smith
- Donald Stufft
- Ernest W. Durbin III (PSF Director of Infrastructure)
- Noah Kantrowitz
The PSF uses several different cloud providers and services for its infrastructure.
- XS4ALL is the by far the oldest Python infrastructure provider. There are two physical servers owned by the PSF at XS4ALL: albatross and dinsdale. (There also used to be one called ximinez, but it seems to be unreachable now.) albatross is the mail server. dinsdale hosts a number of legacy services. We try not to put anything new on the XS4ALL servers, preferring modern cloud providers.
- Oregon State University Open Source Lab hosts VMs for the PSF. These VMs are provisioned using Chef and their configuration management is in the psf-chef git repo.
- Rackspace was the primary cloud provider utilized by the PSF until March 2019. Salt is used for configuration management.
- Dyn & Gandi
- Gandi is our domain registar, and we use Dyn for DNS hosting on most of our domains.
- Pingdom provides monitoring and complains to us when services are down.
- Fastly generously donates its content distribution network (CDN) to the PSF. Our highest traffic services (i.e. PyPI, www.python.org, docs.python.org) use this CDN to improve end-user latency.
- Heroku hosts many of the CPython core workflow bots, ephemeral or proof of concept apps, as well as other web apps that are well suited to it's platform.
- DigitalOcean is the current target for most of the infrastructure we currently host in Rackspace, services deployed here are managed by Salt.
Details of Various Services
This section enumerates PSF services, generalities about their hosting, and contact information for the owners.
- The buildbot master is a service run by firstname.lastname@example.org, particularly Antoine Pitrou and Zach Ware.
- bugs.python.org is hosted by the PSF on DigitalOcean, powered by Roundup. It also hosts bugs.jython.org and issues.roundup-tracker.org.
- The Python documentation is hosted on DigitalOcean, served through Fastly, and owned by Julien Palard.
- The CPython Mercurial repositories are hosted on a Rackspace VM. The service is owned by Antoine Pitrou and Georg Brandl.
- The python.org Mailman instance is hosted on https://mail.python.org as well as SMTP (Postfix). All inquiries should be directed to email@example.com.
- planetpython.org and planet.jython.org
- These are hosted on a Rackspace VM. The Planet code and configuration are hosted on GitHub and maintained by the team at firstname.lastname@example.org.
- pythontest.net hosts services and files used by the Python test suite. email@example.com is ultimately responsible for its maintenance.
- speed.python.org is a Codespeed instance tracking Python performance. The web interface is hosted on a Rackspace VM, benchmarks are run on a beefy machine at OSUOSL and scheduled by the Buildbot master. Maintained by firstname.lastname@example.org and Zach Ware.
- This is hosted on an OSUOSL VM. Marc-André Lemburg owns it.
- This is hosted on a Rackspace VM. The setup is quite simple and shouldn't require much tweaking, but Benjamin Peterson can be poked about it.
- The main Python website is a Django app hosted on a Rackspace VM. Its source code is available on GitHub, and issues with the site can be reported to the GitHub issue tracker. Python downloads (i.e. everything under https://www.python.org/ftp/) are hosted on a separate Rackspace VM. The whole site is behind Fastly. There is also https://staging.python.org for testing the site. http://legacy.python.org is the old website hosted on dinsdale.
- The PyCon website is hosted on Heroku. The contact address is email@example.com.
- The Python Package Index sees the most load of any PSF service. Its source code is available on GitHub. All of its infrastructure runs on AWS configured by pypi-salt, and it is served over Fastly. The infrastructure is maintained by Ernest W. Durbin III, Donald Stufft, and Dustin Ingram. The contact address is firstname.lastname@example.org.
- PyPy properties
- The PyPy website is hosted on a OSUOSL VM and maintained by email@example.com.