Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IUS CDN #14

Open
carlwgeorge opened this issue Apr 30, 2019 · 22 comments

Comments

@carlwgeorge
Copy link
Member

@carlwgeorge carlwgeorge commented Apr 30, 2019

Summary

The IUS project will be migrating from our existing mirror network to utilizing Rackspace CDN.

Current State

Currently the IUS repositories are published to https://dl.iuscommunity.org. We have a network of community mirrors that replicate this content. The ius-release package points users to https://mirrors.iuscommunity.org/mirrorlist, which provides a geographically sorted list of mirrors based on the GeoIP information of the requesting IP.

Future State

https://repo.ius.io has been configured using Rackspace CDN. New packages will be published here going forward. Existing packages on https://dl.iuscommunity.org will continue to be updated until the transition is complete. This will be accomplished by updating the ius-release package to point to https://repo.ius.io instead of https://mirrors.iuscommunity.org/mirrorlist. Users of the ius-release package should not have to take any action other than upgrading ius-release. If you have customized the repo files from this package, you will need to merge in the rpmnew files just like any other package update.

FAQ

Can I start using the CDN now?

Sure! If you are already subscribed to IUS, you can update the ius-release package from the testing repo just update ius-release. Make sure to clear the yum cache afterwards.

yum update ius-release
yum clean all

On a system that isn't subscribed to IUS yet, new ius-release packages can be downloaded at these links:

You can also download the repo config and GPG key individually from https://repo.ius.io.

What if I still want to mirror IUS for my own needs?

We realize that many members of our mirror network still would like to mirror our content for their own needs, commonly referred to as a "private" mirror. This will still be an option even after https://dl.iuscommunity.org is taken offline in the future. We will be reaching out to existing mirror contacts with the new rsync information. If anyone new wants to mirror IUS for their own needs reach out to dev@ius.io.

Why don't I see separate packages for RHEL and CentOS?

In the past IUS built separate packages for RHEL and CentOS due to possible long delays between a RHEL release and CentOS catching up. Now that Red Hat sponsors CentOS that only takes a few weeks at most. It's just not worth the effort of the duplicate builds anymore. In the future, if a library change happens in RHEL that causes a built package to be compatible with the new RHEL but not the yet-to-catch-up CentOS, IUS can keep that package in ius-testing until CentOS catches up. This is the approach that EPEL normally takes.

@carlwgeorge carlwgeorge referenced this issue Apr 30, 2019
@devicenull

This comment has been minimized.

Copy link

@devicenull devicenull commented Apr 30, 2019

Why are you switching away from third party mirrors?

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented Apr 30, 2019

@devicenull Managing a mirror network takes much more effort than using a CDN. On top of that, CDN solves the distribution problem much better. There are more endpoints around the globe, and no work is needed from the IUS team as the CDN service adds and removes nodes. Anyone that still wants to mirror IUS can, but the official ius-release package is going to point to the CDN.

@jjlin

This comment has been minimized.

Copy link

@jjlin jjlin commented Apr 30, 2019

@carlwgeorge For people who run "private" mirrors, could you just post the rsync info publicly?

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented Apr 30, 2019

@jjlin The domain for the rsync is the same as the https origin for the CDN. I'd like to avoid users bypassing the CDN. If you need the rsync info just email dev@ius.io and I'll be happy to share it with you.

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented May 2, 2019

I've created a new ius-release package that points to the CDN. It can be downloaded at the following links for those that want to start using it.

I have not published this package into the stable repo yet.

@kustodian

This comment has been minimized.

Copy link

@kustodian kustodian commented May 9, 2019

What about the CentOS packages? I only see EL. Old repository URL has separate repositories for CentOS and RHEL.

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented May 13, 2019

@kustodian Yes, in the past IUS built separately for RHEL and CentOS. To my knowledge we were the only third party repository doing this. Even EPEL only builds against RHEL. In the past there were concerns with long delays between a RHEL release and CentOS catching up, but now that Red Hat sponsors CentOS that only takes a few weeks at most. It's just not worth the effort of the duplicate builds anymore.

In the future, if a library change happens in RHEL that causes a built package to be compatible with the new RHEL but not the yet-to-catch-up CentOS, IUS can keep that package in ius-testing until CentOS catches up. This is the approach that EPEL normally takes.

@kustodian

This comment has been minimized.

Copy link

@kustodian kustodian commented May 13, 2019

Cool. That means that we can safely switch to the new repo URL. Thanks.

@kustodian

This comment has been minimized.

Copy link

@kustodian kustodian commented May 14, 2019

Is the new repo up-to-date? Because once I changed the URL yum is telling me that the repo data is older from what we already have:

ius                                                                                                                                                                                  | 1.3 kB  00:00:00     
Not using downloaded ius/repomd.xml because it is older than what we have:
  Current   : Wed May  1 10:58:04 2019
  Downloaded: Mon Apr 22 18:45:24 2019
@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented May 14, 2019

Yes, it's up to date. That was just a side effect of the timing of migrating the content. The old system regenerated the repodata every night, even if the content was the same, and that ran a few more times after I seeded the new repo with the old content. A yum clean all would resolve this for you. I went ahead and regenerated the repodata for good measure as well.

@kustodian

This comment has been minimized.

Copy link

@kustodian kustodian commented May 14, 2019

Thanks for the explanation.

@amwestg

This comment has been minimized.

Copy link

@amwestg amwestg commented Jun 30, 2019

https://repo.ius.io/6/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
Need update for PHP71u for i386 - directory missing

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented Jun 30, 2019

As part of this infrastructure refresh, we now build our packages in Cirrus CI. Due to the extremely low usage of our i686 packages, we made this switch without a plan for building i686 packages in the future. If you would like to help re-enable building those, take a look at our build configuration and the Cirrus CI docs and send a pull request. Considering that the Cirrus infrastructure is x86_64 only, some cross compile solution will be needed to create i686 packages.

Personally, I think the effort that would be needed for this would be better spent migrating your systems to RHEL/CentOS 7 x86_64. RHEL/CentOS 6 only has 17 months until it's end of life date. It's also already in Maintenance Support 2 Phase, meaning that only the most critical and urgent fixes are backported, entirely at Red Hat's discretion. EL6 is on life support.

@amwestg

This comment has been minimized.

Copy link

@amwestg amwestg commented Jul 4, 2019

As part of this infrastructure refresh, we now build our packages in Cirrus CI. Due to the extremely low usage of our i686 packages, we made this switch without a plan for building i686 packages in the future. If you would like to help re-enable building those, take a look at our build configuration and the Cirrus CI docs and send a pull request. Considering that the Cirrus infrastructure is x86_64 only, some cross compile solution will be needed to create i686 packages.

Personally, I think the effort that would be needed for this would be better spent migrating your systems to RHEL/CentOS 7 x86_64. RHEL/CentOS 6 only has 17 months until it's end of life date. It's also already in Maintenance Support 2 Phase, meaning that only the most critical and urgent fixes are backported, entirely at Red Hat's discretion. EL6 is on life support.

Thanks for the reply. We would appreciate if the current I386 packages for CentOS 6 are either saved or moved to your new CDN while we transition this server to CentOS 8 when it's ready later this year.. I may download the existing PHP71u packages to my local repo if it comes to that.

Any plans on building packages for RHEL 8/CentOS 8?

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented Jul 4, 2019

We would appreciate if the current I386 packages for CentOS 6 are either saved or moved to your new CDN while we transition this server to CentOS 8 when it's ready later this year.

All of the previous i686 packages are available on the CDN in the archive repository. With they ius-release package installed they can be accessed by adding the --enablerepo ius-archive flag to yum. We keep all our old builds in the archive, so it shouldn't be necessary to mirror them yourself unless you really want to.

Any plans on building packages for RHEL 8/CentOS 8?

Not at this time. I've gone into more detail here.

@ScoobyDone

This comment has been minimized.

Copy link

@ScoobyDone ScoobyDone commented Jul 6, 2019

are the verbiage and links at https://ius.io/GettingStarted/ going to be replaced?

IUS compiles separate packages
{rhel,centos}{6,7}.iuscommunity.org replaced by repo.ius.io

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented Jul 26, 2019

@ScoobyDone Yes, the website is getting overhauled and the update will include refreshed information for getting started.

@khambrecht

This comment has been minimized.

Copy link

@khambrecht khambrecht commented Sep 30, 2019

All of the previous i686 packages are available on the CDN in the archive repository. With they ius-release package installed they can be accessed by adding the --enablerepo ius-archive flag to yum. We keep all our old builds in the archive, so it shouldn't be necessary to mirror them yourself unless you really want to.

looks like the i686 package are gone. Any chance to get them back?

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented Sep 30, 2019

@khambrecht See this comment about i686 packages. The previously built i686 packages are still available from the old URL: https://dl.iuscommunity.org/pub/ius/

@khambrecht

This comment has been minimized.

Copy link

@khambrecht khambrecht commented Oct 1, 2019

@carlwgeorge thanks

iirc, the i686 packages have been available via https://repo.ius.io/archive/6/i686/ also. At least a couple of weeks ago.

But I'll switch over to https://dl.iuscommunity.org/pub/ius/ then.

@carlwgeorge

This comment has been minimized.

Copy link
Member Author

@carlwgeorge carlwgeorge commented Oct 17, 2019

The legacy mirrorlist API will be turned off 2020-01-08.

@BBQigniter

This comment has been minimized.

Copy link

@BBQigniter BBQigniter commented Nov 12, 2019

this info, that iuscommunity.org-repo is being "turned off" and that https://dl.iuscommunity.org/pub/ius/stable/ will not be updated soon, would be super great somehwere in the ius-faq (with mentioning "iuscommunity.org") or so. i couldn't find any info on ius.io in the FAQ or in About - just that there was/will be some change.

because me and my colleagues now wondered why we do not get the latest php72u-common 7.2.24 packages on our internal mirror and with some googleing i found this github issue with the explanation :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.