Skip to content

enhance download_file to fallback to https://ftp.gnu.org if downloading from https://ftpmirror.gnu.org fails - #5224

Merged
smoors merged 4 commits into
easybuilders:developfrom
boegel:download_fallback_url
Aug 6, 2026
Merged

enhance download_file to fallback to https://ftp.gnu.org if downloading from https://ftpmirror.gnu.org fails#5224
smoors merged 4 commits into
easybuilders:developfrom
boegel:download_fallback_url

Conversation

@boegel

@boegel boegel commented Aug 3, 2026

Copy link
Copy Markdown
Member

No description provided.

@boegel boegel added this to the next release (5.3.2 or 5.4.0) milestone Aug 3, 2026
@boegel boegel changed the title enhance download_file to fallback to https://ftp.gnu.org if downloading from https://ftpmirror.gnu.org fails enhance download_file to fallback to https://ftp.gnu.org if downloading from https://ftpmirror.gnu.org fails Aug 3, 2026
Comment thread easybuild/tools/filetools.py Outdated
Comment thread easybuild/tools/filetools.py

@smoors smoors left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@smoors
smoors merged commit 3ce3ebb into easybuilders:develop Aug 6, 2026
41 checks passed
@smoors

smoors commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

tested with eb GCCcore-14.2.0.eb --fetch --force-download --rebuild. works well, but there are a few improvements that we can make:

  • update the easyconfig to avoid trying source_urls for sources that we know aren't there.
  • add trace output for the failed ftpmirror downloads
WARNING: Found file gcc-14.2.0.tar.gz at g/GCCcore/gcc-14.2.0.tar.gz, but re-downloading it anyway...

  >> download succeeded: https://ftp.gnu.org/gnu/gcc/gcc-14.2.0/gcc-14.2.0.tar.gz

WARNING: Found file gmp-6.3.0.tar.bz2 at g/GCCcore/gmp-6.3.0.tar.bz2, but re-downloading it anyway...

  >> download failed: https://ftp.gnu.org/gnu/gcc/gcc-14.2.0/gmp-6.3.0.tar.bz2
  >> download failed: https://sourceware.org/pub/gcc/releases/gcc-14.2.0/gmp-6.3.0.tar.bz2
  >> download succeeded: https://ftp.gnu.org/gnu/gmp/gmp-6.3.0.tar.bz2

WARNING: Found file mpfr-4.2.1.tar.bz2 at g/GCCcore/mpfr-4.2.1.tar.bz2, but re-downloading it anyway...

  >> download failed: https://ftp.gnu.org/gnu/gcc/gcc-14.2.0/mpfr-4.2.1.tar.bz2
  >> download failed: https://sourceware.org/pub/gcc/releases/gcc-14.2.0/mpfr-4.2.1.tar.bz2
  >> download failed: https://ftp.gnu.org/gnu/gmp/mpfr-4.2.1.tar.bz2
  >> download succeeded: https://ftp.gnu.org/gnu/mpfr/mpfr-4.2.1.tar.bz2

WARNING: Found file mpc-1.3.1.tar.gz at g/GCCcore/mpc-1.3.1.tar.gz, but re-downloading it anyway...

  >> download failed: https://ftp.gnu.org/gnu/gcc/gcc-14.2.0/mpc-1.3.1.tar.gz
  >> download failed: https://sourceware.org/pub/gcc/releases/gcc-14.2.0/mpc-1.3.1.tar.gz
  >> download failed: https://ftp.gnu.org/gnu/gmp/mpc-1.3.1.tar.gz
  >> download failed: https://ftp.gnu.org/gnu/mpfr/mpc-1.3.1.tar.gz
  >> download succeeded: https://ftp.gnu.org/gnu/mpc/mpc-1.3.1.tar.gz

WARNING: Found file isl-0.26.tar.bz2 at g/GCCcore/isl-0.26.tar.bz2, but re-downloading it anyway...

  >> download failed: https://ftp.gnu.org/gnu/gcc/gcc-14.2.0/isl-0.26.tar.bz2
  >> download failed: https://sourceware.org/pub/gcc/releases/gcc-14.2.0/isl-0.26.tar.bz2
  >> download failed: https://ftp.gnu.org/gnu/gmp/isl-0.26.tar.bz2
  >> download failed: https://ftp.gnu.org/gnu/mpfr/isl-0.26.tar.bz2
  >> download failed: https://ftp.gnu.org/gnu/mpc/isl-0.26.tar.bz2
  >> download failed: ftp://gcc.gnu.org/pub/gcc/infrastructure/isl-0.26.tar.bz2
  >> download failed: https://gcc.gnu.org/pub/gcc/infrastructure/isl-0.26.tar.bz2
  >> download succeeded: https://libisl.sourceforge.io/isl-0.26.tar.bz2

WARNING: Found file newlib-4.4.0.20231231.tar.gz at g/GCCcore/newlib-4.4.0.20231231.tar.gz, but re-downloading it anyway...

  >> download failed: https://ftp.gnu.org/gnu/gcc/gcc-14.2.0/newlib-4.4.0.20231231.tar.gz
  >> download failed: https://sourceware.org/pub/gcc/releases/gcc-14.2.0/newlib-4.4.0.20231231.tar.gz
  >> download failed: https://ftp.gnu.org/gnu/gmp/newlib-4.4.0.20231231.tar.gz
  >> download failed: https://ftp.gnu.org/gnu/mpfr/newlib-4.4.0.20231231.tar.gz
  >> download failed: https://ftp.gnu.org/gnu/mpc/newlib-4.4.0.20231231.tar.gz
  >> download failed: ftp://gcc.gnu.org/pub/gcc/infrastructure/newlib-4.4.0.20231231.tar.gz
  >> download failed: https://gcc.gnu.org/pub/gcc/infrastructure/newlib-4.4.0.20231231.tar.gz
  >> download failed: https://libisl.sourceforge.io/newlib-4.4.0.20231231.tar.gz
  >> download succeeded: https://sourceware.org/pub/newlib/newlib-4.4.0.20231231.tar.gz

WARNING: Found file nvptx-tools-20240801.tar.gz at g/GCCcore/nvptx-tools-20240801.tar.gz, but re-downloading it anyway...

  >> download failed: https://ftp.gnu.org/gnu/gcc/gcc-14.2.0/3136cf9.tar.gz
  >> download failed: https://sourceware.org/pub/gcc/releases/gcc-14.2.0/3136cf9.tar.gz
  >> download failed: https://ftp.gnu.org/gnu/gmp/3136cf9.tar.gz
  >> download failed: https://ftp.gnu.org/gnu/mpfr/3136cf9.tar.gz
  >> download failed: https://ftp.gnu.org/gnu/mpc/3136cf9.tar.gz
  >> download failed: ftp://gcc.gnu.org/pub/gcc/infrastructure/3136cf9.tar.gz
  >> download failed: https://gcc.gnu.org/pub/gcc/infrastructure/3136cf9.tar.gz
  >> download failed: https://libisl.sourceforge.io/3136cf9.tar.gz
  >> download failed: https://sourceware.org/pub/newlib/3136cf9.tar.gz
  >> download succeeded: https://github.com/MentorEmbedded/nvptx-tools/archive/3136cf9.tar.gz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants