Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

temporary sourceforge sources are being used #16900

Closed
RamKromberg opened this issue Jul 12, 2016 · 4 comments
Closed

temporary sourceforge sources are being used #16900

RamKromberg opened this issue Jul 12, 2016 · 4 comments

Comments

@RamKromberg
Copy link
Contributor

RamKromberg commented Jul 12, 2016

Multiple packages are broken over using sourceforge.net links which are temporary.
Some directly refer to a specific mirror so they succeed so long as the server isn't down-throttled by sourceforge. Others are completely broken. The correct syntax for sourceforge source packages should be "mirror://sourceforge/...".

The following is a partial list compiled from git grep sourceforge.net | grep tar and some light manual editing. It might have false negatives and there could be other offending packages that aren't listed:

pkgs/applications/backup/areca/default.nix:    url = "http://downloads.sourceforge.net/project/areca/areca-stable/areca-7.5/areca-7.5-src.tar.gz";
pkgs/applications/editors/emacs-modes/bbdb/default.nix:    url = "http://bbdb.sourceforge.net/${name}.tar.gz";
pkgs/applications/editors/ht/default.nix:    url = "http://sourceforge.net/projects/hte/files/ht-source/ht-${version}.tar.bz2";
pkgs/applications/graphics/potrace/default.nix:    url = "http://potrace.sourceforge.net/download/${version}/potrace-${version}.tar.gz";
pkgs/applications/misc/abook/default.nix:    url = "http://abook.sourceforge.net/devel/${name}.tar.gz";
pkgs/applications/misc/gammu/default.nix:    url = "http://sourceforge.net/projects/gammu/files/gammu/${version}/gammu-${version}.tar.xz";
pkgs/applications/misc/mlterm/default.nix:    url = "https://downloads.sourceforge.net/project/mlterm/01release/${name}/${name}.tar.gz";
pkgs/applications/misc/xautoclick/default.nix:    url = "http://downloads.sourceforge.net/project/xautoclick/xautoclick/xautoclick-0.31/xautoclick-0.31.tar.gz";
pkgs/applications/science/math/fricas/default.nix:    url    = "http://sourceforge.net/projects/fricas/files/fricas/1.2.2/${name}-full.tar.bz2";
pkgs/applications/science/math/yacas/default.nix:    url = "http://yacas.sourceforge.net/backups/${name}.tar.gz";
pkgs/applications/window-managers/windowmaker/dockapps/wmsystemtray.nix:     url = http://sourceforge.net/projects/wmsystemtray/files/wmsystemtray/wmsystemtray-1.4.tar.gz;
pkgs/build-support/upstream-updater/snippets.sh:#http://downloads.sourceforge.net/webdruid/files/webdruid/0.6.0-alpha5/webdruid-0.6.0-alpha5.tar.gz
pkgs/desktops/lxde/core/lxmenu-data.nix:    url = "http://downloads.sourceforge.net/lxde/${name}.tar.xz";
pkgs/development/compilers/fpc/binary.nix:        url = "http://sourceforge.net/projects/freepascal/files/Linux/2.6.0/fpc-2.6.0.i386-linux.tar";
pkgs/development/compilers/fpc/binary.nix:        url = "http://sourceforge.net/projects/freepascal/files/Linux/2.6.0/fpc-2.6.0.x86_64-linux.tar";
pkgs/development/compilers/mozart/binary.nix:    url = "http://sourceforge.net/projects/mozart-oz/files/v${version}-alpha.0/mozart2-${version}-alpha.0+build.4105.5c06ced-x86_64-linux.tar.gz";
pkgs/development/libraries/armadillo/default.nix:    url = "http://sourceforge.net/projects/arma/files/armadillo-${version}.tar.xz";
pkgs/development/libraries/libgringotts/default.nix:    url = "http://libgringotts.sourceforge.net/current/${name}.tar.bz2";
pkgs/development/libraries/libmpeg2/default.nix:    url = "http://libmpeg2.sourceforge.net/files/${name}.tar.gz";
pkgs/development/libraries/plib/default.nix:    url = "http://plib.sourceforge.net/dist/${name}.tar.gz";
pkgs/development/tools/misc/dfu-util/default.nix:    url = "http://dfu-util.sourceforge.net/releases/${name}.tar.gz";
pkgs/development/tools/omniorb/default.nix:    url = "http://sourceforge.net/projects/omniorb/files/omniORB/omniORB-${version}/omniORB-${version}.tar.bz2";
pkgs/games/blobby/default.nix:    url = "http://softlayer-ams.dl.sourceforge.net/project/blobby/Blobby%20Volley%202%20%28Linux%29/1.0/blobby2-linux-1.0.tar.gz";
pkgs/games/mnemosyne/default.nix:    url    = "http://sourceforge.net/projects/mnemosyne-proj/files/mnemosyne/${name}/Mnemosyne-${version}.tar.gz";
pkgs/games/typespeed/default.nix:    url = http://typespeed.sourceforge.net/typespeed-0.6.5.tar.gz;
pkgs/os-specific/linux/procps/watch.nix:    url = http://procps.sourceforge.net/procps-3.2.8.tar.gz;
pkgs/servers/dict/dictd-db.nix:     url = http://prdownloads.sourceforge.net/freedict/nld-eng.tar.gz;
pkgs/tools/compression/advancecomp/default.nix:    url="http://prdownloads.sourceforge.net/advancemame/advancecomp-1.19.tar.gz?download";
pkgs/tools/filesystems/fuseiso/default.nix:    url = "http://sourceforge.net/projects/fuseiso/files/fuseiso/20070708/fuseiso-20070708.tar.bz2";
pkgs/tools/filesystems/jfsutils/default.nix:    url = "http://jfs.sourceforge.net/project/pub/${name}.tar.gz";
pkgs/tools/filesystems/nilfs-utils/default.nix:    url = "http://nilfs.sourceforge.net/download/nilfs-utils-${version}.tar.bz2";
pkgs/tools/misc/cutecom/default.nix:    url = "http://cutecom.sourceforge.net/${name}.tar.gz";
pkgs/tools/networking/srelay/default.nix:    url = "https://sourceforge.net/projects/socks-relay/files/socks-relay/srelay-0.4.8/srelay-0.4.8b6.tar.gz";
pkgs/tools/system/foremost/default.nix:    url = "http://foremost.sourceforge.net/pkg/${name}.tar.gz";
pkgs/tools/text/poedit/default.nix:    url = "http://prdownloads.sourceforge.net/poedit/${name}.tar.gz";
pkgs/top-level/python-packages.nix:      url = "http://cddb-py.sourceforge.net/${name}.tar.gz";
pkgs/top-level/python-packages.nix:      url = "http://downloads.sourceforge.net/project/cgkit/cgkit/cgkit-${version}/cgkit-${version}-py2k.tar.gz";
pkgs/top-level/python-packages.nix:      url = "http://pycurl.sourceforge.net/download/${name}.tar.gz";

Also note that occasionally sourceforge crashes to "disaster recovery mode" and requires javascript to download tarballs.

@phanimahesh
Copy link
Contributor

Is adding a travis check to prevent addition of sourceforge.net links a good idea?

@RamKromberg
Copy link
Contributor Author

RamKromberg commented Jul 14, 2016

@phanimahesh The problem is that downloads.sourceforge.net is actually the official way to download tarballs from SourceForge. What makes it so unreliable is the same ticketing and load-balancing system SourceForge wants us to use...

That's to say, before modifying travis, the list in pkgs/build-support/fetchurl/mirrors.nix should be edited to start with the official downloads.sourceforge.net to give SF a chance to regulate itself, and only fallback on the known servers list if the download fails or hangs.

@phanimahesh
Copy link
Contributor

No wonder people don't like SF. :/

Okay, I'm opening a new PR to update mirrors. Will follow up with editing out the sourceforge.net links to use mirrors, and then add a travis job to enforce the convention.

@RamKromberg
Copy link
Contributor Author

No wonder people don't like SF. :/

We actually lucked out here. Windows users get their binaries packed with extra malware...

Okay, I'm opening a new PR to update mirrors. Will follow up with editing out the sourceforge.net links to use mirrors, and then add a travis job to enforce the convention.

Sounds great :) I'll close this issue now that you're on it. Thanks!

phanimahesh added a commit to phanimahesh/nixpkgs that referenced this issue Jul 15, 2016
downloads.sourceforge.net is the official way to download tarballs from
SourceForge.  However, it is reported as unreliable due to SF's weird
load balancing system.

This commit gives the official mirror utmost priority, and will use
other configured mirrors (which may be temporary) as a fallback only
when the official one can't be reached/download fails/hangs.

References: NixOS#16900
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants