UnexpectedValueException: continue truncated downloads

[traumschule]

This is a feature request to enable composer to continue/retry interrupted downloads.

It would be great to automatically continue downloads that failed with UnexpectedValueException. Currently composer bluntly raises an error instead of retrying with an offset of the already downloaded part (equivalent to wget -c).

Note that this would be especially useful for people with "slow" internet (~300 kbit/s) and "big" downloads of more than 40 MB. In our case composer install is called through provisioning a vagrant box with ansible and repeatedly fails at this specific tasks.

composer.json

{
  "repositories": [
    {
      "type": "package",
      "package": {
        "name": "wiki",
        "version": "1.29.2",
        "type": "project",
        "dist": {
          "url": "https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.2.tar.gz",
          "type": "tar"
        }
      }
    }
  ],
  "require": {
    "wiki": "*"
  }
}

composer diagnose

$ composer diagnose
Checking composer.json:
  [ErrorException] file_get_contents(/usr/share/php/Composer/../../data/Composer/res/spdx-licenses.json): failed to open stream: No such file or directory  

diagnose

$ composer --version
Composer version @package_branch_alias_version@ (1.0.0-beta2) 2016-03-27 16:00:34

composer install

Loading composer repositories with package information
Updating dependencies (including require-dev)

• Installing wiki (1.29.2)
  Downloading: 100%

[UnexpectedValueException]
phar error: "/var/www/test/vendor/wiki/41ce3a3312e82e1171878cd737e1c3ef.gz" is a corrupted tar file (truncated)

install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-plugins] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--ignore-platform-reqs] [--] []...

Steps to reproduce

apt-get install trickle
cat <<EOF > composer.json
{
  "repositories": [
    {
      "type": "package",
      "package": {
        "name": "wiki",
        "version": "1.29.2",
        "type": "project",
        "dist": {
          "url": "https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.2.tar.gz",
          "type": "tar"
        }
      }
    }
  ],
  "require": {
    "wiki": "*"
  }
}
EOF
trickle -d 50 composer install

[Seldaek]

I agree this sounds good but in practice I am not sure how easy it is to achieve given our current download infrastructure via http streams.. Maybe more a thing for 2.0 or later.

Alternatively what you could do is to make sure you download the files beforehand and include them with the VM, then use an artifact repo to install from instead of downloading during composer install.

[traumschule]

This is what I was told on IRC and I tried it with our ansible play which automates the installation of Mediawiki and several extensions. For downloading I used ansible's uri and the get uri module, both showed the same behavior: the downloaded file was truncated without error and composer phar failed with above message. At the end I added a checksum for the archive to our composer.json file and it worked instantly. Unfortunately this feature is not so well documented (I guessed it from the examples for alternative repositories).

[Seldaek]

Closing as I don't really see how we can reasonably do this.