Prefetching packages: Failed to download from dist

[ihmels]

I have a problem installing dependencies in a Vagrant machine behind a corporate proxy.

The problem occurs when the packages are prefetched after Flex is downloaded:

Failed to download doctrine/lexer from dist: "https://api.github.com/repos/doctrine/lexer/zipball/83893c552fd2045dd78aef794c31e694c37c0b8c" appears broken, and returned an empty 200 response
Now trying to download from source

If I disable Flex during installation with composer install --no-plugins everything is OK.

A similar problem also occurs with composer update (composer update --no-plugins is OK):

$ composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 86 installs, 0 updates, 0 removals
  - Installing symfony/flex (v1.2.1): Downloading (100%)
"https://repo.packagist.org/packages.json" does not contain valid JSON
Parse error on line 1:

^
Expected one of: 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
https://repo.packagist.org could not be fully loaded, package information was loaded from the local cache and may be out of date

In both cases Composer receives an empty response. Composer diagnose:

$ composer diagnose
Checking composer.json: OK
Checking platform settings: OK
Checking git settings: OK
Checking http connectivity to packagist: OK
Checking https connectivity to packagist: OK
Checking HTTP proxy: OK
Checking HTTP proxy support for request_fulluri: OK
Checking HTTPS proxy support for request_fulluri: OK
Checking github.com rate limit: OK
Checking disk free space: OK
Checking pubkeys: FAIL
Missing pubkey for tags verification
Missing pubkey for dev verification
Run composer self-update --update-keys to set them up
Checking composer version: OK
Composer version: 1.8.4
PHP version: 7.2.17
PHP binary path: /opt/remi/php72/root/usr/bin/php

[ihmels]

I found out something: The problem seems to occur since PHP 7.2.17. With PHP 7.2.16 it works fine.

[merigold]

PHP 7.3.4 problem exist.
I try flex 1.2.1, 1.2.0, 1.1.8, the same

[loiclavoie]

Having the same issue. We also notice that the problem seems to diseapear if we disable tls.

composer config disable-tls false

Also, it seem's to only happen with dist on api.github.com. We have private one in our project and they don't have this issue.

[nicolas-grekas]

looks like a PHP bug
can you reproduce when disabling the curl extension? or when enabling it?

[loiclavoie]

@nicolas-grekas curl extension cannot be removed on our server due to other features that requires it. But also, php-curl seem's to be part of php-commons now (with CentOS Remi-PHP7.2). So I cant' remove it.

[loiclavoie]

@nicolas-grekas If I don't install Flex, I don't get that error. Composer uses file_get_content to obtain a dist and flex is using curl. So it's seems to be indeed related to the usage of curl. Pass that point, I don't have more information

[merigold]

Whetn I try php 7.2.14 is ok, 7.2.17 and 7.3.4 cause this error.
Workaraound
composer config disable-tls true

[loiclavoie]

@merigold @ihmels I have make it work on our server for now, but it require downgrading php to version 7.2.16. Currently 7.2.17 contains an issue with CURL which cause the problem.

I've tested every bug declared on PHP.net and none of them seem's to be the cause.

You can downgrade if you are using Remi's package by doing

yum downgrade php-*

[ihmels]

@loiclavoie Unfortunately I can't downgrade PHP, because I just provisioned the VM.

@nicolas-grekas If I disable the cURL extension, it does work without problems.

[phoenix1337]

It looks like it has to do with the following fix which now returns an empty string instead of the buffer:
php/php-src@5025eb0

composer install with php 7.2.16 gives me the following errors:
Retrying download: Unknown cipher in list: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:E

If I execute curl --ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:E https://github.com/php/php-src/commit/5025eb05bde83a0a51eb0668c45c240b366545bf it gives me exactly the same error.

The ciphers are passed in

flex/src/CurlDownloader.php

Line 33 in 7d488f0

'ciphers' => CURLOPT_SSL_CIPHER_LIST,

It looks like it's working on 7.2.17 when this line is commented

I am not sure what causes this error exactly but this is what I found so far.

[nicolas-grekas]

Good catch @phoenix1337, would you mind sending a PR to remove this line? It's not needed anymore since default ciphers are usually good.

[remicollet]

php-curl seem's to be part of php-commons now (with CentOS Remi-PHP7.2). So I cant' remove it.

@loiclavoie you can disable it in 20-curl.ini

[remicollet]

to remove this line? It's not needed anymore since default ciphers are usually good.

@nicolas-grekas +1

Especially as changing default value from code is a terribly bad idea.

On OS where security is properly managed, this is event forbidden, (in packages distributed with the distribution), read
https://www.redhat.com/en/blog/consistent-security-crypto-policies-red-hat-enterprise-linux-8

[phoenix1337]

I have created a pull request but it's failing because of coding standards on lines I did not change.

The problem with the constant exposing the ciphers is that it contains OpenSSL ciphers. On Centos the curl package is build with nss so they don't match. I don't know if this is a curl or php issue though.