The repo.magento.com server's HTTPS/SSL are Not Compatible with OS X

[astorm]

Steps to reproduce

1. Get an OS X machine running Yosemite and latest a version of PHP from http://php-osx.liip.ch/
2. Ensure latest composer is running with composer selfupdate
3. Run composer create-project --repository=https://repo.magento.com/ magento/project-community-edition

Expected result

1. Composer Downloads Magento

Actual result

1. Download fails with the composer error "The "https://repo.magento.com/packages.json" file could not be downloaded: Failed to enable crypto"
2. Developer silently weeps

---

Full composer exception below.

After some debugging in Composer, it appears that the SSL/HTTPS implementation on https://repo.magento.com is incompatible with OS X Yosemite's SSL/HTTPS encryption.

Simplifying the calls composer makes produces similar errors for repo.magento.com. i.e. this PHP script

$context = stream_context_create();
$contents = file_get_contents('https://repo.magento.com/packages.json', false, $context);

produces the error

PHP Warning: file_get_contents(https://repo.magento.com/packages.json): failed to open stream

I'm sure someone there is already thinking something something Apple's fault. However, this only happens with repo.magento.com. i.e., I can still update via the Packagist HTTPS server, and my PHP can download an HTTPS packagist packages.json. Again, breaking it down to a simple script -- this works without the failed crypto error.

<?php
$context = stream_context_create();
$contents = file_get_contents('https://packagist.org/packages.json', false, $context);

I realize this is a hard problem to track down, but if The World's Biggest Composer Repository (i.e. the de-facto standard) works with Apple's crypto, it seems like a reasonable expectation that Magento's should do this same.

"Use a VM" is a reasonable suggestion for a work around, but not an acceptable solution to this problem. "Use a mirror" is a reasonable suggestion for a work around, but not an acceptable solution to this problem.

The aforementioned composer exception traces follow.

[Composer\Downloader\TransportException]                                           
The "https://repo.magento.com/packages.json" file could not be downloaded: Failed  
to enable crypto                                                                  
failed to open stream: operation failed                                            


Exception trace:
() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Util/RemoteFilesystem.php:464
Composer\Util\RemoteFilesystem->get() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Util/RemoteFilesystem.php:101
Composer\Util\RemoteFilesystem->getContents() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Repository/ComposerRepository.php:645
Composer\Repository\ComposerRepository->fetchFile() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Repository/ComposerRepository.php:462
Composer\Repository\ComposerRepository->loadRootServerFile() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Repository/ComposerRepository.php:256
Composer\Repository\ComposerRepository->hasProviders() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/DependencyResolver/Pool.php:99
Composer\DependencyResolver\Pool->addRepository() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Command/CreateProjectCommand.php:287
Composer\Command\CreateProjectCommand->installRootPackage() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Command/CreateProjectCommand.php:156
Composer\Command\CreateProjectCommand->installProject() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Command/CreateProjectCommand.php:143
Composer\Command\CreateProjectCommand->execute() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/vendor/symfony/console/Command/Command.php:259
Symfony\Component\Console\Command\Command->run() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/vendor/symfony/console/Application.php:844
Symfony\Component\Console\Application->doRunCommand() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/vendor/symfony/console/Application.php:192
Symfony\Component\Console\Application->doRun() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Console/Application.php:227
Composer\Console\Application->doRun() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/vendor/symfony/console/Application.php:123
Symfony\Component\Console\Application->run() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/src/Composer/Console/Application.php:102
Composer\Console\Application->run() at phar:///Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar/bin/composer:43
require() at /Users/alanstorm/Documents/github_private/GeneralCode/bin/composer.phar:24

create-project [-s|--stability STABILITY] [--prefer-source] [--prefer-dist] [--repository REPOSITORY] [--repository-url REPOSITORY-URL] [--dev] [--no-dev] [--no-custom-installers] [--no-scripts] [--no-progress] [--no-secure-http] [--keep-vcs] [--no-install] [--ignore-platform-reqs] [--] [<package>] [<directory>] [<version>]

[maksek]

Hi @astorm, just to check, have you installed openssl?

[astorm]

@maksek Yes.

[maksek]

Just did fresh install on my iMac (i used macports):

$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.11.5
BuildVersion: 15F34

~/dev/www/magento2-composer-last$ composer.phar create-project --repository=https://repo.magento.com/ magento/project-community-edition
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
    Authentication required (repo.magento.com):                                                                                 
      Username: b61a6b44761799381efede7ce1ca9bfb
      Password: 
Do you want to store credentials for repo.magento.com in /Users/maksek/.composer/auth.json ? [Yn] Y
Installing magento/project-community-edition (2.1.0)
  - Installing magento/project-community-edition (2.1.0)
    Downloading: 100%         

It works fine. Doesn't look like repo issue.

[hostep]

Hi @astorm

It looks like they use a pretty modern ciphersuite and protocols on repo.magento.com, which only allows TLS 1.2 and 1.1 connections: https://www.ssllabs.com/ssltest/analyze.html?d=repo.magento.com&s=52.203.74.110&latest

There is a possibility that the OpenSSL version your php installation is linked against, doesn't support those protocols yet.

You could try running these commands to figure out if your OpenSSL is able to connect to Magento's composer repo:

$ openssl s_client -connect repo.magento.com:443 -tls1_2 # this one should connect
$ openssl s_client -connect repo.magento.com:443 -tls1_1 # this one should connect
$ openssl s_client -connect repo.magento.com:443 -tls1   # this one shouldn't connect

If the two first two command don't work on your system, then OpenSSL is most likely to blame.

After some researching, it looks like the built-in PHP version of OS X is linked against LibreSSL, instead of OpenSSL, which does support TLS 1.1 and 1.2. The OpenSSL version on OS X is really really old and doesn't support TLS 1.1 or 1.2. So that might explain it.

If you are somehow able to link your PHP installation against the LibreSSL version of OS X, then you might get it to work. Or somehow get a more recent version of OpenSSL on your machine (without overwriting the systems default) and linking it against your PHP installation.

So I guess the issue is in the PHP packages from http://php-osx.liip.ch/, maybe open a support ticket over there?

I myself always use Macports to install PHP versions on my OS X machines: https://www.macports.org/ports.php?by=name&substr=php7
And those get linked against the very latest version of OpenSSL (which Macports also installs)

But if you never worked with Macports before, it might be a bit scary and it might take you a while to figure it out and setup everything properly.

I hope this helps.

[24198]

@hostep

An issue has been reported for the old OpenSSL version at liip/php-osx#175. Currently running into the same problems as @astorm, I've updated OpenSSL but it looks like the PHP version of liip's package is compiled with default OpenSSL of OS X. After running phpinfo(); on my current setup (which is in fact a version of liip's osx, php-5.6.21) I can see that the configure command has the option --with-openssl the only disadvantage is that it's pointing to /usr.

Snippet from phpinfo();
'./configure' '--prefix=/usr/local/php5' '--with-apxs2=/usr/sbin/apxs' '--with-config-file-scan-dir=/usr/local/php5/php.d' '--with-libxml-dir=shared,/usr' '--with-openssl=/usr'

[maksek]

TLS 1.0 is disabled on repo.magento.com due security. Good summary is here - https://blog.varonis.com/ssl-and-tls-1-0-no-longer-acceptable-for-pci-compliance/

[astorm]

@maksek Re: "working for you" -- that reminds of the time, years ago, when I made the team I was on start putting a dollar in a jar anytime they said "Works on my machine" I'm guessing you're installing php where you're building or linking against a custom installed version of PHP? The packages I'm using, and others with the same problem are using, looks like its built using whatever version os SSL ships on OS standard. (http://php-osx.liip.ch/)

Re: TLS 1.0 -- I'm not sure I follow. Disabling TLS 1.0 on servers that process credit cards makes a whole lots of sense -- but we're talking about a composer repository that distributes source code. Is this really required, or is it just it just easier? If its really required, does that mean you're going to remove the source from GitHub.com?

[astorm]

@hostep Yeah, it's the packages on http://php-osx.liip.ch/ --- honestly, I've burned a few days on this, I'll just resort to work arounds for my own work rather than fight upstream on this. If Magento wants to move to a distribution model that blocks a stock OS X and a popular (possibly the most popular?) PHP distribution channel that's their choice. It sends a clear message though that they're looking to build a less open development ecosystem.

[maksek]

So, thanks for @hostep, @24198 for finding issue liip/php-osx#175 on http://php-osx.liip.ch/ which @astorm uses.
Since it is Magento 2 github, and the issue not related to Magento 2, @astorm any more comments?

[astorm]

@maksek While you have a situation here where you can easily point to technical issue that's not yours, your assessment that it's not related to Magento 2 is incorrect. With things setup as they are right now an unknown number (thousands? Tens of thousands?) of OS X can't access Magento 2 via composer.

That's horrible from a product point of view. For developers already working in the ecosystem this means making massive changes to how they work. For developers not working in the ecosystem it creates a horrible first impression that will keep an unknown percentage of them away.

If that's an acceptable tradeoff for your team, then you can close this ticket. If you're not comfortable with that, I'd recommend you product teams takes steps, lateral or otherwise, to fix it. Two suggestions are to provide a composer repository where these developers can access the source code, or by working with the upstream provider (http://php-osx.liip.ch/) themselves to get this fixed.

[maksek]

Thanks for suggestions @astorm, we will investigate them.

[24198]

@maksek so far I've seen two affected packages / setups where this issue applies. One is the php-osx liip package and the other one is MAMP. Both come with compiled php versions, they are compiled with the older OpenSSL 0.9.8x version.
I've created a small overview with the php versions / affected setups:

PHP Version/Setup	PHP LIIP OSX	MAMP (PRO)
5.6.10	N/A	x
5.6.17	x	N/A
5.6.19	x	N/A
5.6.21	x	N/A
7.0.0	x	x
7.0.7	x	N/A

Hopefully this helps :)

Kind regards,

Jaimy

[chregu]

I'm working on a fix (for php-osx), looking good that this will be available soon