New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wp core download gives "unable to decompress gzipped phar archive" error #4370

Closed
Flimm opened this Issue Sep 20, 2017 · 80 comments

Comments

@Flimm

Flimm commented Sep 20, 2017

We have a nightly script that downloads and reinstalls WordPress on a testing server. Last night, it failed like this:

$ wp core --debug download
Debug (bootstrap): No readable global config found (0.067s)
Debug (bootstrap): No project config found (0.068s)
Debug (bootstrap): argv: /usr/bin/wp core --debug download (0.068s)
Debug (bootstrap): ABSPATH defined: /var/www/html/ (0.068s)
Debug (bootstrap): Running command: core download (0.069s)
Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c231a7990f7.tar.gz" to temporary file

I've checked that I have not run out of free disk space. I have also checked that the temporary file is a valid gzipped tarball:

$ cat /tmp/wp_59c231a7990f7.tar.gz | file -
/dev/stdin: gzip compressed data, last modified: Tue Sep 19 21:21:21 2017, from Unix
$ zcat /tmp/wp_59c231a7990f7.tar.gz | file -
/dev/stdin: POSIX tar archive (GNU)
$  tar tf /tmp/wp_59c231a7990f7.tar.gz | head
wordpress/
wordpress/wp-settings.php
wordpress/wp-cron.php
wordpress/wp-comments-post.php
wordpress/wp-activate.php
wordpress/wp-admin/
wordpress/wp-admin/link-parse-opml.php
wordpress/wp-admin/js/
wordpress/wp-admin/js/editor.js
wordpress/wp-admin/js/user-profile.min.js

Here is some more debugging information:

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.3 LTS
Release:	16.04
Codename:	xenial
$ php -v
PHP 7.0.22-0ubuntu0.16.04.1 (cli) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.0.22-0ubuntu0.16.04.1, Copyright (c) 1999-2017, by Zend Technologies
$ wp --version
WP-CLI 1.3.0
@OnnoGeorg

This comment has been minimized.

Show comment
Hide comment
@OnnoGeorg

OnnoGeorg Sep 20, 2017

Same problem here on Linux Mint and PHP 5.6.

$ lsb_release -a  
No LSB modules are available.  
Distributor ID:	LinuxMint  
Description:	Linux Mint 18.1 Serena  
Release:	18.1  
Codename:	serena
$ php -v
PHP 5.6.31-2+ubuntu16.04.1+deb.sury.org+1 (cli) 
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
    with Xdebug v2.5.5, Copyright (c) 2002-2017, by Derick Rethans
$ wp --version
WP-CLI 1.3.0

OnnoGeorg commented Sep 20, 2017

Same problem here on Linux Mint and PHP 5.6.

$ lsb_release -a  
No LSB modules are available.  
Distributor ID:	LinuxMint  
Description:	Linux Mint 18.1 Serena  
Release:	18.1  
Codename:	serena
$ php -v
PHP 5.6.31-2+ubuntu16.04.1+deb.sury.org+1 (cli) 
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
    with Xdebug v2.5.5, Copyright (c) 2002-2017, by Derick Rethans
$ wp --version
WP-CLI 1.3.0
@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

I have same problem with latest nightly version.

$ wp core download --path=/var/www/html/ --version=latest --force
Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c24f71218a2.tar.gz" to temporary file
Member

miya0001 commented Sep 20, 2017

I have same problem with latest nightly version.

$ wp core download --path=/var/www/html/ --version=latest --force
Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c24f71218a2.tar.gz" to temporary file
@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

It is working on MacOS, but not working on Ubuntu.
Following is a result of behat.

$ vendor/bin/behat features/core-download.feature 

PHP binary:	/usr/bin/php
PHP version:	7.0.22-0ubuntu0.16.04.1
php.ini used:	/etc/php/7.0/cli/php.ini
WP-CLI root dir:	/vagrant/core-command/vendor/wp-cli/wp-cli
WP-CLI vendor dir:	/vagrant/core-command/vendor
WP_CLI phar path:	
WP-CLI packages dir:	
WP-CLI global config:	/tmp/wp-cli-package-test/config.yml
WP-CLI project config:	/vagrant/core-command/wp-cli.yml
WP-CLI version:	1.3.0


                                                                                                                                         
  [RuntimeException]                                                                                                                     
  Exception has been thrown in "beforeSuite" hook, defined in FeatureContext::prepare()                                                  
  $ wp core download --force --path='/tmp/wp-cli-test-core-download-cache'                                                               
  Creating directory '/tmp/wp-cli-test-core-download-cache/'.                                                                            
  Downloading WordPress 4.8.2 (en_US)...                                                                                                 
  md5 hash verified: 2e8744a702a3d9527782d9135a4c9544                                                                                    
  Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c256f2e306e.tar.gz" to temporary file  
  cwd:                                                                                                                                   
  run time: 13.057976961136                                                                                                              
  exit status: 1                                                                                                                         
                                                                                                                                         

behat [--init] [-f|--format="..."] [--out="..."] [--lang="..."] [--[no-]ansi] [--[no-]time] [--[no-]paths] [--[no-]snippets] [--[no-]snippets-paths] [--[no-]multiline] [--[no-]expand] [--story-syntax] [-d|--definitions="..."] [--name="..."] [--tags="..."] [--cache="..."] [--strict] [--dry-run] [--stop-on-failure] [--rerun="..."] [--append-snippets] [--append-to="..."] [features]
Member

miya0001 commented Sep 20, 2017

It is working on MacOS, but not working on Ubuntu.
Following is a result of behat.

$ vendor/bin/behat features/core-download.feature 

PHP binary:	/usr/bin/php
PHP version:	7.0.22-0ubuntu0.16.04.1
php.ini used:	/etc/php/7.0/cli/php.ini
WP-CLI root dir:	/vagrant/core-command/vendor/wp-cli/wp-cli
WP-CLI vendor dir:	/vagrant/core-command/vendor
WP_CLI phar path:	
WP-CLI packages dir:	
WP-CLI global config:	/tmp/wp-cli-package-test/config.yml
WP-CLI project config:	/vagrant/core-command/wp-cli.yml
WP-CLI version:	1.3.0


                                                                                                                                         
  [RuntimeException]                                                                                                                     
  Exception has been thrown in "beforeSuite" hook, defined in FeatureContext::prepare()                                                  
  $ wp core download --force --path='/tmp/wp-cli-test-core-download-cache'                                                               
  Creating directory '/tmp/wp-cli-test-core-download-cache/'.                                                                            
  Downloading WordPress 4.8.2 (en_US)...                                                                                                 
  md5 hash verified: 2e8744a702a3d9527782d9135a4c9544                                                                                    
  Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c256f2e306e.tar.gz" to temporary file  
  cwd:                                                                                                                                   
  run time: 13.057976961136                                                                                                              
  exit status: 1                                                                                                                         
                                                                                                                                         

behat [--init] [-f|--format="..."] [--out="..."] [--lang="..."] [--[no-]ansi] [--[no-]time] [--[no-]paths] [--[no-]snippets] [--[no-]snippets-paths] [--[no-]multiline] [--[no-]expand] [--story-syntax] [-d|--definitions="..."] [--name="..."] [--tags="..."] [--cache="..."] [--strict] [--dry-run] [--stop-on-failure] [--rerun="..."] [--append-snippets] [--append-to="..."] [features]
@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001
Member

miya0001 commented Sep 20, 2017

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

This will work

wp --allow-root core download --locale=sv_SE

The problem is with US files ...

ZeroX66 commented Sep 20, 2017

This will work

wp --allow-root core download --locale=sv_SE

The problem is with US files ...

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

Oh, yes...
It looks package problem.

$ ./bin/wp core download --path=/home/vagrant/wp --locale=sv_SE --force
Downloading WordPress 4.8.2 (sv_SE)...
md5 hash verified: bfc6e58c9018e4344620caae85323547
Cleaning up files...
File removed: wp-includes/js/tinymce/skins/lightgray/fonts/readme.md
File removed: wp-includes/js/tinymce/skins/lightgray/fonts/tinymce.json
File removed: wp-includes/js/tinymce/skins/lightgray/fonts/tinymce-small.json
File removed: wp-includes/js/tinymce/skins/lightgray/skin.ie7.min.css
File removed: wp-includes/js/tinymce/plugins/media/moxieplayer.swf
File removed: wp-includes/js/tinymce/plugins/wpembed/plugin.js
File removed: wp-includes/js/tinymce/plugins/wpembed/plugin.min.js
7 files cleaned up.
Success: WordPress downloaded.
Member

miya0001 commented Sep 20, 2017

Oh, yes...
It looks package problem.

$ ./bin/wp core download --path=/home/vagrant/wp --locale=sv_SE --force
Downloading WordPress 4.8.2 (sv_SE)...
md5 hash verified: bfc6e58c9018e4344620caae85323547
Cleaning up files...
File removed: wp-includes/js/tinymce/skins/lightgray/fonts/readme.md
File removed: wp-includes/js/tinymce/skins/lightgray/fonts/tinymce.json
File removed: wp-includes/js/tinymce/skins/lightgray/fonts/tinymce-small.json
File removed: wp-includes/js/tinymce/skins/lightgray/skin.ie7.min.css
File removed: wp-includes/js/tinymce/plugins/media/moxieplayer.swf
File removed: wp-includes/js/tinymce/plugins/wpembed/plugin.js
File removed: wp-includes/js/tinymce/plugins/wpembed/plugin.min.js
7 files cleaned up.
Success: WordPress downloaded.
@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001
Member

miya0001 commented Sep 20, 2017

@gitlost

This comment has been minimized.

Show comment
Hide comment
@gitlost

gitlost Sep 20, 2017

Contributor

@miya0001 I think it's more likely to be a PHP / PharData memory problem or something, and due to a recent change. It fails for me with PHP 7.0.23 and 7.1.9 but succeeds with PHP 7.0.18, PHP 5.6.22, PHP 5.5.38 (all on Ubuntu 17.04) and PHP 7.0.19 (Windows).

What version of php did you run on MacOS?

Contributor

gitlost commented Sep 20, 2017

@miya0001 I think it's more likely to be a PHP / PharData memory problem or something, and due to a recent change. It fails for me with PHP 7.0.23 and 7.1.9 but succeeds with PHP 7.0.18, PHP 5.6.22, PHP 5.5.38 (all on Ubuntu 17.04) and PHP 7.0.19 (Windows).

What version of php did you run on MacOS?

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

@gitlost :)

$ php --version
PHP 7.1.8 (cli) (built: Aug  7 2017 15:02:45) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
Member

miya0001 commented Sep 20, 2017

@gitlost :)

$ php --version
PHP 7.1.8 (cli) (built: Aug  7 2017 15:02:45) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
@gitlost

This comment has been minimized.

Show comment
Hide comment
@gitlost

gitlost Sep 20, 2017

Contributor

Very suspicious!

Contributor

gitlost commented Sep 20, 2017

Very suspicious!

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

I think the problem might be the arguments in this line

tar xz --strip-components=1 --directory=%s -f $tarball

When you use wp core download you get a temp archive in /tmp folder. Try to unpack it with these arguments like
tar xz --strip-components=1 --directory=wordpress -f your_archive.tar.gz

I can't make it work , but if I use a much simple line like this it's work.
tar -xvzf wp_59c228df6f742.tar.gz

I think the US package have som issue here ...

ZeroX66 commented Sep 20, 2017

I think the problem might be the arguments in this line

tar xz --strip-components=1 --directory=%s -f $tarball

When you use wp core download you get a temp archive in /tmp folder. Try to unpack it with these arguments like
tar xz --strip-components=1 --directory=wordpress -f your_archive.tar.gz

I can't make it work , but if I use a much simple line like this it's work.
tar -xvzf wp_59c228df6f742.tar.gz

I think the US package have som issue here ...

@Sidsector9

This comment has been minimized.

Show comment
Hide comment
@Sidsector9

Sidsector9 Sep 20, 2017

Member

I think the compression is faulty.
Steps I followed:

  1. Downloaded the en_US version
  2. $phar = new PharData( 'wordpress-4.8.2.tar.gz' ); echo $phar->getFileName();
    This failed
  3. Extracted the tarball manually and compressed it again.
  4. Tried step 2 and it worked perfectly.
Member

Sidsector9 commented Sep 20, 2017

I think the compression is faulty.
Steps I followed:

  1. Downloaded the en_US version
  2. $phar = new PharData( 'wordpress-4.8.2.tar.gz' ); echo $phar->getFileName();
    This failed
  3. Extracted the tarball manually and compressed it again.
  4. Tried step 2 and it worked perfectly.
@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

Can we made so WP core download use default locale instead of US?

ZeroX66 commented Sep 20, 2017

Can we made so WP core download use default locale instead of US?

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

Hmm it is working on MacOS.

$ wp core download --version=latest --path=wordpress --force
Downloading WordPress 4.8.2 (en_US)...
Using cached file '/Users/miyauchi/.wp-cli/cache/core/wordpress-4.8.2-en_US.tar.gz'...
Success: WordPress downloaded.

And I can extract tar.gz on linux.

I am not sure is it problem of the package or not.

Can we made so WP core download use default locale instead of US?

The default locale is en_US.

Member

miya0001 commented Sep 20, 2017

Hmm it is working on MacOS.

$ wp core download --version=latest --path=wordpress --force
Downloading WordPress 4.8.2 (en_US)...
Using cached file '/Users/miyauchi/.wp-cli/cache/core/wordpress-4.8.2-en_US.tar.gz'...
Success: WordPress downloaded.

And I can extract tar.gz on linux.

I am not sure is it problem of the package or not.

Can we made so WP core download use default locale instead of US?

The default locale is en_US.

@andreapinti

This comment has been minimized.

Show comment
Hide comment
@andreapinti

andreapinti Sep 20, 2017

Hi,
same problem here (ubuntu 14)
Tried with php 5.6, php 7.0 and php 7.1

andreapinti commented Sep 20, 2017

Hi,
same problem here (ubuntu 14)
Tried with php 5.6, php 7.0 and php 7.1

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

I agree, but it should be some misbehavior in the argument vs the package. Like I told before I can use "tar" with a simple line. The arguments troubled it up ...

ZeroX66 commented Sep 20, 2017

I agree, but it should be some misbehavior in the argument vs the package. Like I told before I can use "tar" with a simple line. The arguments troubled it up ...

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

Can we change the default locale to our own lang?

ZeroX66 commented Sep 20, 2017

Can we change the default locale to our own lang?

@OnnoGeorg

This comment has been minimized.

Show comment
Hide comment
@OnnoGeorg

OnnoGeorg Sep 20, 2017

Yes, the problem is with the US package - at least the German one runs well with wp core download --locale=de_DE

The Swedish as well as @ZeroX66 wrote.

OnnoGeorg commented Sep 20, 2017

Yes, the problem is with the US package - at least the German one runs well with wp core download --locale=de_DE

The Swedish as well as @ZeroX66 wrote.

@gitlost

This comment has been minimized.

Show comment
Hide comment
@gitlost

gitlost Sep 20, 2017

Contributor

If it's a memory problem or something then different tars could randomly work.

@DeaZy07 could you give the full PHP version numbers please?

(@ZeroX66 I tried with the exact same arguments tar xz --strip-components=1 --directory=%s -f $tarball and it worked (and changed the source to fall back to this on a PharData exception and it worked also).)

Contributor

gitlost commented Sep 20, 2017

If it's a memory problem or something then different tars could randomly work.

@DeaZy07 could you give the full PHP version numbers please?

(@ZeroX66 I tried with the exact same arguments tar xz --strip-components=1 --directory=%s -f $tarball and it worked (and changed the source to fall back to this on a PharData exception and it worked also).)

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

This is bad coz we use Easy Engine and EE use wp-cli to get the WordPress. So maybe we should talk with these who update WP in US.
What is the differens between a sv/de vs us, I wonder?

ZeroX66 commented Sep 20, 2017

This is bad coz we use Easy Engine and EE use wp-cli to get the WordPress. So maybe we should talk with these who update WP in US.
What is the differens between a sv/de vs us, I wonder?

@andreapinti

This comment has been minimized.

Show comment
Hide comment
@andreapinti

andreapinti Sep 20, 2017

@gitlost sure

PHP 7.1.9-1+ubuntu14.04.1+deb.sury.org+1 (cli) (built: Sep  2 2017 06:17:11) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.1.9-1+ubuntu14.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
 PHP 7.0.21-1+ubuntu14.04.1+deb.sury.org+1 (cli) (built: Jul  6 2017 09:38:10) ( NTS )
 Copyright (c) 1997-2017 The PHP Group
 Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
     with Zend OPcache v7.0.21-1~ubuntu14.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
 PHP 5.6.31-4+ubuntu14.04.1+deb.sury.org+4 (cli) 
 Copyright (c) 1997-2016 The PHP Group
 Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies

Anyway i changed the locale to it_IT and now it works.

andreapinti commented Sep 20, 2017

@gitlost sure

PHP 7.1.9-1+ubuntu14.04.1+deb.sury.org+1 (cli) (built: Sep  2 2017 06:17:11) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.1.9-1+ubuntu14.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
 PHP 7.0.21-1+ubuntu14.04.1+deb.sury.org+1 (cli) (built: Jul  6 2017 09:38:10) ( NTS )
 Copyright (c) 1997-2017 The PHP Group
 Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
     with Zend OPcache v7.0.21-1~ubuntu14.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
 PHP 5.6.31-4+ubuntu14.04.1+deb.sury.org+4 (cli) 
 Copyright (c) 1997-2016 The PHP Group
 Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies

Anyway i changed the locale to it_IT and now it works.

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

If I use arg --version it works
wp --allow-root core download --version="4.8.1"

ZeroX66 commented Sep 20, 2017

If I use arg --version it works
wp --allow-root core download --version="4.8.1"

@BhargavBhandari90

This comment has been minimized.

Show comment
Hide comment
@BhargavBhandari90

BhargavBhandari90 Sep 20, 2017

seems compressed file is not proper.

BhargavBhandari90 commented Sep 20, 2017

seems compressed file is not proper.

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

@ZeroX66
If you want to change default locale, you can set default locale to ~/.wp-cli/config.yml like following.

core download:
  locale: de_DE
Member

miya0001 commented Sep 20, 2017

@ZeroX66
If you want to change default locale, you can set default locale to ~/.wp-cli/config.yml like following.

core download:
  locale: de_DE
@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

@miya0001
I try to add that file , but it seems to still get the us. Do I need to restart something?

ZeroX66 commented Sep 20, 2017

@miya0001
I try to add that file , but it seems to still get the us. Do I need to restart something?

@wojsmol

This comment has been minimized.

Show comment
Hide comment
@wojsmol

wojsmol Sep 20, 2017

Contributor

on enviramant with error plese check if zlib php extesion is enabled
reference composer/composer#984

Contributor

wojsmol commented Sep 20, 2017

on enviramant with error plese check if zlib php extesion is enabled
reference composer/composer#984

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

@ZeroX66
Oh! Sorry, the file name of the config is ~/.wp-cli/config.yml.
https://make.wordpress.org/cli/handbook/config/

(I updated above. 😄 )

Member

miya0001 commented Sep 20, 2017

@ZeroX66
Oh! Sorry, the file name of the config is ~/.wp-cli/config.yml.
https://make.wordpress.org/cli/handbook/config/

(I updated above. 😄 )

@JPry

This comment has been minimized.

Show comment
Hide comment
@JPry

JPry Sep 20, 2017

Contributor

@wojsmol

on enviramant with error plese check if zlib php extesion is enabled

That's a good thought, but it appears that zlib is enabled:

php -i | grep zlib
Registered PHP Streams => https, ftps, compress.zlib, php, file, glob, data, http, ftp, phar, ssh2.shell, ssh2.exec, ssh2.tunnel, ssh2.scp, ssh2.sftp, zip
Registered Stream Filters => zlib.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, convert.iconv.*, mcrypt.*, mdecrypt.*
zlib
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

@ZeroX66

I think the problem might be the arguments in this line

tar xz --strip-components=1 --directory=%s -f $tarball

I don't think that line is the problem. That line is only executed if the PharData class does not exist. The class does exist. The exception happens on $phar = new PharData( $tarball ); as was mentioned by @miya0001

So it seems that there's some issue between the package and the PharData class. I'm not sure how to determine which of them is to blame.

Contributor

JPry commented Sep 20, 2017

@wojsmol

on enviramant with error plese check if zlib php extesion is enabled

That's a good thought, but it appears that zlib is enabled:

php -i | grep zlib
Registered PHP Streams => https, ftps, compress.zlib, php, file, glob, data, http, ftp, phar, ssh2.shell, ssh2.exec, ssh2.tunnel, ssh2.scp, ssh2.sftp, zip
Registered Stream Filters => zlib.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, convert.iconv.*, mcrypt.*, mdecrypt.*
zlib
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

@ZeroX66

I think the problem might be the arguments in this line

tar xz --strip-components=1 --directory=%s -f $tarball

I don't think that line is the problem. That line is only executed if the PharData class does not exist. The class does exist. The exception happens on $phar = new PharData( $tarball ); as was mentioned by @miya0001

So it seems that there's some issue between the package and the PharData class. I'm not sure how to determine which of them is to blame.

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

@miya0001
Great, now it works ...

ZeroX66 commented Sep 20, 2017

@miya0001
Great, now it works ...

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

Hmm,

Following is on Ubuntu and it is not working wp core download.

$ php -i | grep zlib
Registered PHP Streams => https, ftps, compress.zlib, php, file, glob, data, http, ftp, phar, zip
Registered Stream Filters => zlib.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, convert.iconv.*, mcrypt.*, mdecrypt.*
zlib
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

Following is on MacOS and it is working wp core download.

$ php -i | grep zlib
Configure Command =>  './configure'  '--prefix=/usr/local/Cellar/php71/7.1.8_20' '--localstatedir=/usr/local/var' '--sysconfdir=/usr/local/etc/php/7.1' '--with-config-file-path=/usr/local/etc/php/7.1' '--with-config-file-scan-dir=/usr/local/etc/php/7.1/conf.d' '--mandir=/usr/local/Cellar/php71/7.1.8_20/share/man' '--enable-bcmath' '--enable-calendar' '--enable-dba' '--enable-exif' '--enable-ftp' '--enable-gd-native-ttf' '--enable-mbregex' '--enable-mbstring' '--enable-shmop' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--enable-wddx' '--enable-zip' '--with-freetype-dir=/usr/local/opt/freetype' '--with-gd' '--with-gettext=/usr/local/opt/gettext' '--with-iconv-dir=/usr' '--with-icu-dir=/usr/local/opt/icu4c' '--with-jpeg-dir=/usr/local/opt/jpeg' '--with-kerberos=/usr' '--with-mhash' '--with-ndbm=/usr' '--with-png-dir=/usr/local/opt/libpng' '--with-xmlrpc' '--with-zlib=/usr' '--with-readline=/usr/local/opt/readline' '--without-gmp' '--without-snmp' '--with-libxml-dir=/usr/local/opt/libxml2' '--with-pdo-odbc=unixODBC,/usr/local/opt/unixodbc' '--with-unixODBC=/usr/local/opt/unixodbc' '--with-bz2=/usr' '--with-openssl=/usr/local/opt/openssl' '--enable-fpm' '--with-fpm-user=_www' '--with-fpm-group=_www' '--with-curl' '--with-xsl=/usr' '--with-ldap' '--with-ldap-sasl=/usr' '--with-mysql-sock=/tmp/mysql.sock' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--disable-opcache' '--enable-pcntl' '--without-pear' '--enable-dtrace' '--disable-phpdbg' '--enable-zend-signals'
Registered PHP Streams => https, ftps, compress.zlib, compress.bzip2, php, file, glob, data, http, ftp, phar, zip
Registered Stream Filters => zlib.*, bzip2.*, convert.iconv.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk
zlib
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value
Member

miya0001 commented Sep 20, 2017

Hmm,

Following is on Ubuntu and it is not working wp core download.

$ php -i | grep zlib
Registered PHP Streams => https, ftps, compress.zlib, php, file, glob, data, http, ftp, phar, zip
Registered Stream Filters => zlib.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, convert.iconv.*, mcrypt.*, mdecrypt.*
zlib
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

Following is on MacOS and it is working wp core download.

$ php -i | grep zlib
Configure Command =>  './configure'  '--prefix=/usr/local/Cellar/php71/7.1.8_20' '--localstatedir=/usr/local/var' '--sysconfdir=/usr/local/etc/php/7.1' '--with-config-file-path=/usr/local/etc/php/7.1' '--with-config-file-scan-dir=/usr/local/etc/php/7.1/conf.d' '--mandir=/usr/local/Cellar/php71/7.1.8_20/share/man' '--enable-bcmath' '--enable-calendar' '--enable-dba' '--enable-exif' '--enable-ftp' '--enable-gd-native-ttf' '--enable-mbregex' '--enable-mbstring' '--enable-shmop' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--enable-wddx' '--enable-zip' '--with-freetype-dir=/usr/local/opt/freetype' '--with-gd' '--with-gettext=/usr/local/opt/gettext' '--with-iconv-dir=/usr' '--with-icu-dir=/usr/local/opt/icu4c' '--with-jpeg-dir=/usr/local/opt/jpeg' '--with-kerberos=/usr' '--with-mhash' '--with-ndbm=/usr' '--with-png-dir=/usr/local/opt/libpng' '--with-xmlrpc' '--with-zlib=/usr' '--with-readline=/usr/local/opt/readline' '--without-gmp' '--without-snmp' '--with-libxml-dir=/usr/local/opt/libxml2' '--with-pdo-odbc=unixODBC,/usr/local/opt/unixodbc' '--with-unixODBC=/usr/local/opt/unixodbc' '--with-bz2=/usr' '--with-openssl=/usr/local/opt/openssl' '--enable-fpm' '--with-fpm-user=_www' '--with-fpm-group=_www' '--with-curl' '--with-xsl=/usr' '--with-ldap' '--with-ldap-sasl=/usr' '--with-mysql-sock=/tmp/mysql.sock' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--disable-opcache' '--enable-pcntl' '--without-pear' '--enable-dtrace' '--disable-phpdbg' '--enable-zend-signals'
Registered PHP Streams => https, ftps, compress.zlib, compress.bzip2, php, file, glob, data, http, ftp, phar, zip
Registered Stream Filters => zlib.*, bzip2.*, convert.iconv.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk
zlib
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value
@JPry

This comment has been minimized.

Show comment
Hide comment
@JPry

JPry Sep 20, 2017

Contributor

I have an idea for a solution.

We could do a try {} catch () block within the \WP_CLI\Extractor::extract_tarball() method. We could try the PharData class first, but if it throws an exception we could fall back to using a command line exec to do the extraction. If that still fails, then throw an exception and let it bubble back up to the existing block.

Contributor

JPry commented Sep 20, 2017

I have an idea for a solution.

We could do a try {} catch () block within the \WP_CLI\Extractor::extract_tarball() method. We could try the PharData class first, but if it throws an exception we could fall back to using a command line exec to do the extraction. If that still fails, then throw an exception and let it bubble back up to the existing block.

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 20, 2017

Anyway, I think it still some troubling with the package so can you do try-catch is the best now otherwise we all need to change the locale.
The best solution is WP go back in US and then create a new package like the others :)

ZeroX66 commented Sep 20, 2017

Anyway, I think it still some troubling with the package so can you do try-catch is the best now otherwise we all need to change the locale.
The best solution is WP go back in US and then create a new package like the others :)

@danielbachhuber

This comment has been minimized.

Show comment
Hide comment
@danielbachhuber

danielbachhuber Sep 20, 2017

Member

My hunch is that the WordPress US build is corrupt in some way.

Member

danielbachhuber commented Sep 20, 2017

My hunch is that the WordPress US build is corrupt in some way.

@danielbachhuber

This comment has been minimized.

Show comment
Hide comment
@danielbachhuber

danielbachhuber Sep 20, 2017

Member

Extraction also fails using tar on Travis:

image

Member

danielbachhuber commented Sep 20, 2017

Extraction also fails using tar on Travis:

image

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 20, 2017

Member

Should we open new ticket on trac?

Member

miya0001 commented Sep 20, 2017

Should we open new ticket on trac?

@danielbachhuber

This comment has been minimized.

Show comment
Hide comment
@danielbachhuber

danielbachhuber Sep 20, 2017

Member

Meta Trac is the most appropriate place. They'll ask what's corrupt with the file though, to which we should have an answer.

Member

danielbachhuber commented Sep 20, 2017

Meta Trac is the most appropriate place. They'll ask what's corrupt with the file though, to which we should have an answer.

@danielbachhuber

This comment has been minimized.

Show comment
Hide comment
@danielbachhuber

danielbachhuber Sep 20, 2017

Member

Going with a workaround for the time being: #4371

If someone wants to try, here's a Phar build: https://www.dropbox.com/s/taeh1mak9fvmlu6/wp-cli-nightly.phar?dl=0

Member

danielbachhuber commented Sep 20, 2017

Going with a workaround for the time being: #4371

If someone wants to try, here's a Phar build: https://www.dropbox.com/s/taeh1mak9fvmlu6/wp-cli-nightly.phar?dl=0

@danielbachhuber

This comment has been minimized.

Show comment
Hide comment
@danielbachhuber

danielbachhuber Sep 21, 2017

Member

I've also created a meta issue for this: https://meta.trac.wordpress.org/ticket/3150

Member

danielbachhuber commented Sep 21, 2017

I've also created a meta issue for this: https://meta.trac.wordpress.org/ticket/3150

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 21, 2017

Member

I merged #4371 . I would close when https://meta.trac.wordpress.org/ticket/3150 will be closed.
Thanks. :)

Member

miya0001 commented Sep 21, 2017

I merged #4371 . I would close when https://meta.trac.wordpress.org/ticket/3150 will be closed.
Thanks. :)

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 21, 2017

Member

!? I tried php -r "new PharData( 'latest.tar.gz' );" with each version of PHP on Travis CI.
Then, ... It is passed! 😮
https://travis-ci.org/miya0001/test-phardata

Member

miya0001 commented Sep 21, 2017

!? I tried php -r "new PharData( 'latest.tar.gz' );" with each version of PHP on Travis CI.
Then, ... It is passed! 😮
https://travis-ci.org/miya0001/test-phardata

@Codestar

This comment has been minimized.

Show comment
Hide comment
@Codestar

Codestar Sep 21, 2017

Thanks for effort @miya0001

Codestar commented Sep 21, 2017

Thanks for effort @miya0001

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 21, 2017

Hm, if I use (en_US)

wp --allow-root core download

I still have som trouble here ...
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c3b83ee90f5.tar.gz" to temporary file

ZeroX66 commented Sep 21, 2017

Hm, if I use (en_US)

wp --allow-root core download

I still have som trouble here ...
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c3b83ee90f5.tar.gz" to temporary file

@miya0001

This comment has been minimized.

Show comment
Hide comment
Member

miya0001 commented Sep 21, 2017

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 21, 2017

Member

I tried as follows on Amazon Linux too.

$ wget https://wordpress.org/latest.tar.gz
$ php -r "new PharData( 'latest.tar.gz' );"

Then it worked as expected.

Member

miya0001 commented Sep 21, 2017

I tried as follows on Amazon Linux too.

$ wget https://wordpress.org/latest.tar.gz
$ php -r "new PharData( 'latest.tar.gz' );"

Then it worked as expected.

@vintagesucks

This comment has been minimized.

Show comment
Hide comment
@vintagesucks

vintagesucks Sep 21, 2017

Thanks for the fix. 👍

Is there any ETA for a new stable release or should I use the nightly version in my Travis builds for now? @danielbachhuber


Stable

Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c3b89bba349.tar.gz" to temporary file

Nightly

Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Warning: PharData failed, falling back to 'tar gz' (unable to decompress gzipped phar archive "/tmp/wp_59c3ba8e8d989.tar.gz" to temporary file)
Success: WordPress downloaded.

vintagesucks commented Sep 21, 2017

Thanks for the fix. 👍

Is there any ETA for a new stable release or should I use the nightly version in my Travis builds for now? @danielbachhuber


Stable

Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c3b89bba349.tar.gz" to temporary file

Nightly

Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Warning: PharData failed, falling back to 'tar gz' (unable to decompress gzipped phar archive "/tmp/wp_59c3ba8e8d989.tar.gz" to temporary file)
Success: WordPress downloaded.
@gitlost

This comment has been minimized.

Show comment
Hide comment
@gitlost

gitlost Sep 21, 2017

Contributor

I tried php -r "new PharData( 'latest.tar.gz' );" with each version of PHP on Travis CI.

@miya0001 very interesting. I've (sort of) tracked it down locally to this build patch "0022-lp564920-fix-big-files.patch":

From: Debian PHP Maintainers <pkg-php-maint@lists.alioth.debian.org>
Date: Sat, 2 May 2015 10:26:54 +0200
Subject: lp564920-fix-big-files

---
 main/streams/plain_wrapper.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/main/streams/plain_wrapper.c b/main/streams/plain_wrapper.c
index 9dbc4246b..c65571f94 100644
--- a/main/streams/plain_wrapper.c
+++ b/main/streams/plain_wrapper.c
@@ -681,7 +681,13 @@ static int php_stdiop_set_option(php_stream *stream, int option, int value, void
 
 				switch (value) {
 					case PHP_STREAM_MMAP_SUPPORTED:
-						return fd == -1 ? PHP_STREAM_OPTION_RETURN_ERR : PHP_STREAM_OPTION_RETURN_OK;
+						if (fd == -1)
+							return PHP_STREAM_OPTION_RETURN_ERR;
+						/* Don't mmap large files */
+						do_fstat(data, 1);
+						if (data->sb.st_size > 4 * 1024 * 1024)
+							return PHP_STREAM_OPTION_RETURN_ERR;
+						return PHP_STREAM_OPTION_RETURN_OK;
 
 					case PHP_STREAM_MMAP_MAP_RANGE:
 						if(do_fstat(data, 1) != 0) {

which causes memory mapping of the input file to be disabled (as it's > 4MB) so PHP uses a 8KB chunk read-write loop instead, which fails at the last 7 byte chunk for some reason (8192 * 999 + 7 = 8183815, the size of wordpress-4.8.2-en_US.tar.gz). There's a zlib.inflate filter on the output stream which may be the culprit.

But it's a bit surprising that Travis wouldn't use this patch also.

Contributor

gitlost commented Sep 21, 2017

I tried php -r "new PharData( 'latest.tar.gz' );" with each version of PHP on Travis CI.

@miya0001 very interesting. I've (sort of) tracked it down locally to this build patch "0022-lp564920-fix-big-files.patch":

From: Debian PHP Maintainers <pkg-php-maint@lists.alioth.debian.org>
Date: Sat, 2 May 2015 10:26:54 +0200
Subject: lp564920-fix-big-files

---
 main/streams/plain_wrapper.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/main/streams/plain_wrapper.c b/main/streams/plain_wrapper.c
index 9dbc4246b..c65571f94 100644
--- a/main/streams/plain_wrapper.c
+++ b/main/streams/plain_wrapper.c
@@ -681,7 +681,13 @@ static int php_stdiop_set_option(php_stream *stream, int option, int value, void
 
 				switch (value) {
 					case PHP_STREAM_MMAP_SUPPORTED:
-						return fd == -1 ? PHP_STREAM_OPTION_RETURN_ERR : PHP_STREAM_OPTION_RETURN_OK;
+						if (fd == -1)
+							return PHP_STREAM_OPTION_RETURN_ERR;
+						/* Don't mmap large files */
+						do_fstat(data, 1);
+						if (data->sb.st_size > 4 * 1024 * 1024)
+							return PHP_STREAM_OPTION_RETURN_ERR;
+						return PHP_STREAM_OPTION_RETURN_OK;
 
 					case PHP_STREAM_MMAP_MAP_RANGE:
 						if(do_fstat(data, 1) != 0) {

which causes memory mapping of the input file to be disabled (as it's > 4MB) so PHP uses a 8KB chunk read-write loop instead, which fails at the last 7 byte chunk for some reason (8192 * 999 + 7 = 8183815, the size of wordpress-4.8.2-en_US.tar.gz). There's a zlib.inflate filter on the output stream which may be the culprit.

But it's a bit surprising that Travis wouldn't use this patch also.

@danielbachhuber

This comment has been minimized.

Show comment
Hide comment
@danielbachhuber

danielbachhuber Sep 21, 2017

Member

Is there any ETA for a new stable release or should I use the nightly version in my Travis builds for now

Not yet scheduled. You should use the nightly build.

Member

danielbachhuber commented Sep 21, 2017

Is there any ETA for a new stable release or should I use the nightly version in my Travis builds for now

Not yet scheduled. You should use the nightly build.

@miya0001

This comment has been minimized.

Show comment
Hide comment
@miya0001

miya0001 Sep 21, 2017

Member

@gitlost

It worked fine on my custom docker environment which is using phpenv + php-build too.
I saw the php-build repository and I can't find the patch what you said.
https://github.com/php-build/php-build/tree/master/share/php-build/patches

I guess Travis CI is using same solution. :)

Member

miya0001 commented Sep 21, 2017

@gitlost

It worked fine on my custom docker environment which is using phpenv + php-build too.
I saw the php-build repository and I can't find the patch what you said.
https://github.com/php-build/php-build/tree/master/share/php-build/patches

I guess Travis CI is using same solution. :)

vintagesucks added a commit to vintagesucks/caddy-script that referenced this issue Sep 21, 2017

use wp-cli nightly
fixes wordpress build

see wp-cli/wp-cli#4370 (comment)
@gitlost

This comment has been minimized.

Show comment
Hide comment
@gitlost

gitlost Sep 21, 2017

Contributor

Ah maybe. How Travis builds its stuff has always been (and likely always will be) a mystery to me.

The patch above is from the Ubuntu php7.0 package, eg for 17.04 https://launchpad.net/ubuntu/+archive/primary/+files/php7.0_7.0.22-0ubuntu0.17.04.1.debian.tar.xz, for 16.04 https://launchpad.net/ubuntu/+archive/primary/+files/php7.0_7.0.22-0ubuntu0.16.04.1.debian.tar.xz, but is present in older distributions also.

Contributor

gitlost commented Sep 21, 2017

Ah maybe. How Travis builds its stuff has always been (and likely always will be) a mystery to me.

The patch above is from the Ubuntu php7.0 package, eg for 17.04 https://launchpad.net/ubuntu/+archive/primary/+files/php7.0_7.0.22-0ubuntu0.17.04.1.debian.tar.xz, for 16.04 https://launchpad.net/ubuntu/+archive/primary/+files/php7.0_7.0.22-0ubuntu0.16.04.1.debian.tar.xz, but is present in older distributions also.

@jmazzi

This comment has been minimized.

Show comment
Hide comment
@jmazzi

jmazzi Sep 21, 2017

The problem seems to be specific to gzip, not tar.

ubuntu@controller-us-east-1:/tmp/f$ php -r '$f=new PharData( "wp_59c3ce5c83746.tar.gz" );$f->extractTo("/tmp/wordpress");'
PHP Fatal error:  Uncaught exception 'UnexpectedValueException' with message 'unable to decompress gzipped phar archive "/tmp/f/wp_59c3ce5c83746.tar.gz" to temporary file' in Command line code:1
Stack trace:
#0 Command line code(1): PharData->__construct('wp_59c3ce5c8374...')
#1 {main}
  thrown in Command line code on line 1
ubuntu@controller-us-east-1:/tmp/f$ gunzip wp_59c3ce5c83746.tar.gz 
ubuntu@controller-us-east-1:/tmp/f$ php -r '$f=new PharData( "wp_59c3ce5c83746.tar" );$f->extractTo("/tmp/wordpress");'
ubuntu@controller-us-east-1:/tmp/f$ ls -al /tmp/wordpress/
total 80
drwxrwxr-x    3 ubuntu ubuntu  4096 Sep 21 14:55 .
drwxrwxrwt 2593 root   root   69632 Sep 21 14:55 ..
drwxrwxr-x    5 ubuntu ubuntu  4096 Sep 21 14:55 wordpress
ubuntu@controller-us-east-1:/tmp/f$ 

jmazzi commented Sep 21, 2017

The problem seems to be specific to gzip, not tar.

ubuntu@controller-us-east-1:/tmp/f$ php -r '$f=new PharData( "wp_59c3ce5c83746.tar.gz" );$f->extractTo("/tmp/wordpress");'
PHP Fatal error:  Uncaught exception 'UnexpectedValueException' with message 'unable to decompress gzipped phar archive "/tmp/f/wp_59c3ce5c83746.tar.gz" to temporary file' in Command line code:1
Stack trace:
#0 Command line code(1): PharData->__construct('wp_59c3ce5c8374...')
#1 {main}
  thrown in Command line code on line 1
ubuntu@controller-us-east-1:/tmp/f$ gunzip wp_59c3ce5c83746.tar.gz 
ubuntu@controller-us-east-1:/tmp/f$ php -r '$f=new PharData( "wp_59c3ce5c83746.tar" );$f->extractTo("/tmp/wordpress");'
ubuntu@controller-us-east-1:/tmp/f$ ls -al /tmp/wordpress/
total 80
drwxrwxr-x    3 ubuntu ubuntu  4096 Sep 21 14:55 .
drwxrwxrwt 2593 root   root   69632 Sep 21 14:55 ..
drwxrwxr-x    5 ubuntu ubuntu  4096 Sep 21 14:55 wordpress
ubuntu@controller-us-east-1:/tmp/f$ 

vintagesucks added a commit to vintagesucks/caddy-script that referenced this issue Sep 22, 2017

@gitlost

This comment has been minimized.

Show comment
Hide comment
@gitlost

gitlost Sep 22, 2017

Contributor

I tracked down the guilty party to here https://github.com/php/php-src/blob/master/ext/zlib/zlib_filter.c#L124 in the zlib_filter where it returns without updating the bytes_consumed variable, as if a .tar.gz is 1 to about 10 bytes over modulo 8192 then no output might be needed (the last 4 bytes are just a checksum anyway) but the bytes have been consumed, and not reporting them as such causes the calling function to error.

As mentioned above this is only triggered for files > 4MB on distributions with the Debian patch but even so it's surprising it hasn't been noticed before.

Contributor

gitlost commented Sep 22, 2017

I tracked down the guilty party to here https://github.com/php/php-src/blob/master/ext/zlib/zlib_filter.c#L124 in the zlib_filter where it returns without updating the bytes_consumed variable, as if a .tar.gz is 1 to about 10 bytes over modulo 8192 then no output might be needed (the last 4 bytes are just a checksum anyway) but the bytes have been consumed, and not reporting them as such causes the calling function to error.

As mentioned above this is only triggered for files > 4MB on distributions with the Debian patch but even so it's surprising it hasn't been noticed before.

@chriscroome

This comment has been minimized.

Show comment
Hide comment
@chriscroome

chriscroome Sep 25, 2017

Just to say that I have also hit this issue on Debian Stretch:

wp --allow-root core download
Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c96e95f100a.tar.gz" to temporary file

wp --allow-root core download --locale=en_GB
Downloading WordPress 4.8.2 (en_GB)...
md5 hash verified: 28571832a9c0c95cbc8767a896c987d4
Success: WordPress downloaded.

chriscroome commented Sep 25, 2017

Just to say that I have also hit this issue on Debian Stretch:

wp --allow-root core download
Downloading WordPress 4.8.2 (en_US)...
md5 hash verified: 2e8744a702a3d9527782d9135a4c9544
Error: Couldn't extract WordPress archive. unable to decompress gzipped phar archive "/tmp/wp_59c96e95f100a.tar.gz" to temporary file

wp --allow-root core download --locale=en_GB
Downloading WordPress 4.8.2 (en_GB)...
md5 hash verified: 28571832a9c0c95cbc8767a896c987d4
Success: WordPress downloaded.
@gitlost

This comment has been minimized.

Show comment
Hide comment
@gitlost

gitlost Sep 25, 2017

Contributor

@chriscroome Could you post the output of php --version please?

Contributor

gitlost commented Sep 25, 2017

@chriscroome Could you post the output of php --version please?

@ZeroX66

This comment has been minimized.

Show comment
Hide comment
@ZeroX66

ZeroX66 Sep 26, 2017

Hm, It's still some trouble with it. I change to this and it works with "nightly".

wp cli update --nightly

ZeroX66 commented Sep 26, 2017

Hm, It's still some trouble with it. I change to this and it works with "nightly".

wp cli update --nightly

@JPry

This comment has been minimized.

Show comment
Hide comment
@JPry

JPry Sep 26, 2017

Contributor

@ZeroX66

Hm, It's still some trouble with it. I change to this and it works with "nightly".

Nightly is the only way to have the fix. There hasn't been a release yet.

Contributor

JPry commented Sep 26, 2017

@ZeroX66

Hm, It's still some trouble with it. I change to this and it works with "nightly".

Nightly is the only way to have the fix. There hasn't been a release yet.

@matthewsuan

This comment has been minimized.

Show comment
Hide comment
@matthewsuan

matthewsuan Sep 26, 2017

i had this problem as well. the --version=4.8.1 fix worked for me.

matthewsuan commented Sep 26, 2017

i had this problem as well. the --version=4.8.1 fix worked for me.

@danielbachhuber

This comment has been minimized.

Show comment
Hide comment
@danielbachhuber

danielbachhuber Sep 26, 2017

Member

Thanks everyone who spent time tracking this down. Given this isn't an issue with WP-CLI itself but with specific PHP distros, I think we've fixed it the best we can.

We'll publish WP-CLI v1.4.0 on Tuesday, October 17th. For now, those affected can use wp cli update --nightly.

Member

danielbachhuber commented Sep 26, 2017

Thanks everyone who spent time tracking this down. Given this isn't an issue with WP-CLI itself but with specific PHP distros, I think we've fixed it the best we can.

We'll publish WP-CLI v1.4.0 on Tuesday, October 17th. For now, those affected can use wp cli update --nightly.

@chriscroome

This comment has been minimized.

Show comment
Hide comment
@chriscroome

chriscroome Sep 26, 2017

@gitlost sorry not to have replied sooner. the server is running PHP7.0 from Debian Stretch:

php --version
PHP 7.0.19-1 (cli) (built: May 11 2017 14:04:47) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.0.19-1, Copyright (c) 1999-2017, by Zend Technologies

chriscroome commented Sep 26, 2017

@gitlost sorry not to have replied sooner. the server is running PHP7.0 from Debian Stretch:

php --version
PHP 7.0.19-1 (cli) (built: May 11 2017 14:04:47) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.0.19-1, Copyright (c) 1999-2017, by Zend Technologies
@surrealchemist

This comment has been minimized.

Show comment
Hide comment
@surrealchemist

surrealchemist Sep 26, 2017

We saw this on our Ubuntu 16/ php70 server last week, and today a developer got it while trying to provision a VCCW vagrant environment.
Had to manually do 4.8.1 to get around the issue.

Also the message "unable to decompress" was misleading at first. It would be more useful if it said why exactly it is having a problem. At first we thought it was permissions and had to go through checking /tmp etc. and disk space till we realized the file was in /tmp and we could untar it fine by hand.

surrealchemist commented Sep 26, 2017

We saw this on our Ubuntu 16/ php70 server last week, and today a developer got it while trying to provision a VCCW vagrant environment.
Had to manually do 4.8.1 to get around the issue.

Also the message "unable to decompress" was misleading at first. It would be more useful if it said why exactly it is having a problem. At first we thought it was permissions and had to go through checking /tmp etc. and disk space till we realized the file was in /tmp and we could untar it fine by hand.

@Sidsector9

This comment has been minimized.

Show comment
Hide comment
@Sidsector9
Member

Sidsector9 commented Sep 27, 2017

For EasyEngine users, there's a blog with 2 workarounds -
https://easyengine.io/blog/solution-couldnt-extract-wordpress-archive-error/

Thanks to @danielbachhuber, @miya0001 and @gitlost

kadimi added a commit to kadimi/wp.box that referenced this issue Sep 27, 2017

kadimi added a commit to kadimi/wp.box that referenced this issue Sep 27, 2017

@pkarjala

This comment has been minimized.

Show comment
Hide comment
@pkarjala

pkarjala Sep 29, 2017

Contributor

Can confirm we are also seeing this issue on Ubuntu 16.04.3 with PHP 7.0.22.

Can also confirm that using wp cli update --nightly fixed the issue using the .tar.gz fallback.

Contributor

pkarjala commented Sep 29, 2017

Can confirm we are also seeing this issue on Ubuntu 16.04.3 with PHP 7.0.22.

Can also confirm that using wp cli update --nightly fixed the issue using the .tar.gz fallback.

@davidfavor

This comment has been minimized.

Show comment
Hide comment
@davidfavor

davidfavor Oct 3, 2017

Problem: wp-cli attempts unpack files using tar -xJf which works for xz files + not for gzip files.

Solution: Any .tar.gz file should use tar -xzf instead + likely all will be well.

In fact, likely best to write wp-cli to use a switch statement, so any known extensions will be correctly unpacked + if switch falls through, an exception is raised.

This will handle any future cases where WordPress might change their default archive format.

davidfavor commented Oct 3, 2017

Problem: wp-cli attempts unpack files using tar -xJf which works for xz files + not for gzip files.

Solution: Any .tar.gz file should use tar -xzf instead + likely all will be well.

In fact, likely best to write wp-cli to use a switch statement, so any known extensions will be correctly unpacked + if switch falls through, an exception is raised.

This will handle any future cases where WordPress might change their default archive format.

@danielbachhuber

This comment has been minimized.

Show comment
Hide comment
@danielbachhuber

danielbachhuber Oct 3, 2017

Member

Please open a new issue for any topics going forward, thanks.

Member

danielbachhuber commented Oct 3, 2017

Please open a new issue for any topics going forward, thanks.

@wp-cli wp-cli locked and limited conversation to collaborators Oct 3, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.