drush dl drupal fail to work (No such file or directory filesystem.inc:257) #234

Closed
sobi3ch opened this Issue Nov 2, 2013 · 24 comments

Projects

None yet
@sobi3ch
sobi3ch commented Nov 2, 2013

I''m on Ubuntu 13.10 with php 5.5.3 and latest drush (6.1). I'm trying to download fresh drupal but with no success. I've try already with vagrant and ubuntu 13.04 and 12.04 end even with centOS but with same results. Below you can find debug output (-v) I'm getting each time whatever I've try. I also try on each system to downgrade drush to 6.0.0 then to 5.0.0.

sobiech@sobiech-Dell:/var/www/sandbox/drupal $ drush -v dl drupal
Loading version_control engine.                                                                                                          [notice]
Loading package_handler engine.                                                                                                          [notice]
Executing: wget --version
Loading release_info engine.                                                                                                             [notice]
Downloading release history from http://updates.drupal.org/release-history/drupal/7.x                                                    [notice]
Executing: wget --version
Executing: wget -q --timeout=30 -O /tmp/download_filemGFMV4 http://updates.drupal.org/release-history/drupal/7.x
Downloading project drupal to /tmp/drush_tmp_1383414657_52753b81b2c59 ...                                                                [notice]
Executing: wget -q --timeout=30 -O /tmp/download_filecja327 http://ftp.drupal.org/files/projects/drupal-7.23.tar.gz
Downloading drupal-7.23.tar.gz was successful.                                                                                           [notice]
Md5 checksum of drupal-7.23.tar.gz verified.                                                                                             [notice]
Mime type for /tmp/drush_tmp_1383414657_52753b81b2c59/drupal-7.23.tar.gz is application/x-gzip                                           [notice]
Executing: tar -C /tmp/drush_tmp_1383414657_52753b81b2c59 -xzf drupal-7.23.tar.gz
Executing: tar -tzf drupal-7.23.tar.gz
Array to string conversion download.pm.inc:323                                                                                           [notice]
Executing: svn info /var/www/sandbox/drupal
Executing: bzr root /var/www/sandbox/drupal
copy(/tmp/drush_tmp_1383414657_52753b81b2c59/drupal): failed to open stream: No such file or directory filesystem.inc:257             [warning]
Project drupal (7.23) could not be downloaded to /var/www/sandbox/drupal/drupal.                                                      [error]
Command dispatch complete                                                                                                                [notice]

sobiech@sobiech-Dell:/var/www/sandbox/drupal $ php -v
PHP 5.5.3-1ubuntu2 (cli) (built: Oct  9 2013 14:49:12) 
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2013 Zend Technologies
    with Zend OPcache v7.0.3-dev, Copyright (c) 1999-2013, by Zend Technologies
    with Xdebug v2.2.3, Copyright (c) 2002-2013, by Derick Rethans

sobiech@sobiech-Dell:/var/www/sandbox/drupal $ drush version
 Drush Version   :  6.1.0 

@sobi3ch
sobi3ch commented Nov 2, 2013

Same problem when downloading D6

@jacksmith9

Confirmed. This is a major issue for as we can't use our deployment system to add sites.

@chrfritsch

Same problem here. To fix it in makefiles we use now the git download strategy

projects[drupal][type] = "core"
projects[drupal][version] = "7.23"
projects[drupal][download][type] = git
projects[drupal][download][tag] = "7.23"
projects[drupal][download][url] = http://git.drupal.org/project/drupal.git

We have an other problem with downloading themes. They are now located in the modules/contrib folder

@jonhattan
Member

There's a change in the update xml: http://updates.drupal.org/release-history/drupal/7.x

It is missing terms/term/name = Projects. This term indicates the project type (Drupal core, module, theme....). Drush assumes a module by default. Probably if you try to download a theme or a profile it will be placed in sites/all/modules.

I guess it is related to the drupal.org upgrade to Drupal 7.

@jonhattan
Member
@NoDiskInDriveA

The above fix for the core will also work for themes and, in fact, for every project whose type is not 'module'. It works around the issue by completely circumventing the release history.
(It will make 'make_project_needs_release_info' return false, and in consequence, not override the project type later in 'updatexml_determine_project_type')

@sobi3ch
sobi3ch commented Nov 2, 2013

@NoDiskInDriveA yeap I just realize when I download drupal manually then try to download bootstrap theme via drush and suddenly it end up in sitets/all/module dir(??)

@danylevskyi

Also there is problem with Drush make. #236.

@amcgowanca amcgowanca referenced this issue in imagex/imagex_installkit Nov 3, 2013
Merged

Hotfix: Downloads Drupal core using Git #55

@jrbeeman
jrbeeman commented Nov 3, 2013

I found the following works and also avoids a clone of the entire Drupal git repository:

projects[drupal][type] = core
projects[drupal][version] = 7.23
projects[drupal][download][type] = get
projects[drupal][download][url] = http://ftp.drupal.org/files/projects/drupal-7.23.tar.gz
@RikdeBoer

Just to say that we've just tried jrbeeman's workaround in our stub.make and found it works a treat!

@greg-1-anderson
Member

If you are not using make, then the following workaround also works for Drush dl:

drush dl drupal --package-handler=git_drupalorg

This does, of course, download the entire repository, though.

@jonhattan
Member

I'm proposing a fix in this pull request: #237

@nickwshaw

jrbeeman method works but contrib themes are still downloading to modules/contrib

@ncdc ncdc added a commit to ncdc/drupal-quickstart that referenced this issue Nov 4, 2013
@ncdc ncdc Workaround for Drupal installation issue
This is a temporary workaround.

See the following for more info:

drush-ops/drush#234 (comment)

https://drupal.org/node/2126123
1e339b9
@ncdc ncdc added a commit to ncdc/drupal-quickstart that referenced this issue Nov 4, 2013
@ncdc ncdc Workaround for Drupal installation issue
This is a temporary workaround.

See the following for more info:

drush-ops/drush#234 (comment)

https://drupal.org/node/2126123

Bug 1018387
c5c5ab9
@ncdc ncdc referenced this issue in openshift/drupal-quickstart Nov 4, 2013
Merged

Workaround for Drupal installation issue #17

@jrbeeman
jrbeeman commented Nov 4, 2013

@nickwshaw, you can explicitly declare the type of project for the theme, and it'll put it in the right location:

projects[adaptivetheme][type] = theme
projects[adaptivetheme][subdir] = contrib
projects[adaptivetheme][version] = 3.1
This was referenced Nov 4, 2013
@openstack-gerrit openstack-gerrit pushed a commit to openstack-infra/groups that referenced this issue Nov 4, 2013
@mkissam mkissam Workaround for drupal.org update realted drush make bug
Drush make fails to download Drupal core 7.23 after site-wide
upgrade of drupal.org. The related upstream bug could be found
here: drush-ops/drush#234

Change-Id: Ieb0a818518ab7ef3f68a5c00b462836a5074a395
Related-Bug: #1247925
6e46674
@mikecrittenden

FYI, drumm pushed a fix for this to d.o https://drupal.org/comment/8144251#comment-8144251

drush dl drupal is still failing for me but this is presumably because the package needs to be rebuilt now which takes a few hours.

@greg-1-anderson
Member

drush dl drush does in fact appear to be working again this morning. Do we still need anything in #244?

@jstarcher

I can confirm that drush dl drupal is working now, however, Drupal still fails to properly download when using make files.

A simple make file:

api = "2"
core = "7.x"
projects[drupal][type] = "core"

Fails:

drush -vd make drupal-core.make foo
Bootstrap to phase 0. [0.01 sec, 4.57 MB]                            [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drush() [0.01 sec, 4.83 MB] [bootstrap]
Cache HIT cid: 5.7-commandfiles-0-7eff0498028cb7879bd9813be6b0ae6e [0.01 sec, 4.85 MB]                                                                                                                                               [debug]
Bootstrap to phase 0. [0.05 sec, 10.04 MB]                                                                                                                                                                                       [bootstrap]
Bootstrap to phase 0. [0.06 sec, 10.04 MB]                                                                                                                                                                                       [bootstrap]
Found command: make (commandfile=make) [0.06 sec, 10.05 MB]                                                                                                                                                                      [bootstrap]
Loading release_info engine. [0.09 sec, 10.06 MB]                                                                                                                                                                                   [notice]
Downloading release history from http://updates.drupal.org/release-history/drupal/7.x [0.09 sec, 10.32 MB]                                                                                                                          [notice]
http---updates.drupal.org-release-history-drupal-7.x retrieved from cache. [0.09 sec, 10.32 MB]                                                                                                                                     [notice]
Backend invoke: /usr/local/bin/php /usr/local/Cellar/drush/5.7/libexec/drush.php --php=/usr/local/bin/php  --backend=2  make-process /tmp/make_tmp_1383664306_52790ab2bed4a 2>&1 [0.14 sec, 10.37 MB]                              [command]
/usr/local/bin/php /usr/local/Cellar/drush/5.7/libexec/drush.php --php=/usr/local/bin/php  --backend=2  make-process /tmp/make_tmp_1383664306_52790ab2bed4a 2>&1 [0.14 sec, 10.37 MB]                                               [notice]
Bootstrap to phase 0. [0.25 sec, 10.04 MB]                                                                                                                                                                                       [bootstrap]
Bootstrap to phase 0. [0.27 sec, 10.05 MB]                                                                                                                                                                                       [bootstrap]
Found command: make-process (commandfile=make) [0.27 sec, 10.05 MB]                                                                                                                                                              [bootstrap]
Loading release_info engine. [0.28 sec, 10.06 MB]                                                                                                                                                                                   [notice]
Backend invoke: /usr/local/bin/php /usr/local/Cellar/drush/5.7/libexec/drush.php --php=/usr/local/bin/php  --backend=2 --yes --root= --uri=  pm-download drupal-7.23                                                               [command]
--destination=/tmp/make_tmp_1383664306_52790ab2bed4a/__build__/sites/all/modules --package-handler=wget --source=http://updates.drupal.org/release-history --variant=profile-only --cache --drupal-project-rename=drupal 2>&1
[0.35 sec, 10.33 MB]
/usr/local/bin/php /usr/local/Cellar/drush/5.7/libexec/drush.php --php=/usr/local/bin/php  --backend=2 --yes --root= --uri=  pm-download drupal-7.23                                                                                [notice]
--destination=/tmp/make_tmp_1383664306_52790ab2bed4a/__build__/sites/all/modules --package-handler=wget --source=http://updates.drupal.org/release-history --variant=profile-only --cache --drupal-project-rename=drupal 2>&1
[0.35 sec, 10.33 MB]
Bootstrap to phase 0. [0.46 sec, 10.04 MB]                                                                                                                                                                                       [bootstrap]
Found command: pm-download (commandfile=pm) [0.47 sec, 10.05 MB]                                                                                                                                                                 [bootstrap]
Loading version_control engine. [0.49 sec, 10.08 MB]                                                                                                                                                                                [notice]
Loading package_handler engine. [0.49 sec, 10.12 MB]                                                                                                                                                                                [notice]
Loading release_info engine. [0.5 sec, 10.16 MB]                                                                                                                                                                                    [notice]
Including /usr/local/Cellar/drush/5.7/libexec/commands/pm/download.pm.inc [0.5 sec, 10.36 MB]                                                                                                                                    [bootstrap]
Downloading release history from http://updates.drupal.org/release-history/drupal/7.x [0.5 sec, 10.56 MB]                                                                                                                           [notice]
http---updates.drupal.org-release-history-drupal-7.x retrieved from cache. [0.5 sec, 10.56 MB]                                                                                                                                      [notice]
Downloading project drupal to /tmp/drush_tmp_1383664307_52790ab32e2e2 ... [0.51 sec, 10.57 MB]                                                                                                                                      [notice]
http---ftp.drupal.org-files-projects-drupal-7.23.tar.gz retrieved from cache. [0.51 sec, 10.57 MB]                                                                                                                                  [notice]
Downloading drupal-7.23.tar.gz was successful. [0.51 sec, 10.57 MB]                                                                                                                                                                 [notice]
Md5 checksum of drupal-7.23.tar.gz verified. [0.52 sec, 10.57 MB]                                                                                                                                                                   [notice]
Fileinfo extension available. [0.52 sec, 10.57 MB]                                                                                                                                                                                   [debug]
Mime type for /tmp/drush_tmp_1383664307_52790ab32e2e2/drupal-7.23.tar.gz is application/x-gzip [0.52 sec, 10.58 MB]                                                                                                                 [notice]
Verifying signature for svn version control engine. [0.75 sec, 10.83 MB]                                                                                                                                                             [debug]
Verifying signature for bzr version control engine. [0.76 sec, 10.59 MB]                                                                                                                                                             [debug]
copy(/tmp/drush_tmp_1383664307_52790ab32e2e2/drupal): failed to open stream: No such file or directory filesystem.inc:229 [0.77 sec, 10.58 MB]                                                                                   [warning]
Project drupal (7.23) could not be downloaded to /private/tmp/make_tmp_1383664306_52790ab2bed4a/__build__/sites/all/modules/drupal. [0.77 sec, 10.42 MB]                                                                         [error]
Command dispatch complete [0.77 sec, 10.53 MB]                                                                                                                                                                                      [notice]
Peak memory usage was 11.27 MB [0.77 sec, 10.53 MB]                                                                                                                                                                                 [memory]
Command dispatch complete [0.87 sec, 10.3 MB]                                                                                                                                                                                       [notice]
Peak memory usage was 11.59 MB [0.87 sec, 10.3 MB]                                                                                                                                                                                  [memory]
Command dispatch complete [0.89 sec, 10.34 MB]                                                                                                                                                                                      [notice]
Peak memory usage was 11.63 MB [0.89 sec, 10.34 MB]

This is after a drush cc drush as well as clearing files in /tmp.

Update: Very strange, I'm able to build with drush make again on my FreeBSD machine but I get the above error on OSX. There must be a cache somewhere that is not getting cleared with drush cc drush. Otherwise maybe this is only broken on OSX now? Can someone confirm?

Update 2: Okay, so this appears to be another bug which is that drush cc drush with drush 5.7 on OSX does not actually clear all of the drush cache. I manually removed the ~/.drush/cache directory and everything is working now.

@jonhattan
Member

I think we're done. Lets wait the tests https://travis-ci.org/drush-ops/drush/builds

@danylevskyi

drush make works great now!

@barraponto
Contributor

Anyone having issues might need to clean up ~/.drush/cache.

@sobi3ch
sobi3ch commented Nov 5, 2013

Sweet! Working. I assuming is done now, last 3 builds were fine https://travis-ci.org/drush-ops/drush/builds so closing this now. Thanks guys for good work.

@sobi3ch sobi3ch closed this Nov 5, 2013
@jonhattan
Member

update xml is still rebuilding. Some projects won't work yet.

@jonhattan jonhattan reopened this Nov 5, 2013
@jonhattan
Member

Everything in place now. Start making!

@jonhattan jonhattan closed this Nov 6, 2013
@damz
Contributor
damz commented Nov 6, 2013

Now that the fire is gone, let's see how to implement a long term fix, see #249.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment