drush dl drupal fail to work (No such file or directory filesystem.inc:257)

[sobi3ch]

I''m on Ubuntu 13.10 with php 5.5.3 and latest drush (6.1). I'm trying to download fresh drupal but with no success. I've try already with vagrant and ubuntu 13.04 and 12.04 end even with centOS but with same results. Below you can find debug output (-v) I'm getting each time whatever I've try. I also try on each system to downgrade drush to 6.0.0 then to 5.0.0.

sobiech@sobiech-Dell:/var/www/sandbox/drupal $ drush -v dl drupal
Loading version_control engine.                                                                                                          [notice]
Loading package_handler engine.                                                                                                          [notice]
Executing: wget --version
Loading release_info engine.                                                                                                             [notice]
Downloading release history from http://updates.drupal.org/release-history/drupal/7.x                                                    [notice]
Executing: wget --version
Executing: wget -q --timeout=30 -O /tmp/download_filemGFMV4 http://updates.drupal.org/release-history/drupal/7.x
Downloading project drupal to /tmp/drush_tmp_1383414657_52753b81b2c59 ...                                                                [notice]
Executing: wget -q --timeout=30 -O /tmp/download_filecja327 http://ftp.drupal.org/files/projects/drupal-7.23.tar.gz
Downloading drupal-7.23.tar.gz was successful.                                                                                           [notice]
Md5 checksum of drupal-7.23.tar.gz verified.                                                                                             [notice]
Mime type for /tmp/drush_tmp_1383414657_52753b81b2c59/drupal-7.23.tar.gz is application/x-gzip                                           [notice]
Executing: tar -C /tmp/drush_tmp_1383414657_52753b81b2c59 -xzf drupal-7.23.tar.gz
Executing: tar -tzf drupal-7.23.tar.gz
Array to string conversion download.pm.inc:323                                                                                           [notice]
Executing: svn info /var/www/sandbox/drupal
Executing: bzr root /var/www/sandbox/drupal
copy(/tmp/drush_tmp_1383414657_52753b81b2c59/drupal): failed to open stream: No such file or directory filesystem.inc:257             [warning]
Project drupal (7.23) could not be downloaded to /var/www/sandbox/drupal/drupal.                                                      [error]
Command dispatch complete                                                                                                                [notice]

sobiech@sobiech-Dell:/var/www/sandbox/drupal $ php -v
PHP 5.5.3-1ubuntu2 (cli) (built: Oct  9 2013 14:49:12) 
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2013 Zend Technologies
    with Zend OPcache v7.0.3-dev, Copyright (c) 1999-2013, by Zend Technologies
    with Xdebug v2.2.3, Copyright (c) 2002-2013, by Derick Rethans

sobiech@sobiech-Dell:/var/www/sandbox/drupal $ drush version
 Drush Version   :  6.1.0 

[sobi3ch]

Same problem when downloading D6

[jacksmith9]

Confirmed. This is a major issue for as we can't use our deployment system to add sites.

[chrfritsch]

Same problem here. To fix it in makefiles we use now the git download strategy

projects[drupal][type] = "core"
projects[drupal][version] = "7.23"
projects[drupal][download][type] = git
projects[drupal][download][tag] = "7.23"
projects[drupal][download][url] = http://git.drupal.org/project/drupal.git

We have an other problem with downloading themes. They are now located in the modules/contrib folder

[jonhattan]

There's a change in the update xml: http://updates.drupal.org/release-history/drupal/7.x

It is missing terms/term/name = Projects. This term indicates the project type (Drupal core, module, theme....). Drush assumes a module by default. Probably if you try to download a theme or a profile it will be placed in sites/all/modules.

I guess it is related to the drupal.org upgrade to Drupal 7.

[jonhattan]

Related issue https://drupal.org/node/2126123

[NoDiskInDriveA]

The above fix for the core will also work for themes and, in fact, for every project whose type is not 'module'. It works around the issue by completely circumventing the release history.
(It will make 'make_project_needs_release_info' return false, and in consequence, not override the project type later in 'updatexml_determine_project_type')

[sobi3ch]

@NoDiskInDriveA yeap I just realize when I download drupal manually then try to download bootstrap theme via drush and suddenly it end up in sitets/all/module dir(??)

[danylevskyi]

Also there is problem with Drush make. #236.

[jrbeeman]

I found the following works and also avoids a clone of the entire Drupal git repository:

projects[drupal][type] = core
projects[drupal][version] = 7.23
projects[drupal][download][type] = get
projects[drupal][download][url] = http://ftp.drupal.org/files/projects/drupal-7.23.tar.gz

[RikdeBoer]

Just to say that we've just tried jrbeeman's workaround in our stub.make and found it works a treat!

[greg-1-anderson]

If you are not using make, then the following workaround also works for Drush dl:

drush dl drupal --package-handler=git_drupalorg

This does, of course, download the entire repository, though.

[jonhattan]

I'm proposing a fix in this pull request: #237

[nickwshaw]

jrbeeman method works but contrib themes are still downloading to modules/contrib

[jrbeeman]

@nickwshaw, you can explicitly declare the type of project for the theme, and it'll put it in the right location:

projects[adaptivetheme][type] = theme
projects[adaptivetheme][subdir] = contrib
projects[adaptivetheme][version] = 3.1

[crittermike]

FYI, drumm pushed a fix for this to d.o https://drupal.org/comment/8144251#comment-8144251

drush dl drupal is still failing for me but this is presumably because the package needs to be rebuilt now which takes a few hours.

[greg-1-anderson]

drush dl drush does in fact appear to be working again this morning. Do we still need anything in #244?

[jstarcher]

I can confirm that drush dl drupal is working now, however, Drupal still fails to properly download when using make files.

A simple make file:

api = "2"
core = "7.x"
projects[drupal][type] = "core"

Fails:

drush -vd make drupal-core.make foo
Bootstrap to phase 0. [0.01 sec, 4.57 MB]                            [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drush() [0.01 sec, 4.83 MB] [bootstrap]
Cache HIT cid: 5.7-commandfiles-0-7eff0498028cb7879bd9813be6b0ae6e [0.01 sec, 4.85 MB]                                                                                                                                               [debug]
Bootstrap to phase 0. [0.05 sec, 10.04 MB]                                                                                                                                                                                       [bootstrap]
Bootstrap to phase 0. [0.06 sec, 10.04 MB]                                                                                                                                                                                       [bootstrap]
Found command: make (commandfile=make) [0.06 sec, 10.05 MB]                                                                                                                                                                      [bootstrap]
Loading release_info engine. [0.09 sec, 10.06 MB]                                                                                                                                                                                   [notice]
Downloading release history from http://updates.drupal.org/release-history/drupal/7.x [0.09 sec, 10.32 MB]                                                                                                                          [notice]
http---updates.drupal.org-release-history-drupal-7.x retrieved from cache. [0.09 sec, 10.32 MB]                                                                                                                                     [notice]
Backend invoke: /usr/local/bin/php /usr/local/Cellar/drush/5.7/libexec/drush.php --php=/usr/local/bin/php  --backend=2  make-process /tmp/make_tmp_1383664306_52790ab2bed4a 2>&1 [0.14 sec, 10.37 MB]                              [command]
/usr/local/bin/php /usr/local/Cellar/drush/5.7/libexec/drush.php --php=/usr/local/bin/php  --backend=2  make-process /tmp/make_tmp_1383664306_52790ab2bed4a 2>&1 [0.14 sec, 10.37 MB]                                               [notice]
Bootstrap to phase 0. [0.25 sec, 10.04 MB]                                                                                                                                                                                       [bootstrap]
Bootstrap to phase 0. [0.27 sec, 10.05 MB]                                                                                                                                                                                       [bootstrap]
Found command: make-process (commandfile=make) [0.27 sec, 10.05 MB]                                                                                                                                                              [bootstrap]
Loading release_info engine. [0.28 sec, 10.06 MB]                                                                                                                                                                                   [notice]
Backend invoke: /usr/local/bin/php /usr/local/Cellar/drush/5.7/libexec/drush.php --php=/usr/local/bin/php  --backend=2 --yes --root= --uri=  pm-download drupal-7.23                                                               [command]
--destination=/tmp/make_tmp_1383664306_52790ab2bed4a/__build__/sites/all/modules --package-handler=wget --source=http://updates.drupal.org/release-history --variant=profile-only --cache --drupal-project-rename=drupal 2>&1
[0.35 sec, 10.33 MB]
/usr/local/bin/php /usr/local/Cellar/drush/5.7/libexec/drush.php --php=/usr/local/bin/php  --backend=2 --yes --root= --uri=  pm-download drupal-7.23                                                                                [notice]
--destination=/tmp/make_tmp_1383664306_52790ab2bed4a/__build__/sites/all/modules --package-handler=wget --source=http://updates.drupal.org/release-history --variant=profile-only --cache --drupal-project-rename=drupal 2>&1
[0.35 sec, 10.33 MB]
Bootstrap to phase 0. [0.46 sec, 10.04 MB]                                                                                                                                                                                       [bootstrap]
Found command: pm-download (commandfile=pm) [0.47 sec, 10.05 MB]                                                                                                                                                                 [bootstrap]
Loading version_control engine. [0.49 sec, 10.08 MB]                                                                                                                                                                                [notice]
Loading package_handler engine. [0.49 sec, 10.12 MB]                                                                                                                                                                                [notice]
Loading release_info engine. [0.5 sec, 10.16 MB]                                                                                                                                                                                    [notice]
Including /usr/local/Cellar/drush/5.7/libexec/commands/pm/download.pm.inc [0.5 sec, 10.36 MB]                                                                                                                                    [bootstrap]
Downloading release history from http://updates.drupal.org/release-history/drupal/7.x [0.5 sec, 10.56 MB]                                                                                                                           [notice]
http---updates.drupal.org-release-history-drupal-7.x retrieved from cache. [0.5 sec, 10.56 MB]                                                                                                                                      [notice]
Downloading project drupal to /tmp/drush_tmp_1383664307_52790ab32e2e2 ... [0.51 sec, 10.57 MB]                                                                                                                                      [notice]
http---ftp.drupal.org-files-projects-drupal-7.23.tar.gz retrieved from cache. [0.51 sec, 10.57 MB]                                                                                                                                  [notice]
Downloading drupal-7.23.tar.gz was successful. [0.51 sec, 10.57 MB]                                                                                                                                                                 [notice]
Md5 checksum of drupal-7.23.tar.gz verified. [0.52 sec, 10.57 MB]                                                                                                                                                                   [notice]
Fileinfo extension available. [0.52 sec, 10.57 MB]                                                                                                                                                                                   [debug]
Mime type for /tmp/drush_tmp_1383664307_52790ab32e2e2/drupal-7.23.tar.gz is application/x-gzip [0.52 sec, 10.58 MB]                                                                                                                 [notice]
Verifying signature for svn version control engine. [0.75 sec, 10.83 MB]                                                                                                                                                             [debug]
Verifying signature for bzr version control engine. [0.76 sec, 10.59 MB]                                                                                                                                                             [debug]
copy(/tmp/drush_tmp_1383664307_52790ab32e2e2/drupal): failed to open stream: No such file or directory filesystem.inc:229 [0.77 sec, 10.58 MB]                                                                                   [warning]
Project drupal (7.23) could not be downloaded to /private/tmp/make_tmp_1383664306_52790ab2bed4a/__build__/sites/all/modules/drupal. [0.77 sec, 10.42 MB]                                                                         [error]
Command dispatch complete [0.77 sec, 10.53 MB]                                                                                                                                                                                      [notice]
Peak memory usage was 11.27 MB [0.77 sec, 10.53 MB]                                                                                                                                                                                 [memory]
Command dispatch complete [0.87 sec, 10.3 MB]                                                                                                                                                                                       [notice]
Peak memory usage was 11.59 MB [0.87 sec, 10.3 MB]                                                                                                                                                                                  [memory]
Command dispatch complete [0.89 sec, 10.34 MB]                                                                                                                                                                                      [notice]
Peak memory usage was 11.63 MB [0.89 sec, 10.34 MB]

This is after a drush cc drush as well as clearing files in /tmp.

Update: Very strange, I'm able to build with drush make again on my FreeBSD machine but I get the above error on OSX. There must be a cache somewhere that is not getting cleared with drush cc drush. Otherwise maybe this is only broken on OSX now? Can someone confirm?

Update 2: Okay, so this appears to be another bug which is that drush cc drush with drush 5.7 on OSX does not actually clear all of the drush cache. I manually removed the ~/.drush/cache directory and everything is working now.

[jonhattan]

I think we're done. Lets wait the tests https://travis-ci.org/drush-ops/drush/builds

[danylevskyi]

drush make works great now!

[barraponto]

Anyone having issues might need to clean up ~/.drush/cache.

[sobi3ch]

Sweet! Working. I assuming is done now, last 3 builds were fine https://travis-ci.org/drush-ops/drush/builds so closing this now. Thanks guys for good work.

[jonhattan]

update xml is still rebuilding. Some projects won't work yet.

[jonhattan]

Everything in place now. Start making!

[damz]

Now that the fire is gone, let's see how to implement a long term fix, see #249.