Branch settings in hosts.yml are ignored

[vaxul]

Hi,

I have the following setup with Deployer v6.5.0:
hosts.yml:

typo3.example.com:
  hostname: example.com
  user: typo3
  stage: production
  roles: app
  branch: master
  deploy_path: "~/"
typo3-staging.example.com:
  hostname: example.com
  user: typo3-staging
  stage: staging
  roles: app
  branch: develop
  deploy_path: "~/"

deploy.php:

<?php
namespace Deployer;

require_once 'recipe/common.php';

/**
 * Setup
 */
inventory('hosts.yml');

set('allow_anonymous_stats', false);
set('typo3_webroot', 'public');
set('deploy_path_local', '/tmp');
set('release_name', date('Y-m-d_H-i-s'));

/*
 * Repository
 */
set('repository', 'git@some.com/location.git');

// [Optional] Allocate tty for git clone. Default value is false.
set('git_tty', true);

/*
 * Shared files / directories
 */
set('shared_files', [
    '{{typo3_webroot}}/typo3conf/LocalConfiguration.php',
    '{{typo3_webroot}}/.htaccess',
    '.env',
]);

set('shared_dirs', [
    '{{typo3_webroot}}/fileadmin',
    '{{typo3_webroot}}/typo3temp',
    '{{typo3_webroot}}/typo3conf/l10n',
    'var',
]);

/*
 * Cleanup
 */
set('clear_paths', [
    '.git/',
    'build/',
    '.editorconfig',
    '.env.example',
    '.gitignore',
    '.gitattributes',
    'composer.json',
    'composer.lock',
    'deploy.php',
    'hosts.yml',
    '*.md',
]);

/*
 * Helper
 */
set('copy_dirs', [
    '.',
]);

set('writable_dirs', [
    '{{typo3_webroot}}/fileadmin',
    '{{typo3_webroot}}/typo3temp',
    '{{typo3_webroot}}/typo3conf',
]);

/**
 * Custom Tasks
 */
/*
 * Local build
 */
task('deploy:build', function () {
    set('deploy_path', get('deploy_path_local'));
    invoke('deploy:prepare');
    invoke('deploy:release');
    invoke('deploy:update_code');
    invoke('deploy:vendors');
    invoke('deploy:clear_paths');
})->local();

/*
 * Upload local => external
 */
task('deploy:upload', function () {
    upload(
        '{{deploy_path_local}}/release/',
        '{{release_path}}',
        [
            'options' => [
                '--delete'
            ]
        ]
    );
});

/*
 * Updates
 */
task('deploy:cli_scripts', function () {
    run('{{release_path}}/vendor/bin/typo3cms install:fixfolderstructure');
    run('{{release_path}}/vendor/bin/typo3cms install:generatepackagestates');
    run('{{release_path}}/vendor/bin/typo3cms database:updateschema');
});

/**
 * Deployment
 */
task('deploy', [
    'deploy:info',
    'deploy:prepare',
    'deploy:lock',
    'deploy:release',
    'deploy:build',
    'deploy:copy_dirs',
    'deploy:upload',
    'deploy:shared',
    'deploy:writable',
    'deploy:cli_scripts',
    'deploy:symlink',
    'deploy:unlock',
    'cleanup',
])->desc('Deploy your project');

after('deploy', 'success');
after('deploy:failed', 'deploy:unlock');

What happens:
If I deploy the staging stage with vendor/bin/dep deploy staging it deploys a version on the correct host, but always the master-Branch.
It works only, if a explicitly add the --branch-option to the deploy-command.

Result on the command line:

user@host:~$ vendor/bin/dep deploy staging
✈︎ Deploying develop on example.com
➤ Executing task deploy:prepare
✔ Executing task deploy:lock
✔ Executing task deploy:release
➤ Executing task deploy:build
✔ Executing task deploy:prepare
✔ Executing task deploy:release
➤ Executing task deploy:update_code
Klone nach '/tmp/releases/2019-10-28_09-23-36' ...
remote: Enumerating objects: 4188, done.
remote: Counting objects: 100% (4188/4188), done.
remote: Compressing objects: 100% (2072/2072), done.
remote: Total 4188 (delta 1820), reused 4147 (delta 1786)
Empfange Objekte: 100% (4188/4188), 6.26 MiB | 0 bytes/s, Fertig.
Löse Unterschiede auf: 100% (1820/1820), Fertig.
✔rüfe Konnektivität ... Fertig.
✔ Executing task deploy:vendors
✔ Executing task deploy:clear_paths
✔ Ok
✔ Executing task deploy:copy_dirs
➤ Executing task deploy:upload
✔ Executing task deploy:shared
✔ Executing task deploy:writable
✔ Executing task deploy:cli_scripts
✔ Executing task deploy:symlink
✔ Executing task deploy:unlock
✔ Executing task cleanup
Successfully deployed!

What should happen:
Deployer should respect the branch-setting in the hosts.yml.

What I know so far:
This condition isn't resolved to true in my case:

deployer/recipe/deploy/update_code.php

Lines 61 to 64 in 861f1a3

	$at = '';
	if (!empty($branch)) {
	$at = "-b $branch";
	}

How to resolve this issue? Do I missed something?

[vaxul]

How can I help to fix this issue?

[antonmedv]

Hi, we need to take a closer look at how branch options is overriding each other from different places.

[antonmedv]

Also, I think here ->local(); may cause some trouble. Try to use localhost() instead of.

[vaxul]

Is it possible to use localhost() in combination with hosts.yml?
I'd like to have one source of truth for the hosts.

[antonmedv]

Hmm. B n ever was trying to use it as you describe.

[vaxul]

As far as I understand is the usecase of hosts.yml, that you can have the same deploy.php for each project and only configure the hosts.
But the docs recommend something different. 🤔

I try to add localhost(). Since only the target (external) hosts are currently in the hosts.yml

Do you have any example for these? The docs aren't helping much.

I have to basically build it locally and push it to the hosts (as in the deploy.php in the first post).

I've testet:
hosts.yml

typo3.example.com:
  hostname: example.com
  user: typo3
  stage: production
  roles: build, app
  branch: master
  deploy_path: "~/"
typo3-staging.example.com:
  hostname: example.com
  user: typo3-staging
  stage: staging
  roles: build, app
  branch: develop
  deploy_path: "~/"

deploy.php

<?php
namespace Deployer;

require_once 'recipe/common.php';

/**
 * Setup
 */
inventory('hosts.yml');

localhost()
    ->roles('build');

set('allow_anonymous_stats', false);
set('typo3_webroot', 'public');
set('deploy_path_local', '/tmp');
set('release_name', date('Y-m-d_H-i-s'));

/*
 * Repository
 */
set('repository', 'git@some.com/location.git');

// [Optional] Allocate tty for git clone. Default value is false.
set('git_tty', true);

/*
 * Shared files / directories
 */
set('shared_files', [
    '{{typo3_webroot}}/typo3conf/LocalConfiguration.php',
    '{{typo3_webroot}}/.htaccess',
    '.env',
]);

set('shared_dirs', [
    '{{typo3_webroot}}/fileadmin',
    '{{typo3_webroot}}/typo3temp',
    '{{typo3_webroot}}/typo3conf/l10n',
    'var',
    'private/fileadmin',
]);

/*
 * Cleanup
 */
set('clear_paths', [
    '.git/',
    'build/',
    '.editorconfig',
    '.env.example',
    '.gitignore',
    '.gitattributes',
    'composer.json',
    'composer.lock',
    'deploy.php',
    'hosts.yml',
    '*.md',
]);

/*
 * Helper
 */
set('copy_dirs', [
    '.',
]);

set('writable_dirs', [
    '{{typo3_webroot}}/fileadmin',
    '{{typo3_webroot}}/typo3temp',
    '{{typo3_webroot}}/typo3conf',
    'private/fileadmin',
]);

/**
 * Custom Tasks
 */
/*
 * Local build
 */
task('deploy:build', function () {
    set('deploy_path', get('deploy_path_local'));
    invoke('deploy:prepare');
    invoke('deploy:release');
    invoke('deploy:update_code');
//    invoke('deploy:vendors');
//    invoke('deploy:clear_paths');
})->onRoles('build');

/*
 * Upload local => external
 */
task('deploy:upload', function () {
    upload(
        '{{deploy_path_local}}/release/',
        '{{release_path}}',
        [
            'options' => [
                '--delete'
            ]
        ]
    );
});

/*
 * Updates
 */
task('deploy:cli_scripts', function () {
    run('{{release_path}}/vendor/bin/typo3cms install:fixfolderstructure');
    run('{{release_path}}/vendor/bin/typo3cms install:generatepackagestates');
    run('{{release_path}}/vendor/bin/typo3cms database:updateschema');
    run('{{release_path}}/vendor/bin/typo3cms cache:flush');
});

/**
 * Deployment
 */
task('deploy', [
    'deploy:info',
//    'deploy:prepare',
//    'deploy:lock',
//    'deploy:release',
    'deploy:build',
//    'deploy:copy_dirs',
//    'deploy:upload',
//    'deploy:shared',
//    'deploy:writable',
//    'deploy:cli_scripts',
//    'deploy:symlink',
//    'deploy:unlock',
//    'cleanup',
])->desc('Deploy your project');

after('deploy', 'success');
after('deploy:failed', 'deploy:unlock');

Sadly this doesn't checkout anything from git.

[vaxul]

I try to debug why the branch is empty in my initial case.

When update_code tries to get('branch'), the context doesn't hold any of this information available and it's run the else block:

deployer/src/Configuration/Configuration.php

Lines 55 to 79 in 5f32216

	public function get(string $name, $default = null)
	{
	if ($this->collection->has($name)) {
	if ($this->isClosure($this->collection[$name])) {
	$value = $this->collection[$name] = call_user_func($this->collection[$name]);
	} else {
	$value = $this->collection[$name];
	}
	} else {
	$config = Deployer::get()->config;
	if (isset($config[$name])) {
	if ($this->isClosure($config[$name])) {
	$value = $this->collection[$name] = call_user_func($config[$name]);
	} else {
	$value = $this->collection[$name] = $config[$name];
	}
	} else {
	if (null === $default) {
	throw new ConfigurationException("Configuration parameter `$name` does not exist.");
	} else {
	$value = $default;
	}
	}
	}

$this->collection is:

result = {Deployer\Collection\Collection} [1]
 values = {array} [9]
  deploy_path_local = "/tmp"
  deploy_path = "/tmp"
  env = {array} [0]
  working_path = false
  releases_list = {array} [1]
  release_name = "2019-12-18_10-43-09"
  use_relative_symlink = false
  bin/symlink = "ln -nfs"
  repository = "git@some.com/location.git"

And $config = Deployer::get()->config

result = {Deployer\Collection\Collection} [1]
 values = {array} [40]
  ssh_multiplexing = true
  default_stage = null
  release_name = "2019-12-18_10-43-09"
  releases_list = {Closure} [0]
  release_path = {Closure} [0]
  branch = {Closure} [0]
  git_cache = {Closure} [0]
  hostname = {Closure} [0]
  user = {Closure} [0]
  target = {Closure} [0]
  keep_releases = {int} 5
  repository = "git@some.com/location.git"
  shared_dirs = {array} [5]
  shared_files = {array} [3]
  copy_dirs = {array} [1]
  writable_dirs = {array} [4]
  writable_mode = "acl"
  writable_use_sudo = false
  writable_recursive = true
  writable_chmod_mode = "0755"
  writable_chmod_recursive = true
  http_user = false
  http_group = false
  clear_paths = {array} [11]
  clear_use_sudo = false
  cleanup_use_sudo = false
  use_relative_symlink = {Closure} [0]
  use_atomic_symlink = {Closure} [0]
  composer_action = "install"
  composer_options = "{{composer_action}} --verbose --prefer-dist --no-progress --no-interaction --no-dev --optimize-autoloader --no-suggest"
  env = {array} [0]
  current_path = {Closure} [0]
  bin/php = {Closure} [0]
  bin/git = {Closure} [0]
  bin/composer = {Closure} [0]
  bin/symlink = {Closure} [0]
  allow_anonymous_stats = false
  typo3_webroot = "public"
  deploy_path_local = "/tmp"
  git_tty = true

So the branch isn't inherited from the target host.

Would it be possible to read the current target host within this context?

deployer/recipe/deploy/update_code.php

Line 62 in 5f32216

if (!empty($branch)) {

$host = Context::get()->getHost(); get's localhost, but not the target host with it's roles and branch.

[antonmedv]

What if you override branch with set function?

set('branch', ...);

[vaxul]

At which context/place?

[antonmedv]

Something like this: https://github.com/deployphp/deployer/blob/master/recipe/deploy/update_code.php#L13-L29

[vaxul]

When I add this override in my deploy.php, I can't read the origin target host:

set('branch', function () {
# ----
    $host = Context::get()->getHost();
    $config = $host->getConfig();
# ----
    try {
        $branch = runLocally('git rev-parse --abbrev-ref HEAD');
    } catch (\Throwable $exception) {
        $branch = null;
    }
    if ($branch === 'HEAD') {
        $branch = null; // Travis-CI fix
    }

    if (input()->hasOption('branch') && !empty(input()->getOption('branch'))) {
        $branch = input()->getOption('branch');
    }

    return $branch;
});

It's still localhost.

[vaxul]

I did some puzzling, but sadly without success:

<?php
namespace Deployer;

use Deployer\Task\Context;

require_once 'recipe/common.php';

/**
 * Setup
 */
inventory('hosts.yml');

// ...

// --- NEW
set('target_host_branch', function () {
    $hostConfig = Context::get()->getHost()->getConfig();
    return $hostConfig->get('branch');
});
// --- NEW

// ...

/**
 * Custom Tasks
 */

// --- NEW
task('config:load_branch', function () {
	// Get's the correct branch, but doesn't set it
    set('branch', get('target_host_branch'));
});
// ---


/*
 * Local build
 */
task('deploy:build', function () {
    set('deploy_path', get('deploy_path_local'));
    invoke('deploy:prepare');
    invoke('deploy:release');
    invoke('deploy:update_code');
//    invoke('deploy:vendors');
//    invoke('deploy:clear_paths');
})->local();

// --- NEW
before('deploy:build', 'config:load_branch');
// --- 

// ...

/**
 * Deployment
 */
task('deploy', [
    'deploy:info',
//    'deploy:prepare',
//    'deploy:lock',
//    'deploy:release',
    'deploy:build',
//    'deploy:copy_dirs',
//    'deploy:upload',
//    'deploy:shared',
//    'deploy:writable',
//    'deploy:cli_scripts',
//    'deploy:symlink',
//    'deploy:unlock',
//    'cleanup',
])->desc('Deploy your project');

after('deploy', 'success');
after('deploy:failed', 'deploy:unlock');

Withing set('target_host_branch') the correct branch is loaded, but it doesn't is set within task('config:load_branch').

The set('branch') from recipe/deploy/update_code.php is invoked.

[vaxul]

Why was this issue closed without comment? I still not found a solution to this issue.

[antonmedv]

It's should be fixed in master. Now all settings live in configuration and globally/per host overrateable.

[vaxul]

Thanks, I will test it. :)

[antonmedv]

Note, what master isn't finished yet. A lot of new and awesome stuff going on right now there.

[ggolda]

Was this issue fixed? Have the exact same problem with deployer 6.8.0.

[vaxul]

@ggolda
Sadly, I wasn't able to test this.

@antonmedv
Maybe reopen this issue?

[antonmedv]

I already fixed it in master. For bookkeeping I’d like to mark it as closed to not popup in search when I’m working on other issues.