Deploy to multiple boxes simultaneously

[pointlessone]

If I'm not mistaken it's possible to deploy only to single box. I.e. if I have a bunch of identical boxes I have to deploy to each of them explicitly specifying to which box I deploy. Something like this:

mina deploy to=prodbox1 && mina deploy to=prodbox2 && mina deploy to=prodbox3

Is parallel deployment planned anytime soon?

[ghost]

I would also be interested in the answer to this.

[rstacruz]

Yes, I'd want to see this before 1.0 myself. Discussions: https://trello.com/c/cukv1ztZ

[sarguru]

set :domains, ['test.deeptest.in,'test1.deeptest.in']
task :multi_deploy do
domains.each do |domain|
set :domain,domain
invoke :deploy
run!
end
end

This works for me. Any better strategies anybody else is using?

[phlipper]

I used a solution similar to @sarguru but I needed to add the isolate block. Without it, each call to invoke in the loop basically duplicates the deployment. The 2nd host is deployed twice, the 3rd host is deployed three times, etc. Here is the code:

set :domains, %w[host1 host2 host3]

desc "Deploy to all servers"
task :deploy_all do
  isolate do
    domains.each do |domain|
      set :domain, domain
      invoke :deploy
      run!
    end
  end
end

While this approach does deploy to multiple hosts, it does so one at a time, which can be an issue for longer deploys. It would be nice to see a natively supported parallel deployment approach at some point.

[Markus99999]

chances that parallel deployment is becoming reality ... ?

[rsbrown]

bumping this question. is the approach recommended by @sarguru and @phlipper the official solution or is there a plan to add parallel deployment support directly within mina?

fyi, mina is a great tool -- fantastic work.

[tomas]

+1 to this issue. This is the only thing that's keeping me from switching entirely from Capistrano.

@phlipper's trick works, but I think it's a bit dangerous to deploy in sequence -- e.g. what happens when the last server you're deploying to fails? -- so I'm still waiting for a parallel implementation.

Keep up the good work @rstacruz!

[zeroasterisk]

I agree - this is an excellent question, and a good approach has been proposed...

Ideally a segmentation between "build" and "release/symlinking to current";

that is... build on all servers in parallel and wait for confirmation (no errors)... if confirmed, then "release and restart"

I know this is a bit complicated with the "1 shell script to rule them all" strategy, but I think it's an excellent feature/approach and something worth considering.

[jrhorn424]

I'm not sure if this issue is close enough to supporting multistage to warrant a new issue. Just adding my voice to say multistage deploy would be great.

[calvinl]

+1!

[masterkain]

I took this feature for granted while migrating away from Capistrano. +1

[mbahar]

the above solution din't work for me...

My solution is encapsulating mina command with grunt; and use multiple server deployment loop with grunt.js

module.exports = function(grunt) {

// Project configuration.
grunt.initConfig({
    exec: {
        mina: {
            cmd: function(action, server, key) {
                var mina_command = 'mina ' + action + ' key=' + key + ' server=' + server + ' --verbose  ' ;
                grunt.log.write(mina_command).ok();

                return  mina_command;
            }
        },
        pwd : {

            cmd: 'pwd'
        }
    }

});

 // -----------------// -----------------// -----------------// -----------------
 // HELPER FUNCTIONS
 // -----------------// -----------------// -----------------// -----------------

// mina is used for SSH client for server deployements and configuration management
// configuration for mina is under /config/deploy.rb
// here is the function for wrapping mina functionality for our purposes

var runMina= function (action, server_type,user,branch) {

    // import list of servers
    var servers= grunt.file.readJSON('config/servers/'+ server_type+'.json');

    grunt.log.write('Setting up '+server_type+' servers...').ok();

    var key='';
    if (server_type=='production') key= '---- your pem here ------- '; else key= '---- your pem here -------';

    for (var server in servers) {
        grunt.task.run('exec:pwd');
        grunt.task.run('exec:mina:'+action+':'+user+'@'+servers[server]+':'+key + ' branch=' + branch);
    }

}


// -----------------// -----------------// -----------------// -----------------
// TASKS
// -----------------// -----------------// -----------------// -----------------


// A very basic default task.
grunt.registerTask('default', 'Log some stuff.', function() {
    grunt.log.write('Logging some stuff...').ok();
});




// setup servers for deployment

// deployments
grunt.registerTask('deploy-setup', 'Setting up server for deployments...', function(arg1) {

    var user='ec2-user';

    if (arguments.length === 0) {
        grunt.log.error(this.name + ", define a setup target ");
        return false;
    }

    switch (arg1) {

        case 'production':

            runMina('setup','production',user);

            break;

        case 'testing':

            runMina('setup','testing',user);

            break;

        default:

    }


});



// deployments
grunt.registerTask('deploy', 'Deploy to Servers...', function(arg1,arg2) {

    var user='ec2-user';

    if (arguments.length === 0) {
        grunt.log.error(this.name + ", define a deployment target ");
        return false;
     }


    // defined the branch; default is the master
    var branch='master';
    if (arg2) {  branch =arg2; }

    switch (arg1) {

        case 'production':

            runMina('deploy','production',user,branch);

            break;

        case 'testing':

            runMina('deploy','testing',user,branch);

            break;

         default:

    }


});

// https://github.com/jharding/grunt-exec
grunt.loadNpmTasks('grunt-exec');

};

[zacksiri]

Actually it's not that hard. I setup the app directory as an NFS share. And basically all host has a reference to that directory. Once the changes are pushed all the host receive changes via NFS and I have a small socket server client setup that sends a message from the main server to all host to restart the server. Mina only interfaces with 1 primary host and the primary host handles the rest.

[misteral]

+1

[introvert]

• 1

[ronwsmith]

+1

[ellefsen]

+1

[fazibear]

What about https://github.com/grosser/parallel gem ?

[neersighted]

Still waiting on this...

[modx-space]

any solutions?

[sergeifilippov]

+1

[hollodotme]

+1

[ShonM]

👍

[ghost]

+1

[huobazi]

+1

[frozzare]

Any solutions @gabskoro?

[citrus]

+1

[warmwind]

+1

[ronwsmith]

@bbnnt, you are missing the key issue -- parallel vs serial. The workaround is for multiple hosts in serial, but it's slow on large infrastructures. Ideally, there's a way to release to all nodes at the same time.

[benbonnet]

@ronwsmith thanks for pointing this out

[livecube]

Thanks @phlipper - you saved my life.

+1 for true parallel deployments!

[iMoses]

Almost three years and not a step forward, let alone a decent response?

[zacksiri]

capistrano 3 is actually really good i've switched to using it.

On Tue, Mar 17, 2015 at 9:27 AM, iMoses notifications@github.com wrote:

Almost three years and not a step forward, let alone a decent response?

—
Reply to this email directly or view it on GitHub
#8 (comment).

[josericardo]

Based on the first scenario described by @cheba, it could be as simple as:

cat list_of_servers.txt | parallel mina deploy to={}

Are there any drawbacks on running multiple mina processes?

[pointlessone]

@josericardo It's sort of working but not quite. Specifically, you may not want to perform complete deploy on each box. You want to run migrations only once (say, if you have many app servers and one db server). You may want to upload your assets to CDN only once. There are legitimate reasons for having native support for multiple destinations.

[charneykaye]

https://www.npmjs.com/package/mina

[brendonrapp]

Just as I was digging into Mina for our new deployment setup, I run into this and get stopped dead in my tracks.

Deploying to multiple application servers isn't just a matter of "run the task on all boxes". There's also coordination, eg. if the deploy on one box fails, recognize that and roll back the deployment on all of them, lest you end up with different versions running at the same time. Particularly if you're doing something like building and pushing assets to a remote CDN, this kind of coordination is crucial. Hence just looping through a domain list and firing off the deploy task on each of them is no solution.

Very disappointing, complete deal-breaker.

[charneykaye]

Hey @brendonrapp yes, I agree. I love the concept of Mina, but the bottom line is that Capistrano is a vastly more mature product. It represents many years of real-world experience deploying complex production environments. The Mina project is thousands of developer hours short of it.

[d4be4st]

We need help on this. As we do not use multiple servers in our company there really is not a high prioriy.
But we would appreciate PRs.
Thx

[akbarbin]

I just want to share how to use multiple deploying server or environment with mina. Mina have a 3rd party to deploy in multiple stages. You can use mina-multistages this the url https://github.com/endoze/mina-multistage. It's really powerful to manage your deploy env.
I hope this help you all. Thanks

[gknapp]

I read about using Mina in conjunction with Ansible for multi host deployments.

+1 to see this added natively to mina

[grsahil20]

• 1 BUMP

[spalenza]

+1

[Mothirajha]

any solution for parallel deployment using mina ??

[at-binhcq]

Still waiting for native sollution.
While waiting, I'm still using @phlipper 's solution.

[d4be4st]

Mina doesn't do multiple boxes and probably never, because it is meant to be used as one quick script on one remote host.
I am sorry about this.
There is a great tool [capistrano] that does this job well.

[Juanmcuello]

I created a very simple gem called mina-multi_server that adds support for multiple servers. It runs tasks sequentially, but probably it is not an issue if you don't have a lot of servers.

All comments are appreciated.

[hugopl]

This is a blocker to anyone using e.g. sidekiq server in another dedicated box.

[tinyc-z]

mina 2.x

set :domains, %w[host1 host2 host3]
desc "Deploy to all servers"
task :deploy_all do
    fetch(:domains).each do |domain|
      set :domain, domain
      invoke :deploy
    end
end

[msshahanshah]

+1

[volkov-sergey]

We built mina-multideploy - useful tool for parallel deploying on multiple servers.

[emptyflask]

Something like this could work for handling roles. Again, not parallel (yet).

task :staging do
  set :domains, 'staging-1.app.com' => [:db], 'staging-2.app.com' => []
end

task :deploy do
  invoke :'git:ensure_pushed'

  fetch(:domains).each do |domain, roles|
    set :domain, domain

    deploy do
      invoke :'git:clone'
      invoke :'deploy:link_shared_paths'
      invoke :'bundle:install'
      invoke :'rails:db_migrate' if roles.include?(:db)
      invoke :'rails:assets_precompile'
      invoke :'deploy:cleanup'

      on :launch do
        in_path(fetch(:current_path)) do
          command %(mkdir -p tmp/)
          command %(touch tmp/restart.txt)
          command %(direnv allow)
        end
      end
    end
  end
end