Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support --detach flag in docker stack deploy / rm #373

Open
dedalusj opened this issue Jul 21, 2017 · 32 comments
Open

Support --detach flag in docker stack deploy / rm #373

dedalusj opened this issue Jul 21, 2017 · 32 comments

Comments

@dedalusj
Copy link

@dedalusj dedalusj commented Jul 21, 2017

The docker stack deploy command should support a --detach flag much like the docker service create and docker service update commands.

@dnephin
Copy link
Contributor

@dnephin dnephin commented Jul 26, 2017

When --detach=false is used, what should we show as output? I assume progress bars, should we have a progress bar per task (like we have with service commands), or should it be a progress bar per service?

cc @thaJeztah @vdemeester

Loading

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Jul 26, 2017

Perhaps the same (set of) progress bars as for service create/update - grouped per service (although I can see that becoming too verbose with large stacks containing lots of services

Should we have someone from the UX / design team craft a design?

Loading

@dnephin
Copy link
Contributor

@dnephin dnephin commented Jul 26, 2017

That might be overkill. We need to keep it consistent with the service commands, so I think there are only really two options. Either progress of tasks (grouped if possible as you've suggested) or progress of services.

Loading

@dedalusj
Copy link
Author

@dedalusj dedalusj commented Jul 27, 2017

Progress of services seems a good choice to me.

When I'm deploying a stack of a few services the number of underlying tasks can be large and progress can be overwhelming. I would quickly tune out of it.

Loading

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Jul 27, 2017

IIRC, the progress bars for services already have a threshold; if more than X tasks for a service, then a single progress bar is shown, instead of per task (I should check the code)

Loading

@akalipetis
Copy link
Contributor

@akalipetis akalipetis commented Aug 8, 2017

I'd say that anything returning 0 if all the updates are successful and something else otherwise (like the total number of failed services?) should be more than enough as a start.

Progress bars are definitely a nice addition, but it's more important IMO to be able to know if your deployment succeeded or not.

Currently, you cannot reliably deploy using compose file from a CI service.

Given that waitOnService is already implemented, we could simply use the exact same logic for stacks and revisit if needed. Otherwise I believe that just waiting and giving an exit code is more than enough.

Loading

@dnephin
Copy link
Contributor

@dnephin dnephin commented Aug 28, 2017

I just noticed the origin issue (moby/moby#32367) suggested the progress bars would be one per service, instead of per task.

Loading

@sirlatrom
Copy link
Contributor

@sirlatrom sirlatrom commented Aug 28, 2017

@dnephin It is my impression that that is how docker service scale --detach works too. Am I right?

Loading

@dnephin
Copy link
Contributor

@dnephin dnephin commented Aug 28, 2017

No, I'm pretty sure all the docker service commands all show progress bars for each task, not for each service.

Loading

@sirlatrom
Copy link
Contributor

@sirlatrom sirlatrom commented Aug 28, 2017

@dnephin You are correct, of course. I suppose any command that is supposed to show the status of multiple services interactively should provide a similar user experience? Which would that be, one progress bar per task, or one per service?

Loading

@dustin-decker
Copy link

@dustin-decker dustin-decker commented Sep 11, 2017

detach=false on a service changes behavior when you exceed the maxProgressBars constant for both replicated and global services:
d8ab384#diff-5bd6b950bc904a376a90157b719668a0R255

Based on that, a similar approach should probably be used for stack deploy, showing tasks if tasks<maxProgressBars, and services if it exceeds that constant.

Loading

@shouze
Copy link
Contributor

@shouze shouze commented Nov 20, 2017

I'm very interested about that one. I don't know about you guys but typically we use to run our e2e tests on a true docker (mono host) swarm... so very handy to easily know when we can launch our e2e test suite against the deployed infra.

I'm ok to work on this one but 1st I have to understand well on what I can rely. I've listed the current/desired states returned by docker stack ps:

  • Current state:
    • pending
    • starting
    • complete
    • running
    • failed
  • Desired state:
    • running
    • shutdown
    • accepted

Also the Error field can be used I guess in case of desired state shutdown?

Note: Since few weeks we also have some services deployed with a restart policy condition to none, this is a good illustration of a short lived / run once service I guess that the --detach=false should support too and should be illustrated in tests.

Loading

@shouze
Copy link
Contributor

@shouze shouze commented Nov 20, 2017

Ok, after some further investigation in fact it's pretty straight to the point as docker stack deploy implementation call under the hood some service update/create that already support --detach=false option.

Loading

@shouze
Copy link
Contributor

@shouze shouze commented Nov 21, 2017

I have created this project until this feature is available in docker cli: https://github.com/karibbu/docker-php

Loading

@nullobject
Copy link

@nullobject nullobject commented Feb 7, 2018

Until this feature is implemented, how do I tell if a stack was updated successfully?

Loading

@mattmattmatt
Copy link

@mattmattmatt mattmattmatt commented Feb 13, 2018

Any update on this, or on nullobject's question above?
This tool tries to help with that but native support is certainly preferred.

Loading

@AdrianSkierniewski
Copy link

@AdrianSkierniewski AdrianSkierniewski commented Apr 19, 2018

Is there any news when this could be implemented? I'd like to use docker stack deploy together with ansible but don't know how to monitor if that deployment went successful.

Loading

@ushuz
Copy link

@ushuz ushuz commented Apr 19, 2018

@AdrianSkierniewski

For deploying stack with Ansible, I modified docker_stack module (ansible/ansible#24588) to deploy stacks and mimic docker service's behaviour to wait for services to be converged.

You can find the modified module in this gist, hope it's useful.

Loading

@AdrianSkierniewski
Copy link

@AdrianSkierniewski AdrianSkierniewski commented Apr 19, 2018

@ushuz I didn't know about that module. Thank you!

Loading

@thaJeztah thaJeztah changed the title Support --detach flag in docker stack deploy Support --detach flag in docker stack deploy / rm May 7, 2018
@yangm97
Copy link

@yangm97 yangm97 commented Jul 5, 2018

Updates?

Loading

@aselvan
Copy link

@aselvan aselvan commented Jul 20, 2018

Is there a target release date for this enhancement?

Loading

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Jul 21, 2018

I don't think someone is working on this currently, but contributions welcome if someone wants to work on this

Loading

@drozzy
Copy link

@drozzy drozzy commented Aug 15, 2018

Without this feature, it is not clear whether we should proceed to the next step in CI/CD or not.

Also, it is not clear, what happens if we run docker stack deploy and then exit the shell session that executed that command. Is the deploy only partially done? Or completely done?

Loading

@sorenhansendk
Copy link

@sorenhansendk sorenhansendk commented Aug 15, 2018

@drozzy: We have the same problem, but we solved it with few lines of Powershell.

Run this right after your docker stack deploy command:

Do
{
    $Services = $(docker stack services [YOUR-STACK-NAME] -q);
    $Inspects = (docker service inspect $Services) | ConvertFrom-Json

    $Updating = @();
    $Completed = @();
    $Rolledback = @();

    ForEach ($Service in $Inspects | Where-Object { $_.UpdateStatus -ne $null })
    {
        if ($Service.UpdateStatus.State -eq 'completed') {
            $Completed += $Service
        } elseif ($Service.UpdateStatus.State -like 'rollback*') {
            $Rolledback += $Service
        }

        $Updating += $Service;
    }

    "Waiting for {0} service(s) to update ({1} completed / {2} rollback)" -f $Updating.Count, $Completed.Count, $Rolledback.Count
    Start-Sleep 5

} Until ($Updating.Count -eq ($Completed.Count + $Rolledback.Count))

Loading

@drozzy
Copy link

@drozzy drozzy commented Aug 15, 2018

Thanks, but my understanding was that swarm accepts the command right away and you’re free to logout of the shell?

Are you saying you need to hang around until it’s all done? What if there is an error and rollback or failure with an unhealthy service?

Loading

@kinghuang
Copy link

@kinghuang kinghuang commented Aug 15, 2018

@drozzy Once the deploy command returns, there's no need to wait around. You can freely exit the shell.

That doesn't mean that the deployment is done, in the sense that all services are running. But, the CLI's job is done, and it's up to the swarm to reach the desired state.

Loading

@sorenhansendk
Copy link

@sorenhansendk sorenhansendk commented Aug 15, 2018

@drozzy: Yes. We wait on all our updates is completely done - so we can warn the developers about a rollback action, if anything has failed in the deployment. We have some extra code to mark a deployment as failed. But I don't copied that, into the code snippet above :-)

Loading

@drozzy
Copy link

@drozzy drozzy commented Aug 16, 2018

Thanks guys! Is there any way to know that a rollback was done (if I set my docker swarm to rollback automatically)?
Otherwise, it basically looks like I'm just waiting and waiting for a result, thinking that my deployment is not done (when I'm using CI/CD) - but in reality rollback has been performed!

Perhaps some sort of event is fired?

If not, I guess I would have to do something along the lines as @sorenhansendk suggested. But I think this is something docker swarm should provide out of the box (unless I'm missing something?)

Loading

@sorenhansendk
Copy link

@sorenhansendk sorenhansendk commented Aug 16, 2018

@drozzy: Yes - please change this line:

($Service.UpdateStatus.State -like 'rollback*')

to this one:

($Service.UpdateStatus.State -eq 'rollback_completed')

Loading

@sudo-bmitch
Copy link
Contributor

@sudo-bmitch sudo-bmitch commented Oct 8, 2018

Adding to the list of external implementations, here's a script to run after a docker stack deploy to wait until the deploy completes.
https://github.com/sudo-bmitch/docker-stack-wait

Loading

@vitalets
Copy link

@vitalets vitalets commented Aug 23, 2020

I'd like to share my script waiting for docker stack deploy to complete.
https://github.com/vitalets/docker-stack-wait-deploy

Compared to existing solutions it shows update progress per instance of each service.
Example output:

STATUS (nodejs_service): updating
ID                  NAME                   IMAGE                     NODE                DESIRED STATE       CURRENT STATE             ERROR               PORTS
h1isbgdgmwv1        nodejs_service.1       nodejs_service_prod:1.5   swarm-manager       Running             Preparing 2 seconds ago                       
8zu8v3waxyjx         \_ nodejs_service.1   nodejs_service_prod:1.4   swarm-manager       Running             Running 6 minutes ago                         
b45471xoq9d4        nodejs_service.2       nodejs_service_prod:1.4   swarm-manager       Running             Running 6 minutes ago                         
xyagddf48560        nodejs_service.3       nodejs_service_prod:1.5   swarm-manager       Running             Running 3 seconds ago                         

Loading

@decentral1se
Copy link

@decentral1se decentral1se commented Jun 15, 2021

If someone would like to discuss being funded to work on this issue, please gimme a shout lukewm@riseup.net. We need this as part of a project which we're using swarm as the underlying orchestrator and having to write custom workarounds for the fact that --detach is missing is not ideal. There are so many +1s on this issue, clearly a lot of people need this to get done.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet