Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a roadmap for docker swarm? #2665

Open
bitsofinfo opened this issue Jun 18, 2018 · 25 comments
Open

Is there a roadmap for docker swarm? #2665

bitsofinfo opened this issue Jun 18, 2018 · 25 comments

Comments

@bitsofinfo
Copy link

@bitsofinfo bitsofinfo commented Jun 18, 2018

Is there a roadmap for swarm? Some of us out here using swarm are starting to get concerned about Docker's future plans for swarm given all the focus from Docker seems to be around Kubernetes

Just as an example, looking at the recent 2018 Dockercon agenda there 67 Kubernetes references and only 11 for swarm.

The wiki page here lists 1.3.0 w/ a target date of September of 2017
https://github.com/docker/swarm/wiki

Shows about 44% complete here: https://github.com/docker/swarm/milestones

What is the current and future roadmap for swarm?

Is swarm being put into a maintenance mode only in favor of Kubernetes?

https://forums.docker.com/t/roadmap-for-swarm/48680

@BretFisher

This comment has been minimized.

Copy link

@BretFisher BretFisher commented Jun 21, 2018

Swarm is alive and well. There's no roadmap that I know of that's kept up to date. The docker/swarm links you refer to is Swarm Classic, a solution used before SwarmKit was released in 1.12 (July 2016) that replaced the similarly named Swarm repo. docker/swarm is now deprecated in favor of this repo (aka Swarm Mode, or just Docker Services). Lots of Swarm fans at DockerCon.

I wrote a few months ago about Swarm's future, which received 👏 from the Docker engineering team. http://www.bretfisher.com/is-swarm-dead-answered-by-a-docker-captain/

Also, I gave a session on Swarm production and it was voted a top session and was repeated on the last day. It was a full house: https://twitter.com/BretFisher/status/1007099831783862272

So true, there is no public roadmap for SwarmKit, though one could argue you might look at recently updated PR's but that's not a guaranteed list of future features.

If you want to learn what's changed recently, you can look at the Docker CE release notes... looking for anything Swarm Mode and Overlay Networking related.

@BretFisher

This comment has been minimized.

Copy link

@BretFisher BretFisher commented Jun 21, 2018

Also of note, as Docker Captains, we were lucky enough to sit with various Docker Inc. product managers At DockerCon 2018 and they were all committed to the future of SwarmKit/Swarm Mode, saying that customers loved it. At this point I think it's just a question of engineering priorities across the Docker toolset and bugfixes vs. new features... but I can't speak for Docker Inc. directly :)

@bitsofinfo

This comment has been minimized.

Copy link
Author

@bitsofinfo bitsofinfo commented Jun 21, 2018

Bret - yes have read your blog post previously and appreciated that work and post you made, it was a good read.

Glad to hear your insights and it sounds positive

The drum being beat coming out of Docker has just been so heavily kubernates oriented maybe swarm info is getting drowned out. All for multiple orchestrator support.

@olljanat

This comment has been minimized.

Copy link
Contributor

@olljanat olljanat commented Jul 9, 2018

@bitsofinfo Interesting question also is that what kind of stuff you would like to see on Docker Swarm roadmap?

IMO, one recent bit thing was Windows named pipe mount support (moby/moby#33852) which was added to Docker CE on 17.09.0 and with 18.03.1-ee-1 it is also available for enterprise version users.

That makes possible to create Docker management/logging/etc tools which are running inside of container (like logstash on Linux) for WIndows too. I'm currently working on to add that feature support to Portainer portainer/portainer#2018 and adding support for these mounts to service/stack configs too: moby/moby#37400

@bitsofinfo

This comment has been minimized.

Copy link
Author

@bitsofinfo bitsofinfo commented Jul 18, 2018

Nothing in particular I need to see, would just be good for Docker to at least have one that declares what is being worked on w/ swarm, goals, new features etc. The message is so heavy k8 that those of us using swarm are a tad concerned.

@BretFisher

This comment has been minimized.

Copy link

@BretFisher BretFisher commented Jul 18, 2018

If you look at the PR's on SwarmKit all year, they've been focused heavily on bug fixes, improving performance, and improving scalability and networking. While we wait for future blog posts about new features, let's see if we can read from the tea leaves of GitHub PR's for what's happened so far:

After the teams were all focused on K8s integration last year, it looks like lines-of-code on SwarmKit are picking up. You'll notice that they've been crankin' out the PR's on the SwarmKit repo all year, but it's mostly bugfixes, which is good IMO... I don't want new features if existing ones aren't rock solid, but that kind of work doesn't typically make it into a blog post or keynote ;)

libnetwork is a significant part of the SwarmKit functionality (overlay driver) and if you look at any PR's with swarm or overlay in them, it's also quite a lot of quality improvements.

One big focus this year that I was tracking was zero-downtime rolling updates. It was tricky to get all the connections to act properly in that senerio, so even though it's been a feature since SwarmKit's initial release, it wasn't until 12.07+ that you could run docker service update and get anything close to to a zero-downtime deployment. A recent example is this libnetwork PR 2112, which then got merged into docker engine (moby/moby) with 36638, but I could likely find a dozen more bugs over the last year that were squashed like this.

Another big area of improvement is IPAM (IP address management inside overlay networks). The larger your cluster gets with the more containers being allocated, the more edge cases you would hit. Expecially if you deleted and created services/tasks quickly. You can see merged PR's in libnetwork for this effort that also show up in SwarmKit and moby/moby sometimes.

Eventually these changes get rolled into moby/moby PR's, which is how you know they are going into the Docker CE/EE engine at some future date. You can filter to swarm-only or network-only PR's there too. A big merge in June was improving network scability.

@bitsofinfo

This comment has been minimized.

Copy link
Author

@bitsofinfo bitsofinfo commented Aug 30, 2018

@olljanat

This comment has been minimized.

Copy link
Contributor

@olljanat olljanat commented Sep 30, 2018

It is easy to understand that issues/feature request reported by Docker EE customers are on highest priority. Luckily Docker, Inc have made big favor for us and released their technology as open source so community is able to implement also these smaller priority things.

What comes to longstanding issues I have interested to see these ones fixed:

@BretFisher

This comment has been minimized.

Copy link

@BretFisher BretFisher commented Oct 6, 2018

Some updates including a year's worth of "what's changed in swarm" and an interview with the team:

https://www.bretfisher.com/the-future-of-docker-swarm/

@olljanat

This comment has been minimized.

Copy link
Contributor

@olljanat olljanat commented Oct 6, 2018

For us to eventually see a feature in "Docker Engine Release 18.XX," the maintainers first need to add the functionality to the library repos. This means that sometimes, if you see a new feature in SwarmKit/libnetwork, it may take months or more for the feature to surface in the next Docker engine release.

Good point from @BretFisher. As far I know code on swarmkit and moby master branches already contains features which will be released earliest on 19.03.

IMO, best way to see that on which version new features will be released is look these two:

Docker Swarm Team Is Hiring

That is good to hear as it have been visible here that Swarm team have been suffering of lack of resources.

@olljanat

This comment has been minimized.

Copy link
Contributor

@olljanat olljanat commented Dec 15, 2018

I noticed that at least on our environment most of the placement, etc... settings are same for all the services so it would be very useful to be able to set own Swarm wide defaults for them.

So I created proposal that these should be configurable on #2794
those who thinks that it is good idea please go and add 👍 for it.

@olljanat

This comment has been minimized.

Copy link
Contributor

@olljanat olljanat commented May 13, 2019

There looks to be also some discussion/afraid people on https://forums.docker.com/t/afraid-for-the-future-of-swarm/52672 and I also noticed there is now one new item #2852 with roadmap label.

@bitsofinfo

This comment has been minimized.

Copy link
Author

@bitsofinfo bitsofinfo commented Jul 24, 2019

@trajano

This comment has been minimized.

Copy link

@trajano trajano commented Sep 28, 2019

Looking at https://github.com/docker/docker-ce/releases it seems that Swarm items are only bug fixes, but not any new features for a while. So we likely won’t get things like Auto-Scaling in the near future.

The milestone links do not appear to indicate any work on new features. Looking at the branches it seems there's some work on "jobs" but not much detail about it.

@BretFisher

This comment has been minimized.

Copy link

@BretFisher BretFisher commented Sep 29, 2019

https://github.com/docker/docker-ce/releases/tag/v19.03.0 has the list of new features in the July 2019 release. Also, other major features that are in the proposal phase for future releases:

#2682
#2852

@trajano

This comment has been minimized.

Copy link

@trajano trajano commented Sep 29, 2019

I still wish they had a road map that shows what they plan to do for

  • Autoscaling #1663

  • Volumes that are managed by the swarm included with Docker (rather than the plugins which require plugin install on every node)

  • Swarm deployed plugins (would ease the above)

  • Easier node management. Right now it requires a lot of care when you add or remove nodes in the swarm

    docker node demote && docker node update --availablity drain; shutdown the node ; docker node rm

  • Not to mention ghost networks #2737

And as an aside allowing the capability of getting the source IP from Docker for Desktop installations.

@BretFisher

This comment has been minimized.

Copy link

@BretFisher BretFisher commented Sep 29, 2019

Swarm volumes CSI-ish proposal: moby/moby#39624

I don't see autoscaling on anyone's radar. It would require metric collection and there are lots of types of autoscaling (node, replica, resources), which adds to the complexity of a built-in solution. Best to do it with external tools. One way to do it is deploy something like Prometheus and have alerts fire off to do stuff, like https://monitor.dockerflow.com/auto-scaling/

@hlidotbe

This comment has been minimized.

Copy link

@hlidotbe hlidotbe commented Nov 15, 2019

Revisiting this issue in the light of the recent sale of Docker EE to Mirantis. Clearly they announced the switch to K8s and the drop of swarm. I haven't really see any details on Docker side regarding this.

Does that mean swarm is now dead (or at least moribund)? I really do hope it's not the case as swarm fits perfectly our need but k8s is way too complicated for our small team, even with k3s and the like.

@BretFisher

This comment has been minimized.

Copy link

@BretFisher BretFisher commented Nov 15, 2019

SwarmKit and Docker CE are still open source, and stay with Docker Inc (and with the community at large). Mirantis has committed to at least 2 years of Swarm support (blog) and their CEO said publicly (YouTube) that the existing Docker Enterprise customers love it. We'll have to wait and see if Docker and the SwarmKit community re-commit to finishing 2019 features like CSI and Jobs. We talked about this at length yesterday on YouTube Live, and thanks to @dperny and others for showing up in chat to answer questions.

@olljanat

This comment has been minimized.

Copy link
Contributor

@olljanat olljanat commented Nov 15, 2019

@BretFisher thanks for good video/interview (I have not look those earlier). There was lot of good points on discussion and chat which testified my point which I have used many times. Swarm really is choice number 1 for environments which have less containers/simpler environment (I would use thumb rule < 10 000 containers) but it is not part of hype because it is very hard sell anything extra top of it.

@hlidotbe I think that this put bit more demand for all of us on Swarm community. If we decide that we want to keep it running then it will.

@ithinco

This comment has been minimized.

Copy link

@ithinco ithinco commented Dec 7, 2019

@BretFisher Glad to know that it won't be deprecated, I've also wrote a post "A Customer’s voice: what could Docker.Inc provides for developers"

@ggilmore ggilmore mentioned this issue Jan 9, 2020
4 of 5 tasks complete
@stevenfollis

This comment has been minimized.

Copy link

@stevenfollis stevenfollis commented Feb 25, 2020

In the Mirantis will continue to support and develop Docker Swarm blogpost, roadmap items Swarm Jobs and CSI Support are discussed with dates to come at Kubecon EU.

@edijon

This comment has been minimized.

Copy link

@edijon edijon commented Feb 25, 2020

Many thanks for the link, can we assume this issue as solved ?

@bitsofinfo

This comment has been minimized.

Copy link
Author

@bitsofinfo bitsofinfo commented Feb 25, 2020

I'd say no; a blog post and one issue is not a roadmap

@avkonst

This comment has been minimized.

Copy link

@avkonst avkonst commented Mar 6, 2020

In the Mirantis will continue to support and develop Docker Swarm blogpost, roadmap items Swarm Jobs and CSI Support are discussed with dates to come at Kubecon EU.

Have they announced to develop Docker Swarm or Swarm Mode of Docker Engine (which is swarmkit as far as I understand) ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants
You can’t perform that action at this time.