Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The default naming scheme for containers created by Compose #6316

Closed
j4r3kb opened this issue Nov 2, 2018 · 52 comments · Fixed by #6377
Closed

The default naming scheme for containers created by Compose #6316

j4r3kb opened this issue Nov 2, 2018 · 52 comments · Fixed by #6377
Milestone

Comments

@j4r3kb
Copy link

j4r3kb commented Nov 2, 2018

The default naming scheme for containers created by Compose in this version
has changed from <project>_<service>_<index> to
<project>_<service>_<index>_<slug>

Is there any way to disable this behavior apart from using container_name in yaml file?
We have many scripts that rely on container names and are not using swarm, just a single container stack. This change is very inconvenient for us.

@shin-
Copy link

shin- commented Nov 2, 2018

Unfortunately no, sorry.

@ramonmoraes8080
Copy link

Same thing here. This slug feature should be optional. How hard it's to pass a flag on the up or build command to turn it off?

@theoturner
Copy link

I agree - this should be optional. Several programs making use of Docker Compose now only run if you downgrade to 1.22.

@crossdot
Copy link

crossdot commented Nov 6, 2018

I agree, this should be optional. We don't need this slug feature at all and a lot of bash-scripts are using old naming format.

@milan-janecek
Copy link

Same for us, we must stay on 1.22 as we use "volumes from" feature and rely on old naming scheme. Please, make it optional feature in 1.24.

@guglielmo
Copy link

This feature makes it difficult to refer to the containers after they've been upped.

@alex-lion
Copy link

I don't get the point of this change. Currently neither docker nor compose offer any service discovery. So, if I want to get a hostname from inside the container, what am I supposed to do? Roll my own service discovery?

BTW, what are benefits of this change? Is it really worth breaking backward compatibility?

@ryanwoodsmall
Copy link

ryanwoodsmall commented Nov 8, 2018

The default _slug name change is the correct approach, but not having a way to disable it via an environment variable or command line option breaks a lot of existing workflows. Please consider this a feature request to add a flag of some sort to bring back <= 1.22.0 docker-compose naming behavior.

@jamct
Copy link

jamct commented Nov 9, 2018

Sorry, but "external_links" are unusable now. I have to know the name of a container to link it.
+1 for making it optional

@lig
Copy link

lig commented Nov 9, 2018

@shin- this new behavior completely ruins a lot. Before that random slug it was possible to address a container launched from one docker-compose file inside another file like

services:
  app:
    image: app
    external_links:
      - "other-project_other-app_1:other-app"
networks:
  default:
    external:
      name: other-project_default

And that's it. Now, this won't work at all.

How this thing is not optional at least?

@lig

This comment has been minimized.

@lig
Copy link

lig commented Nov 9, 2018

So, there is a workaround. container_name parameter is used as is and slug is not used on it. At least, that helps.

@shin- please, don't consider this a bug report about slug is not being used with container_name. Leave it as it is. I'm literally bagging here.

@lig

This comment has been minimized.

@achekulaev
Copy link

While I understand the incentive behind the change, it is highly disrupting, especially since there is no grace period to at least give developers time to adjust their script. Lots of scripts were written with this naming convention in mind which existed literally forever. Basically this change makes docker-compose 1.23 non-backwards compatible with any other docker-compose.

@lig
Copy link

lig commented Nov 14, 2018

@shin- You personally and all the docker-compose and docker team should learn already what backward incompatible change is and how to bring it to a widely adopted project the entire industry relies on.

First, backward incompatible change is not about you breaking what you've guaranteed earlier to not to. Nobody cares if this thing is used by nobody.

Backward incompatible change is when you break something your user base actually rely on. And it doesn't matter what guarantees were because it's Apache 2.0 after all and the entire project provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND. Your users decide what they rely on. And you (the entire team) should start to learn how to be aware who uses your project, why, and how.

Second, you should learn how to introduce such a change to your user base. This "random suffix thing" definitely breaks the way your users use external_links and volumes_from. So, the deprecation warning in case there is no explicit container_name set or when it looks like so would be highly appreciated. Please, consider these documents to be familiar with "what others do":

Third, @shin- , you personally should learn how to talk to people who are not direct contributors to your project and who are just your users but at the same time they are very busy people who are loyal to your project enough to invest their precious time filing requests and participating in discussions here with the only hope to bring some sense to the project future development and to try to avoid investing more resources in the migration away to another solution.

Fourth, Docker team is trying to make money out of the Docker but it is not about Docker itself. The only way Docker can be a good thing to pay for is if the entire Docker infrastructure proofs it's worth it to pay. I don't see how it is a product oriented to user success if the development team turns its back to the community most of the time.

Fifth, once again, we all here are your colleges and friends rather than enemies and we are really trying to help you with all this. And because we are your friends the exodus from the docker stack will be painful but in the way, it goes now, that farewell doesn't look unrealistic at all.

@ShotSkydiver
Copy link

This is a terrible implementation of a major version-breaking change, it completely breaks the ability to deploy our framework and I cannot even imagine how many others will experience the same exact issue, with absolutely zero advance warning; unless someone manually goes looking for the latest changelogs for docker, they'd have no idea about this, and I imagine most people won't do that

@slaughter550
Copy link

@shin- This change is highly disruptive to existing workflows and there should be a flag provided to not generate slugs. Thousands of hours of development time has gone into writing scripts and automating services which rely on docker-compose's predictability. Without proper service discovery, what is the team's plan for allowing automated targeting of individual containers?

@mauricioabreu
Copy link

mauricioabreu commented Nov 14, 2018

I tried to use docker-compose container_name directive but it did not take effect on GitLab.
But it looks that on my latest version of docker-compose on Mac Os Mojave it is working. Not sure what is happening :)

My .gitlabci.yml file is downloading the last image from tmaier (it should be 1.23.1).

@shin-
Copy link

shin- commented Nov 15, 2018

Hey all, a few general comments that I hope address some of the concern expressed here:

  • I definitely understand that this breaks some scripts. Our release notes since 1.23.0-rc1 say so at the very top. This is very much a "ripping the bandaid off" type of situation where the momentary pain helps us moving forward with a healthier base.
  • The benefits of the current solution are numerous and expressed in part in this long-standing issue: Proposal: Stop using sequential container numbers #1516 - and the issue with docker-compose run especially has been reported several times here: service_run_+1 #4688 Parallel execution #4549 Running docker-compose run from different shell scripts in same folder can lead to race condition leading to error & abort #5240
  • Service discovery shouldn't be an issue as it should be handled using service names (rather than container names) and network aliases, which are static on their associated network. You may read the networking docs for more details on that specifically
  • I reckon that this makes volumes_from and external_links across Compose projects more difficult to use. Please consider that even before this change, there was no guarantee that Compose would respect the "expected" format for a given container name (see e.g. Exec Fails If Container #1 Does Not Exist #6155 or the prefixing that is mentioned in Only prefix container name once during recreate #3415), making it a flawed solution prone to run into obscure issues in the long run.
    • The recommended way to allow containers from different projects to communicate is for them to share an external network and use the other service's alias(es). Similarly, volumes_from across projects should instead leverage external named volumes.
  • I'm interested in suggestions on how this change could have been communicated better. For reference, the change has been called out in our release notes and available for testing since 1.23.0-rc1, which was released September 26. 1.23.0 GA was released more than a month later, and knowing the change would be controversial, we put it as a big warning at the top of the changelog. If there's anything more I could have done to make this change visible, I'm happy to listen and do these things the next time we consider a risky change like this one.
    • On the flipside, if the general takeaway here is "don't make any breaking change ever, or let me opt out of all of them indefinitely", I'm sure most of you realize that it's not a healthy, reasonable way to go about maintaining a software project the size of Compose, which already jumps through a lot of hoops to maintain backward-compatibility with a wide array of Docker Engine versions, Compose file formats, and deprecated idioms.

Let me know if there's anything that isn't addressed here. I understand that this is a major speed bump for a lot of you and tempers may flare when we're dealing with unforeseen circumstances. Take the time you need to go through the upgrade and pin your Compose version for the time being. Happy to help with questions like "How do I do XYZ without predictable names?" until then, in this thread or on Slack. And please be considerate in the way you interact with others on these forums, double-check that the information you're sharing is correct (I've already seen a couple of threads getting mentioned or redirected there that had nothing to do with this change, which creates an unnecessary sense of alarm and doesn't help the people with the problem who are getting misdirected here), and don't derail the conversation. Thank you for your time and patience.

@achekulaev
Copy link

achekulaev commented Nov 15, 2018

On the communications part, I would expect a change like this to be communicated and explained through some communications channel beforehand, basically in the way you just did it here in this thread, explaining why, and what to do for now, but doing it before all the aftermath.

@lig
Copy link

lig commented Nov 15, 2018

@shin- You haven't read my last comment and links I've provided, have you?

The last one from you looks kind but feels the same.

I'm interested in suggestions on how this change could have been communicated better.

TL;DR

  • Introduce the new version of the docker-compose.yml
  • Add DeprecationWarning for external_links with a link to a document which describes the update and provides a sample solution for those willing to migrate.
  • Support the old behavior for all compose file format versions prior the new one.

For reference, the change has been called out in our release notes

The changelog is a thing for contributors. Users don't read it. A user installs docker-compose and prays it still works after yesterday.

On the flipside, if the general takeaway here is "don't make any breaking change ever, or let me opt out of all of them indefinitely", I'm sure most of you realize that it's not a healthy, reasonable way to go about maintaining a software project the size of Compose, which already jumps through a lot of hoops to maintain backward-compatibility with a wide array of Docker Engine versions, Compose file formats, and deprecated idioms.

I have a good news for you. You already have guaranteed "don't make any breaking change ever, or let me opt out of all of them indefinitely". And more, you have a feature for this. It's the version in docker-compose.yml file. So, consider reverting this change ASAP and introducing it for the next version number.

Happy to help with questions like "How do I do XYZ without predictable names?" until then, in this thread or on Slack.

Please, provide the solution for external_links and external volumes without container_name defined (as we see it doesn't always work) in another docker-compose.yml in this thread.

This is a popular way your users use your project. Most of your users use docker-compose for the local development. Often one has several interconnected projects in development. In such a case it is a common practice to direct several docker-compose files to the same network and to define external_links for services from different projects to be able to reach each other on the developer's machine.

@lig

This comment has been minimized.

@shin-
Copy link

shin- commented Nov 15, 2018

Please, provide the solution for external_links and external volumes without container_name defined (as we see it doesn't always work) in another docker-compose.yml in this thread.

Already did, please refer to my earlier comment:

The recommended way to allow containers from different projects to communicate is for them to share an external network and use the other service's alias(es). Similarly, volumes_from across projects should instead leverage external named volumes.

The rest of your post is needlessly adversarial and I won't respond to it. I'm here and willing to have a discussion, but I don't have to take your bullying.

@er1z

This comment has been minimized.

@jonnrb
Copy link

jonnrb commented Nov 26, 2018

s/disabled/enabled/. At least on existing compose file versions.

This behavior needs to be tied to the compose file’s version number. Regardless of whether this behavior was “specified”, it’s pretty widely used, and therefore is a backwards-incompatible change. Why version the spec at that point anyway if what a compose file does isn’t guaranteed to work the same way when I upgrade my binary? Deprecating a compose file spec (with plenty of warning when docker-compose is run of course) and then removing it, it would definitely be my fault all of my stuff is no longer working, but expecting me to read through your release notes for a program that has the privilege of printing me warning messages at my face?

Making changes like this seems like pretty good motivation for versioning the spec’s behavior in addition to its format: you can deprecate others’ code’s assumptions with warning. I think the amount of external issues name-dropping this issue speaks to how surprising this was to a lot of people. It’s real great to see legacy behavior being smashed so that code doesn’t get complicated, but changes like that cannot be applied all at once. Can I expect other assumptions about compose container names to break? Will the service name always be in the container name or can that be removed like the predictable names?

This really reminds me of when Consistent Network Device Naming received widespread adoption in Linux (but this is the opposite thing wrt naming... idk). A lot of code was hardcoded for ethN (and still is in some surprising places) and that hasn’t really changed. There is however a way to get back the old behavior on systemd systems (and on other systems, but it can vary) by adding net.ifnames=0 to the kernel args.

(Sorry for the second-pass rant, but I don’t think my first one was very constructive.)

stmh added a commit to factorial-io/phabalicious that referenced this issue Nov 26, 2018
Needed because of change in recent versions of docker-compose, more info here: docker/compose#6316
@skors
Copy link

skors commented Nov 26, 2018

It seems this change makes the --scale option useless. because the random postfix must be known to reach scaled instances of a service.

For example, multiple service instances as upstream servers for Nginx.

Edit: see #6365 for further informations

@shin- shin- added this to the 1.24.0 milestone Nov 26, 2018
@dawnerd
Copy link

dawnerd commented Nov 26, 2018

Wow, what an awful change. 👎

How are we supposed to get the randomly generated string for use in scripts?

@saramburke
Copy link

I ran into this issue as well and would like to express my recommendation for future backwards incompatible changes. We too have scripts relying on the project_service_index format, with many people using those scripts on Mac. In an ideal world, we would be able to control the version of Docker for Mac that people use, but for now people upgrade when the auto-upgrade dialog appears.

My issues and recommendations are:

  1. The upgrade dialog had no indication of this significant change, so we had no obvious heads up on this change. In an ideal world we would check all the release notes for each upgrade, but we don’t at the moment. So, it would be greatly appreciated if something significant like this was called on in the update dialog explicitly.

  2. Due to 1), there was no obvious warning. It would be great if a change like this was called out in the previous upgrade. For example, the upgrade before the latest would say something like “container naming scheme is changing in the next version, please upgrade your scripts”

  3. I understand that, after reading this thread, the naming scheme we are using isn’t guaranteed, and I realize there are better ways we can be referring to containers. However, we all have busy work lives and need to plan out our tasks, so as the maintainer of our scripts, I’m not able to convert our use of container names to something better right away. I am happy to use the recommended best practice, but need time to migrate our code. Therefore, it would be great to have a deprecation strategy for a change like this.

The key takeaway here is most of the world assumes a container naming scheme that is fundamental to their use of docker compose, and changing the default behavior without an obvious deprecation strategy can be detrimental to these workflows.

People don’t always use software exactly the way it was intended, and the ownership is on those people to fix things when their assumptions fail. However, a few simple communications can help us prepare for future changes and would help motivate us to move to the latest best practices.

@jtreminio
Copy link

jtreminio commented Nov 27, 2018

If you previously depended on docker container exec -it project_php_1 bash, you can no longer use this.

You have to use docker ps to find the service ID.

However, you can't use docker ps -q --filter=name=project_php because that'll match on services named project_php and project_php_xdebug or anything else that left-matches project_php.

The previously-readable docker container exec -it project_php_1 bash now have to become this:

docker container exec -it \
    $(docker ps -q \
        --filter label=com.docker.compose.project=project \
        --filter label=com.docker.compose.service=php) \
    bash

This was a poorly thought-out change.

@jonnrb
Copy link

jonnrb commented Nov 27, 2018

@jtreminio FWIW, from the project you can still do docker-compose exec php

@shin-
Copy link

shin- commented Nov 29, 2018

Thanks to everyone who took time to share their thoughts on the change. We agree this was poorly handled and a bit overzealous on our part, and it has been partially reverted in Compose 1.23.2 (we're keeping random suffixes for containers created by docker-compose run, which lets us handle these more gracefully: #4688 #4549 #5240, as was initially intended).

Let me know if there are any outstanding issues that need to be addressed.

@rwlogel
Copy link

rwlogel commented Dec 6, 2018

Is there any plan to add either a command line option to turn this off --no-slug or more preferably --slug so that the default is to work like it used to.

@NicolasDorier
Copy link

1.23.2 revert to how it was before.

@rwlogel
Copy link

rwlogel commented Dec 7, 2018

It was only reverted for docker-compose up but not for docker-compose run

@saramburke
Copy link

saramburke commented Dec 7, 2018 via email

@jcrben
Copy link

jcrben commented Feb 3, 2019

@shin- I presume that this will be returned at some point - possibly in 1.24 (since it is such a breaking change?). If so, could you work with the community to understand and recommend alternative lightweight mechanisms for people who were using these without slugs? Not sure about the best place to have that conversation.

@shin-
Copy link

shin- commented Feb 4, 2019

#6316 (comment)

@abdennour
Copy link

changed from <project>_<service>_<index> to <project>_<service>_<index>_<slug>

This is a bug not a feature.
Thanks!
Think about ansible playbooks that are running with ansible_connection=docker... 😔!!
should we give explicit container_name ? or shall we keep updating our inventory with the random <slug> !!.
Really Very bad!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.