Proposal: make project-name persistent.

[thaJeztah]

By default, Compose bases the project name on basename of the directory compose
commands are run from. The project name can be overridden either by
passing a -p / --project-name option for each command or setting the
COMPOSE_PROJECT_NAME environment variable.

Using the --project-name option for each command is error-prone and for-
getting to pass the option when doing (for example) docker-compose up, will result in
another instance of the project being created under a different name or even
containers of a different project being replaced.

Using the COMPOSE_PROJECT_NAME environment variable is not useful when dealing
with multiple projects and can cause similar problems.

Proposal: make project-name persistent

To solve these problems, compose will save the name of a project to a file in the
build-context when the project is first started / built.

If a project-file is found, compose automatically uses the project-name from that
file and throw an exception if the user is trying to override the project-name
using the --project-name option.

File location

To allow further expansion of options, compose creates a hidden .docker-compose directory
in the "root" directory of the build-context. Inside that directory, a
project-name file is created, containing just the name of the project;

tree -a
.
├── .docker-compose
│   └── project-name
└── docker-compose.yml

Request for comment

There are a couple of things left to be discussed;

• Should the configuration-file be created automatically or should this
  be an explicit action by the user (e.g. docker-compose init --project-name=foobar)
• Should a command be added to remove the configuration file(s)? (e.g.
  docker-compose destroy)
• Compose allows specifying an alternative compose-file (--file) should the
  project-name also be applied to those? Should each compose-file get its own
  configuration?

And, in a wider scope;

• Is the project-name actually important? If not, compose could create a random
  project-name and store that inside the configuration. This would greatly reduce
  the risk of conflicting names with other projects running on the same server.

edit: renamed Fig to Compose

[bfirsh]

The project name isn't important. I like the idea of a random project name. Even better – could we generate a hash of the path to fig.yml and use that?

[dnephin]

I actually think the project name is pretty important. If you want to be able to do anything with the images in a CI pipeline, you need to know the project name to be able to retag them as something else.

Being able to override project name from an environment variables makes it easy to keep these unique, and predictable. I think no matter what happens, we need to support the override from environment variables.

I do like the idea of making the project name configurable from a file. I think a similar discussion happened in (#45). Having the project name in.fig/project-name seems like it will lead to a lot of really small files. I think it would be easier to just put it into the fig.yml itself (like it is suggested in #45, and one of the docker compose proposals).

What are the advantages of putting it in a separate directory and file?

[thaJeztah]

The reason for putting it in a separate file has a number of reasons, I'll try to explains my thinking behind this;

• To preserve the name that was used to start the project (i.e., --project-name=foobar), which can be different from the automatic project-name.
• Having multiple instances or versions of a project running on the same server, without them conflicting each other
• Or, in short, the .fig/project-name holds the name of that instance of the project.

Perhaps the file should be called instance-name or similar.

I do like the idea of making the project name configurable from a file

Although possible to do this with this proposal (by manually creating the project-name file), my primary use case was to have fig create the file in order to save the name that was used.

will lead to a lot of really small files.

True, the alternative is to skip the .fig directory, however, if additional things are needed in future, extra file(s) can be added without cluttering the root of your project. Git does it similar and I think it's a clean approach.

I think it would be easier to just put it into the fig.yml

I think both options can be complementary; use the name from fig.yml as the default if the name is not overridden by a flag or env-variable. Put the name that is actually used to start the project inside the .fig/project-name

Being able to override project name from an environment variables makes it easy to keep these unique, and predictable. I think no matter what happens, we need to support the override from environment variables.

Curious; how do you handle managing multiple projects? I.e. cd project-a && fig ps then cd project-b fig <something>?

Thanks for your feedback!

[dnephin]

Curious; how do you handle managing multiple projects?

I usually run fig from python-tox, which lets me set the environment, so I'm never really setting it in my shell. I also tend to use tmux, so I have a different shells for every project.

I think both options can be complementary

Cool, I agree.

I don't know if I'm sold on .fig/instance-name vs say .fig-instance.yml that could contain any instance specific overrides, but I think this is more of a minor point.

[thaJeztah]

Thinking of the project-name; If I understand your situation correctly, it's important to be able to have control over the images being produced, e.g. myapp:build12345, or also names of the containers? Just to get a "feel" for your situation.

My thoughts behind "random" project-names was that, as long as Fig is able to locate the containers for a project, nice-looking names aren't important. If I, as a user, can access a services' container via its service-name (e.g. fig stop web), the name of the actual container doesn't matter.

Even have been thinking that fig could keep track of containers by ID in a local storage inside the .fig directory (which could be a performance gain on machines with a lot of running containers). But that's really something for a separate issue.

Any thoughts on "implicitly" creating the 'project-name' file or "explicitly" via fig init? Perhaps if I have some time during the coming holidays I can experiment a bit (never coded anything in Python, so it will be really experimental :))

[dnephin]

it's important to be able to have control over the images being produced, e.g. myapp:build12345, or also names of the containers?

I guess image is the really important one, but being able to ensure that container names don't conflict is also really important on shared hosts. I see this proposal actually aims to deal with that issue as well.

But I do think that predictable names for containers are still important. I can think of a couple tasks where it's important for external systems (not just fig) to be able to identify a container by name:

• cleanup scripts to remove old containers, being able to identify what project created a container
• debugging/inspecting containers

Both of these are pretty easy right now because I already know what the container name will be. If I have to lookup a name it's somewhat more involved.

Even have been thinking that fig could keep track of containers by ID in a local storage inside the .fig directory

It is true this could be a performance gain, but I worry this is the wrong approach. Adding any persistent state to fig introduces the possibility of lots of bugs (that come along with state). I would much rather see this being handled by a labeling system in dockerd. If fig is able to just label containers and then query for all containers with that label , it keeps all the state in a single place (dockerd). I know there was talk of this at some point, I'm not sure if it's on the roadmap.

Any thoughts on "implicitly" creating the 'project-name' file or "explicitly" via fig init?

I guess implicitly would be more backwards compatible. I would like to avoid a fig init unless absolutely necessary. Instead of using the basedir, I could see it implicitly creating a project name of <basename>-<4 digit random id>

[frank-dspeed]

i can only say if fig starts generating random names now i will switch away from it because i run 1000+ containers on a single host and when i cant identify them by name any more its nothing for me.

[thaJeztah]

@frank-dspeed thanks for adding your use-case. If I understand correctly, you start your containers with fig, but "manage" them directly via Docker after that, making use of the naming convention that Fig uses?

Perhaps this is of interest to you as well: moby/moby#9882 - having meta-data would offer more ways to identify containers, not only their name.

[frank-dspeed]

ah yes i did such proposals a lot for example simply adding new filds and that but i ended in that case with simply adding a ENV and then i have that as costum fild i can parse it 🎯

[funkyfuture]

I think managing the unambiguity of the project name shouldn't be fig's responsibility.

when i understood your proposal correctly, the generation of a random name would render the FIG_PROJECT_VARIABLE and the --project-name-option useless. which is handy for 'templating' services.

[thaJeztah]

I think managing the unambiguity of the project name shouldn't be fig's responsibility.

I'm not so sure; Fig "silently" overwriting another projects' containers because if happened to be in a directory with the same name is sometimes nasty.

when i understood your proposal correctly, the generation of a random name would render the FIG_PROJECT_VARIABLE and the --project-name-option useless. which is handy for 'templating' services.

Following the discussion above, I think something like this would work

• If --project-name is provided, use that (and write/update the .project-name? Not sure)
• No --project-name is provided, but FIG_PROJECT_NAME is, use FIG_PROJECT_NAME (and write/update to .project-name?)
• None of the above, but .project-name is set, use .project-name
• None of the above; create a random name and write it to .project-name

[funkyfuture]

from someone's perspective who uses fig in scripts that take care of a name and pass it to fig, storing a name in the project (which is in my case rather a template) location makes no sense.

and still, i sensed it very intuitive that the directory name is used in the name of the resulting containers when i just ran fig up.
and let say, you look at docker ps -a, that won't be very helpful if it's filled up with random names.

would an option --random-name help in your situation?
beside that i think the ability to prepend some [a-zA-Z0-9_].* to reflect some namespacing would be useful for broader deployments.

[jderusse]

What's about using the fig.yml to store such informations ?

schema-version: 1.1
project_name: foo
containers:
  web:
    build: .
   command: python app.py
    links:
     - db
    ports:
     - "8000:8000"
  db:
    image: postgres

And to keep BC, in compose/project.py::from_config

@classmethod
    def from_config(cls, name, config, client):
        if 'schema-version' not in config:
            config = {
                'schema-version': '1.0',
                'containers': config
            }
        dicts = []
        for service_name, service in list(config.containers.items()):
            if not isinstance(service, dict):
                raise ConfigurationError('Service "%s" doesn\'t have any configuration options. All top level keys in your docker-compose.yml must map to a dictionary of configuration options.' % service_name)
            service['name'] = service_name
            dicts.append(service)
        return cls.from_dicts(name, dicts, client)

[mikehaertl]

@jderusse I'm very much for something like you suggested. Maybe you like to add your proposal to #45 as this is probably the better place to discuss it?

[michaeljs1990]

anyway to come to resolution on this ticket? It looks like the proposed #1233 gives you the best of both worlds by letting the user decide the intended action to take while not breaking any backwards compatibility.

[dnephin]

Originally I wanted the name in the docker-compose.yml, but after thinking about it more, I really like the idea of a separate file.

A separate file give you the option to keep it out of version control if you want (for cases where you want each user to be able to have a different project name).

With the new networking support there is feature request to be able to configure the network name as well (it defaults to the project name). So with a separate config file we could include things like default network name, default scale etc. All the meta-configuration that isn't part of the application definition, but is still relevant to running the composition.

[mikehaertl]

I agree to what dnephin said. How about a file .docker-compose? We could list default options for all command line arguments there.

[thaJeztah]

@dnephin sgtm

@mikehaertl I used a .docker-compose folder in my example, but a file could work as well, depending on how much information we want to store

[mikehaertl]

@thaJeztah Oh, right, sorry. I missed that.

[woodcockjosh]

Maybe we need to organize a real life protest with cardboard signs and all. At least in the US, if project members were elected like public officials, these developers would have been voted out of office in 2016.

[ndeloof]

@woodcockjosh I guess in some of the US states, we could also have be hung or shot by the sherif :P

Kidding apart, I'm not strictly against this proposal, just don't see this as the most common pattern, let's process forward starting by the specification:
compose-spec/compose-spec#206

just consider, while some are Docker Inc employees, most of the people here are maintaining docker-compose on their spare time after "daily job" trying to keep it a pleasant tool.

[rit001]

Well, I've just walked headlong into this issue as I learn docker - I'm working with a docker-compose.yml file but the execution of that file basically creates settings based on the (random) directory that the .yml file is run from. This is all very transparent until the writer of the .yml file starts to do more complicated tasks - such as use traefik. At which point I had to try and understand why things were not working as expected.

Selecting random information such as a directory name if explicit information is not provided, is an incredibly poor design. As there seems to be a thread war on resolving this can I request that we at least have a setting in the .yml file that indicates that a value must be provided by the -p option or the .env file. I request this has I have a role to create docker-compose files, I do not have control over the run script or even the day to day management of the .env file (if needed).

Having entries defined under networks: just change due to external settings/environment is a very poor design choice and traefik makes it worse by just warning that as it can not find the network defined in the .yml it is going to connect to the first available network.

[and3rson]

This ticket will be 7 years old in a week...

Convenience is subjective. However inconveniences that affect many people (298 comments in this ticket so far) might suggest that people really have use cases & arguments that are significant enough for this to be included. Are we going to have any decision on this? Because if this ticket is rejected, forking will become a viable option to develop a version that actual people will use, not just maintainers.

EDIT: I'm writing this comment since I've just shot myself in my leg AGAIN due to this implicit project name guessing mechanism and wasted solid hour guessing why my volumes were not being removed properly. Turns out I had 2 sets of containers with different project names due to different folders.

[CharlieReitzel]

This ticket will be 7 years old in a week...

For the last few years, I have not had occasion to use docker-compose in production. All my container based projects have been deployed either on Amazon ECS or Kubernetes. If starting a new project today, I would recommend Kubernetes+Helm for portability. You can run such a container on your laptop, on-premise or on most commercial clouds.

Kubernetes can be described, fairly, as baroque in its complexity. And Helm adds more complexity of its own. But portability is a big benefit.

I guess I am saying don't bother forking docker-compose. Just use Kubernetes. Just my $0.02 worth.

[and3rson]

@CharlieReitzel using Kubernetes for everything (even 2-container local projects) sounds like buying another house every time you get new furniture.

docker-compose is an industry standard for many people. Honestly, I don't like an option of moving to Kubernetes just because compose has some issues. If only those issues were solved in less than 7 years...

Disclaimer: I worked with Kubernetes for 2 years. I love it. But it's not a replacement for compose.

Edit: congrats on 300th comment!

[CharlieReitzel]

@CharlieReitzel using Kubernetes for everything (even 2-container local projects) sounds like buying another house every time you get new furniture.

Fair point, well made. But for simpler projects in the data center (vs. External/edge deployment), it's unlikely that the app would be running on its own k8s cluster. Prolly just a pod, which is roughly equivalent to a compose file.

docker-compose is an industry standard for many people.

Any legit standard has multiple implementations available and some public process for updating the specification. If that were the case for docker-compose, I doubt this common sense enhancement request would have languished for 7 years.

At this juncture, the committers have made it clear they do not want to make this proposed change. Projects and teams committed to docker-compose, at least for now, should consider using a .env file to set the project name and avoid the associated problems of the default behavior.

Edit: congrats on 300th comment!

Thanks! :--)

[artnaseef]

"Just use a .env file" - that answer doesn't meet everybody's needs. "Just do it my way" feels like a great answer to the person giving it, but lacks consideration and understanding of the problems others face. I wonder at the aggregate numbers of hours of time spent on this problem over the years by everybody impacted. The most frustrating part - this is so simple to fix. No loss of backward compatibilty. Just an additional option, noted in the docs ... and.... done. I've tried to engage on this topic to get feedback as to if/why it's not really that simple and ... silence. One developer offers a patch that does the hard part of the work - it's rejected. This is not a great way to build community and encourage future use of docker-compose.

…

On Mon, Dec 13, 2021 at 6:40 AM Charlie Reitzel ***@***.***> wrote: @CharlieReitzel <https://github.com/CharlieReitzel> using Kubernetes for everything (even 2-container local projects) sounds like buying another house every time you get new furniture. Fair point, well made. But for simpler projects in the data center (vs. External/edge deployment), it's unlikely that the app would be running on its own k8s cluster. Prolly just a pod, which is roughly equivalent to a compose file. docker-compose is an industry standard for many people. Any legit standard has multiple implementations available and some public process for updating the specification. If that were the case for docker-compose, I doubt this common sense enhancement request would have languished for 7 years. At this juncture, the committers have made it clear they do not want to make this proposed change. Projects and teams committed to docker-compose, at least for now, should consider using a .env file to set the project name and avoid the associated problems of the default behavior. Edit: congrats on 300th comment! Thanks! :--) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABCCNZWEUPQNQ2ZS2YQIM3DUQXZVNANCNFSM4AZMCNWA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[arjndr]

This is such a disappointment, can't believe how long ago this issue was opened and it still thrives to this day.

[nathanweeks]

A top-level project name property was added to the Compose spec recently:

compose-spec/compose-spec#206
https://github.com/compose-spec/compose-spec/blob/master/spec.md#name-top-level-element

Hopefully it gets implemented in docker compose soon!

[chuoke]

The first time I saw such a scene. It was a basic and generic feature.

[Hronom]

Yep crazy, now it's closed

[glours]

@chuoke @Hronom what is the problem?
The feature was implemented on the Compose Specification and Compose-Go level, then the version of Compose-Go was upgraded in Compose v2.
So you can now add a project-name to your Compose files

[chuoke]

Just an exclamation. Thank you work very much.

[artnaseef]

Agreed. This is most welcome.

…

On Sun, Mar 6, 2022 at 11:58 PM Chuoke ***@***.***> wrote: Just an exclamation. Thank you work very much. — Reply to this email directly, view it on GitHub <#745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABCCNZVUOYRHP2UX2GWENN3U6WSH3ANCNFSM4AZMCNWA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[henrik242]

So you can now add a project-name to your Compose files

@glours project-name or name? compose-spec says Project name can be set explicitly by top-level name attribute.

Also, does anyone know which version of docker-compose this is coming to? And what schema version that would be?

[ndeloof]

@henrik242
attribute to set is indeed name, see https://github.com/compose-spec/compose-go/blob/master/schema/compose-spec.json#L14-L17
not sure about the minimal required version, but at least v2.3.3 supports this new attribute
You don't need anymore to set a version: xx in your compose.yaml file, this il all obsolete with the Compose Specification

[henrik242]

attribute to set is indeed name

Thanks!

not sure about the minimal required version, but at least v2.3.3 supports this new attribute

Ah, I'm on the latest Docker Desktop (4.5.0) and it's on Compose v2.2.3. Guess I'll have to wait (or install docker-compose separately)

You don't need anymore to set a version: xx in your compose.yaml file, this il all obsolete with the Compose Specification

Great to know, thanks again :)

[johnthagen]

@ndeloof

attribute to set is indeed name, see https://github.com/compose-spec/compose-go/blob/master/schema/compose-spec.json#L14-L17

Should this be documented at https://docs.docker.com/compose/compose-file/? I had to hunt back for this issue to find how to set the project name.

[ndeloof]

https://docs.docker.com/compose/compose-file/ is not (yet) kept in sync with this repository, need to discuss with documentation team to get this automated

[glours]

We started part of the sync work with 2 PRs, we need to include a sync with the compose-spec upstream repo
#9296
and docker/docs#14457

[nicolasmafraintelipost]

Now "name" is a valid field inside docker compose file, but it's not working. Bug opened: #9530