Docker volume create should not be idempotent (by default)

[thaJeztah]

This issue is to discuss the behavior of docker volume create for named volumes. I'd like to have consensus on this before the 1.9 release to prevent awkward, backward-compatibility breaking changes after that.

Currently, docker volume create --name foo will create a volume if it doesn't exist, and will be a "no-op" if a volume with that name already exists.

I think this behavior should be changed for a couple of reasons;

• It's inconsistent with other parts of Docker (e.g. docker run --name foo will fail if container foo already exists)
• It can lead to unexpected behavior; the user does not know if a new volume was created, or an existing (possibly non-related) volume was re-used.
• Given that the Volume API does not keep track of the driver options that were used to create a volume, there is no guarantee that the volume is using the options that the user passed (example below)

Example 1 - using the wrong volume

The user creates volume dbdata, and uses the volume for a container

docker volume create --name dbdata
docker run -d -v dbdata:/var/lib/mysql/ mysql

At a later stage, the user does the same (e.g. for a different project),
leading to the existing volume being used unknowingly; this can lead to data-loss if data in the existing volume is overwritten

docker volume create --name dbdata
docker run -d -v dbdata:/var/lib/postgresql/data/ postgres

Example 2 - using the wrong options

The user creates volume myvol with some options

docker volume create --name myvol --driver foo --opt something=A --opt bar=baz

At a later stage, creating a volume (using different options), will be silently ignored:

docker volume create --name myvol --driver foo --opt something=B --opt baz=example

Note that it's currently not possible to detect that the options are not applied, because docker volume inspect does not return the options that were used to create the volume.

[thaJeztah]

/cc @cpuguy83 @calavera @icecrime just so we are sure we have everything "right" before it's in a release

[cpuguy83]

I'm inclined to leave this be. It is something can be changed if needed.
But if we want to track opts and things we have to store it on disk, which adds a lot of complexity.

In general I'm of the mindset that we should be pushing off the storage management to the storage backend.
We could, instead of just returning the volume we know about issue the create command against the plugin (if and only if it is the same driver) and let it decide if it should error or not.

[thaJeztah]

I'm inclined to leave this be. It is something can be changed if needed.

I think it would be much harder to deny now (i.e., report a conflict if a volume with that name exists), and "allow" it in the future than the other way around.

For now, only checking for the volume name would be (I think) fairly easy to implement; if we decide to include "options" to the equation in future, designing a storage mechanism for those options can be postponed in the future.

Again, I think an error on conflicting names would be consistent with other parts of Docker.

We could, instead of just returning the volume we know about issue the create command against the plugin (if and only if it is the same driver) and let it decide if it should error or not.

This will need modifications to all plugins, and could lead to inconsistent behavior (i.e., 'plugin A' does not error, whereas 'plugin B' does).

[cpuguy83]

There should be no modification of plugins.
It is expected that Create with the same name would yield the same volume.
We don't send create twice today because 1) extra call, and 2) names must be unique anyway

[clintkitson]

@cpuguy83 @thaJeztah

The existing functionality where idempotency is driven into the plugin works out pretty well. While I don't deny that what is being mentioned here would be a move in the right direction, it seems like a breaking change today. I believe to do what is mentioned it would be an updated plugin API and volume driver abstraction for versioning.

Today even the docker run -v is an idempotent call, which could be re-addressed as well.

IMO, docker volume create as is today aligns to existing docker run -v and relevant plugins.

[nazar-pc]

I'm 👍 for this proposal. It is generally a bad idea to rely on such unexpected behavior. It doesn't give any benefits in my opinion, while might cause problems. If user is fine with the case that container might already exist - he should handle such situation explicitly.

[duglin]

I agree with the proposal. As with any good rule there may be exceptions but in general, if I do any kind of docker "create" I expect a new thing to be created or an error generated. Returning me a previously create thing is not what I asked for. If we really want to support "conditional creates" then it should be explicitly asked for via some flag as I think that will not be the norm for most use cases.

[cpuguy83]

Agree -- I think the original functionality was just so that using a volume name on "run" would always give you the same volume, and "volume create" is just following the same code path.
Once we fully push all volume commands to drivers instead of relying on the daemon to manage a list, this is easy.

[nickwilliams-eventbrite]

I agree. docker volume create and the respective client/API calls should fail if a volume already exists with the specified name. I was extremely confused and could not figure out what was going wrong today when it didn't fail on duplicate creates.

[gyliu513]

+1 to report errors when creating a same name volume, this "fail fast" can help a lot for administrating the docker volumes.

[nathanleclaire]

It seems that this functionality has been this way for quite a while and nobody's world is ending, so it's probably not a big deal, but FWIW I would also expect a non-zero exit when a volume already exists, just as trying to docker run a container with the same name does not exit zero.

[msterin]

So what's up with this one? It does create very confusing experience when customers feel the data is lost (when in reality an empty local volume is created on failure to communicate with a plugin) .

[faucct]

Bump. What about hiding the fail-fast behaviour behind a parameter?

[IronSean]

My reverse 2 cents, I was hoping this behaviour would be the case for a CD job to create the volumes if they're missing but no-op otherwise. For things like the docker network create I have to do checks first to see if it exists, but this issue seems to tell me I can just call docker volume create safely on each deploy. It does make my life easier in this case.